HackMD
Samba Debugging
Always run testparm
Maybe one of the most important tools we have. The testparm utility checks if the the smb.conf is valid. Run
# testparm -s
If somone provided you with a smb.conf file use:
# testparm -s /path/to/smb.conf
If modifications are done to a smb.conf file you should always run testparm to verify it is correct!
Also if you're repoting a bug please always share the output of testpam -s
How to collect logs
We use winbind as an example here:
# systemd stop winbind.service
Add the following options to the smb.conf:
log level = 10
debug pid = true
max log size = 0
winbind debug traceid = yes
Log level 10 gives a lot of information but if we want to see network packet dumps, we can use log level 50. This also dumps request/response packets to /tmp. To only increase the logs for certain debug classes you can do the following:
log level = 1 winbind:10
This only sets log level 10 for winbind processes. Better get full logs if you're not sure what you're doing.
After rasing the debug level remove old logs and start winbind again:
# rm -f /var/log/samba/log.*
# systemd start winbind.service
Reproduce the issue, for example:
# date; wbinfo --name-to-sid <name>; date
NOTE: The date command is important because it will print the timestamp when the command got executed. It makes it a lot easier for a developer to find the related lines in the log files! This is an extreme time saver.
Collect log files from /var/log/samba/log.*
Winbind in an AD Environment
Check DNS
For Active Directory, a correctly set up DNS infrastructure is essential! Active Directory uses Kerberos and has a fallback to NTLM. If the DNS setup doesn't work it will always fallback to NTLM increasing the load on DCs. Often customers don't notice that DNS is not set up probably.
A client normally should be able to reach its DC server via DNS and be able to lookup SRV records to find the Kerberos KDC.
Get information about current domain
Winbind talks and gets all the information it has been joined to. That's the domain or realm the machine account has been created. You can get some information about the DC it is joined to with the following command:
# net primarytrust dumpinfo
Get information about trusts
This the following command will list the trusted domain and the type of trust and direction the domain has winbind is joined to:
# wbinfo --trusted-domains --verbose
How to find out in which groups a user is member of
This will not be able to produce a correct membership information:
root:~ # id <user>
Log in as the domain user and run:
DOMAIN\user:~ # id
The above is what you should teach a customer!
For debugging you can authenticate a user also as root with either NTLM (wbinfo -a) or Kerberos (wbinfo -K):
Example:
root:~ # wbfino -a DOMAIN\user
The information are now cached in the samlogon cache. You can inspect the cache using:
net cache samlogon list
net cache samlogon show <sid>
Another way is using S4U2Self impersonation using the machine account credentials.
net ads kerberos pac dump impersonate=user@REALM -P
S4U2Self is a privileged information. Often machine accounts lack permissions on the AD DC side to perform it, so the impersonation might fail.
Samba File Server
Enable debug logs on the fly
smbcontrol <pid> debug 10
Samba Pool Usage
Samba use talloc as it's memory allocater. This is a hierarchy-based allocator with destructors.
As of this we are able to dump the memory hierachy with the following command:
smbcontrol smbd pool-usage
If you experience a memory leak in a daemon you can use the following to track it down:
smbcontrol <pid> pool-usage > start-pool-usage.log
wait till the memory increases and then do:
smbcontrol <pid> pool-usage > current-pool-usage.log
diff -u start-pool-usage.log current-pool-usage.log
This works wit smbd and winbindd!
Valgrind
valgrind --tool=memcheck -v --num-callers=20 --track-origins=yes --leak-check=full --log-file=valgrind.winbind.log /usr/sbin/winbindd --foreground
Network traces
If you're requrested to create network traces make sure they match to the Samba log files you provide. Always make sure to capture only one problem per network trace file. This makes it easier to understand the problem.
You can use tcpdump to create the network traces using the following command:
tcpdump -n -i eth0 -s 0 -w samba-problem-description.pcap
Limiting to an IP address
Network traces should just capture which is going on between client(s) and server. So limiting the captures makes it easier to process the huge amount of information. You can do that using the ip of the host you're connecting too or which is connecting to the server:
tcpdump -n -i eth0 -s 0 -w samba-problem-description.pcap host <ip>
SMB client traffic
To limit the traffic only to smb client traffic, use the following command:
tcpdump -n -i eth0 -s 0 -w smb.pcap host <ip> and port 445
