Memo of Handy Commands
It happens from time to time that at some point it is hard to recall the familiar commands used before, meanwhile it is somewhat annoying and time consuming to find them again. As the old saying goes, “the palest ink is better than the best memory”. Instead of relying on muscle memory, this post serves as a growing memo for those handy commands.
Network Configuration
iproute2 is designed to be a more organized tool chain to replace net-tools which consists of /sbin/ifconfig
and so on. At the core of iproute2 is /sbin/ip
.
ip help
,ip neigh help
to view helping info at different levelsip addr
to show all addresses related infoip -s link
to view statistics (RX/TX pkts, dropped, bytes etc) of all interfacesip link set enp101s0f0 up
orifconfig enp101s0f0 up
to bring the interface upip link set enp101s0f0 down
: bring the interface downip link set enp101s0f0 promisc on
: set the interface in promisc modeip addr add 10.1.1.3/8 dev enp101s0f1
: assign ip to the device interfaceip route get 192.168.88.77
: query the route for the specified ipip route add 10.1.1.0/24 dev enp101s0f1
: add new route entryip route del default
andip route add default [via 192.168.1.1] dev enp101s0f1
to change the default gatewayip neigh
orip n
as a replacement ofarp -a
to view the arp table-
ip n add 10.1.1.2 lladdr ec:0d:9a:6d:e1:6d dev enp101s0f0 nud perm
as a replacement ofarp -s 10.1.1.2 ec:0d:9a:6d:e1:6d
to set a permanant arp entry sudo ethtool enp101s0f1
prints the speed of the interface-
sudo ethtool -i enp101s0f1
shows the driver version of the interface traceroute <ip/url>
is useful to gain knowledge of the network topology
iptables
is an interface to operate on top of netfilter
.
iptables
defines chains as when rules will be evaluated, including PREROUTING, INPUT, FORWARD, OUTPUT, POSTROUTING
, mirroring the corresponding hooks in netfilter
(NF_IP_PRE_ROUTING, NF_IP_LOCAL_IN, NF_IP_FORWARD, NF_IP_LOCAL_OUT, NF_IP_POST_ROUTING
).
Example usage:
iptables [-t nat] -L [-v]
lists current rules (-v to show verbose details) for the specified table.iptables
table consists of filter (default), nat, and mangle.iptables -D OUTPUT 1 -t nat
deletes the first rule (1-indexing) rule from table nat, chain OUTPUTiptables -t nat -A POSTROUTING -s 192.168.1.2 -o eno1 -j SNAT --to-source 158.130.4.222
configures a server A with internet connection to uses SNAT protocol to modify all outbound packets’ source ip (from priv to pub) so that packets from its cable connected server B (without internet access) can reach the internet/etc/resolv.conf
configures the DNS resolver, e.g., a linenameserver 192.168.1.1
in the file will ask the resolver to query for the name server with the specified IP.
Network Traffic Monitoring and Manipulation
tshark -s 64 -B 1024 -i enp101s0f1 > tmp
: capture high-speed traffic (with 64B cap of packet size and 1024MB buffer) at the specified interface and redirect the outputs for future analysistshark -c 100 -i enp101s0f1
: stop capture after specified number of pkt count, useful when only a small snapshot is needed- tshark
-Y
option allows an expressive specification of the filter, example filters:tshark -Y 'icmp'
tshark -Y ip.addr == 10.0.0.1
tcp.dstport != 80
ip.addr == 8.8.8.8 and !(icmp or dns)
- tshark
-V
option shows verbose pkt output (including all layers) tshark -i enp5s0 -Tfields -E separator=, -e eth.dst -e eth.src
prints only the dst and src address delimitered by “,”tshark -i ens1 -Y "eth.type == 0x4321" -Tfields -E separator=, -e eth.dst -e eth.src
similar to above, but filters the ether types being non 0x4321
As a prerequisite for installing DPDK and pktgen, Mellanox NIC Linux driver installation steps:
lspci -v | grep Mellanox
verifies the existence of Mellanox network adapter- Install a compatible mellanox NIC driver version, e.g., MLNX_OFED_LINUX-4.9-0.1.7.0-ubuntu18.04-x86_64.tgz for ConnectX-4, Ubuntu 18.04 LTS, arch x86.
md5sum <file>
before the actual installation - Verification
/etc/init.d/openibd restart
loads/restarts the driver- Double check the driver version installed using MST (Mellanox Software Tools):
mst start; ofed_info -s
- Check that Mellanox kernel modules are loaded:
lsmod | grep mlx
ibdev2netdev -v
maps the adapter port to the net device
Useful tips for .pcap
manipulation:
editcap -F pcap -i 60 bigfile.pcap tmp.pcap
splitsbigfile.pcap
into 60s chunkstshark [-V] -r bigfile.pcap -Y "frame.time_relative <= 1.0"
reads 1s chunk ofbigfile.pcap
tcpdump -r bigfile.pcap -c <num> -w firstnum.pcap
saves firstpackets tshark -r bigfile.pcap -Y "frame.time_relative <= 60.0" -w tmp.pcap
saves first 1min packetstshark -r <original.pcap> -Y "eth.type == 0x0800" -w clean.pcap
focuses on ether type 0x0800tshark -r 1s_chunk.pcap -Y "ip.proto == 6 or ip.proto == 17" -w tcpudp_1s_chunk.pcap
filters non TCP, UDP packetscapinfos -Trc 1s_chunk.pcap
prints number of packets in the pcaptcprewrite --dlt=enet --enet-dmac=00:11:22:33:44:55 --enet-smac=66:77:88:99:AA:BB --infile=<chunk_file> --outfile=output.pcap
adds fake Ether headertcpreplay -i enp101s0f0 --pps 450000 -T nano -K --preload-pcap -l 1 test.pcap
replaystest.pcap
at 450000 pkt/s with loop number 1tcpreplay -i enp101s0f0 -x 2.5 -T nano -K --preload-pcap -l 1 test.pcap
replaystest.pcap
with 2.5 times of the capture speed (not that it shouldn’t coexist with –pps or –mbps)
iperf
is a handy command line tool to launch TCP/UDP server/client pairs.
- Configure the interface ip, arp/route table etc a priori for the client/server pair(s)
iperf -s -i 0.5
launches an iperf server with 0.5s report intervaliperf -c 10.1.1.3 -t 10 -P 16
launches an iperf client targeting server 10.1.1.3 transmitting 10s with 16 parallel client threads
File Management
find / -name "libpython3.4m.so.*"
finds recursively the file with the specified name regex starting from the file system rootgrep -r "bf_tm_ppg_hdl" . -I
fines the files with the specified content, skipping binary filestar -xvzf <resource tarball>.tar.gz
is commonly used to unzip a tar.gz filefdisk -l
views all disk partitionssudo dd if=<.iso file path> of=/dev/sdb1
creates a bootable image on a USB device partition (which appears to be file/dir in Linux) on Ubuntu.mount /dev/sdb1 /mnt
mounts the file system (dev/dsb1
, can also be in the form of .iso etc) to the mount point/mnt
(existing dir),umount /mnt
is the reverse operation to cancel the mount pointmount [-l]
lists all mounted file systemssshfs <myuser>@<ip>:<remote dir path> <local dir (mount point)>
mounts the remote file system
Shell Execution
sudo -E ./script.sh
to launch the script with sudo while keeping the current env variables, one could compare the output amongenv
,sudo env
, andsudo -E env
env
prints the env vars of the current envenv [<env var name>=<value>] [<command> [<args>]]
enables the executation under a modified env where the specified env vars are replaced with the new values, the rest remains. E.g, the only diff ofenv USER=test env
output versus originalenv
is the USER env var value. We could also manipuatePATH
env var to the path containing a specific version of a prog without modifying the global env (while the defaultPATH
prioritizes the path of another version).
Compilation/Linkling
ldd <executable>
prints the shared objects/libraries dependencies (with specific version) of the program, useful when debuggingcannot open shared object file: No such file or directory
errors. Note that often there is only one actual .so filelibX.so.0.0
while others such aslibX.so
,libX.so.0
are just symlinks tolibX.so.0.0
, i.e., only the latest version is used by default.LD_LIBRARY_PATH
env var specifies a set of dirs to search for shared libraries by the dynamic linker during run time rather than compilation time. E.g.,sudo env LD_LIBRARY_PATH=/path1:/path2 ./prog
will enables the dynamic linker to search path1 and path2 for the shared libraries needed by the executable prog.
Hotkey
Ctrl-Alt-F1
: the terminal prompt displayCtrl-Alt-Del
: reboot
Kernel
uname -r
prints the current kernel versionlsmod
prints the current kernel modulessudo insmod kdrv.ko
to insert a kernel module andrmmod
is the opposite operation
dmesg
is a powerful tool to debug by monitoring the kernel ring buffer:
dmesg -T
prints the ring buffer with human readable time (rather than default time showing the time since the machine boot)dmesg -x
appends the message level (info, err, debug, warn…)dmesg --level=err,warn
prints only the message with the specified leveldmesg
output is often combined with pipes, e.g.,dmesg | head -30
prints the first 30 lines of dmesg original output, similarlydmesg | tail -20
prints the last 20 lines of the output,dmesg | grep -i dma
prints messages with dma substring (case insensitive with-i
option)
Authentication
ssh-keygen -t rsa [-b 4096]
generates an RSA public/private key pairssh-keygen -l -f ~/.ssh/leoyu.pub
prints the number of bits of the specified key
RSA is similar to Diffie–Hellman key exchange in that they both leverage the property of the one-way function. RSA secretly picks 2 distinct prime numbers p, q and releases the modulus \(n=pq\). The public exponent e and private exponent d are chosen such that \(ed\equiv 1 \mod \phi(n)\) where \(\phi(n)=N-p-q+1=(q-1)(p-1)\) denotes the Euler’s totient function. The magic here is that \((m^{e})^{d}\mod n = m^{k\times \phi(n)+1}\mod n=(m^{\phi(n)})^{k}\times m\mod n\) and with Euler’s theorem, \((m^{e})^{d}\equiv m \mod n\). One could also show such equivalence using Fermat’s little theorem. This indicates that by applying both encryption and decryption (order doesn’t matter), we would recover the original message \(m\).
ssh -i <privkey path> <user>@<server ip>
to ssh with the specified private key when it is absent in ssh-agentssh-add -l
prints the currently added private keys,ssh-add ~/.ssh/id_rsa
adds the specified private key to ssh-agent (needs appropriate access permissions, e.d., 600), andssh-add -d ~/.ssh/id_rsa
deletes an added identity-
passwd [user name]
to update the password for the current/specified user openssl genrsa [-aes128|-aes192|-aes256] -out private.pem 512
generates a 1024b private key (with optional AES encryption in CBC mode of PEM output)openssl rsa -in private.pem -pubout > public.pem
derives the corresponding public key for the specific private keyopenssl rsa -in private.pem -text -noout
displays the modulus and the private exponentopenssl dgst -sha1 -sign private.pem -out sha1.sign msg.txt
signs the specified file using SHA-1 digest algorithm using PKCS1v1.5, one could read the signaturehexdump sha1.sign
openssl dgst -sha1 -verify public.pem -signature sha1.sign msg.txt
verifies the signature
Signature generation workflow under the hood:
- A cryptographic-hash function generates a digest (160 bit hash with SHA-1) for the message
alice says hello to bob
. E.g., runshasum -a 1 msg.txt
on MAC orsha1sum msg.txt
on Linux to generates SHA-1 digestf669524891e50a82cb6e355af792b0a8f6351241
for the message (cut -d ' ' -f1
).- The digest is padded to RSA key size
k
bits w.r.t. a padding scheme (why? because it is deterministic otherwise), e.g., PKCS1v1.5 or a latest version PKCS1v2.2. In PKCS1v1.5,00||01||PS||00||T||MD
where01
is the mode for digital signature,PS
is a variable number ofFF
bytes (\(k/8-38\)),T
is a a string of magic ASN.1 (nothing but a specification to describe data structure, example below) bytes specifying which digest mechnism to use, for SHA-1, it is3021300906052b0e03021a05000414
(see P46 of PKCS1v2.2), andMD
is the previously generated digest.DigestInfo ::= SEQUENCE { digestAlgorithm DigestAlgorithmIdentifier, digest Digest }
- To summarize, the padded message in the example above is
0001ffffffffffffffffffffffffffffffffffffffffffffffffffff003021300906052b0e03021a05000414f669524891e50a82cb6e355af792b0a8f6351241
- Now we can sign the padded msg via \(c=(PKCS1V1\_5(MD))^{d}\mod n\) leveraging Python’s big number arithmetic. One could verify that the resulting sign should be identical to sha1.sign using openssl direcly.
md_padded = int('0001ffffffffffffffffffffffffffffffffffffffffffffffffffff003021300906052b0e03021a05000414f669524891e50a82cb6e355af792b0a8f6351241', 16) n = int('00bb78017b6de556689dfd97f737003ab49fc5e463a73f02d6adf6cb725aebbe8db9f49fb45b5d56b9d9fe68309ccead978e33f8f03a43e4e40686a7078ac9db57', 16) d = int('0090ef9ad5be850c651da0ae18a7213a41e334affa8b971faeb21897f9f569458c223adf2bb8bb9a23aa7522d95495e50b2f87095e5b9c461cc7d5764c8dcbbde1', 16) sign = hex(pow(md_padded, d, n))[2:] # Remove the 0x prefix
- The verification of a signature is similar. \(MD=PKCS1V1\_5^{-1}((c)^{e}\mod n)\)
- Note that though the scheme under PKCS1V1_5 is IND-CPA (semantically secure), it turns out to be vulnerable to Bleichenbacher attacks. PKCS1V2_2 addresses the issue using a more sophisticated padding scheme for both signature/encryption to achieve IND-CCA.
Version Control
Git is probably the most popular version control systems (VCS) among others: Subversion (SVN) is a centralized VCS and Mercurial (Hg) is another distributed VCS.
git reset --hard <commit hash>
points HEAD to a history commit AND rolls back the files- To mirror an existing Git repo A to another Git repo B, deletes the existing remote via
git remote rm origin
and adds a new remote viagit remote add origin https://github.com/liangcheng-yu/another.git
. Then overwrites repo B withgit push -f --set-upstream origin master
.
Job Monitoring
top
is commonly used to monitor currently dominant jobs and their usages. Pressc
to show full command for each job andY
to pause at current snapshotps -aux | less
displays all currently running jobs one page at a time
Cables and Wiring
- Serial cable connects the serial/console ports of two devices communicating via serial communication protocol.
minicom
is used to configure the baud, parity, stop bit and so on. - Breakout/fanout cables are often used to split the port bandwidth, e.g., a QSFP28-4SFP28 breakout cable split a 100G port of a switch to 4x25G server ports.
- Ethernet cables that run inside walls and crawl spaces are different from those meant to plugged in/out the computer repeatedly. The former is fragile to rewire and ideally settled once while the latter is more flexible to bent.
- Fixed Optical Attenuator, Variable Optical Attenuator
- Video Graphics Array (VGA) cable
- Transceiver: Fibre Channel and Ethernet Transceiver both have their own targeting use cases
- Common form factors (a set of specifications (size, shape…) during hardware design, in this context, for transceivers): 1G SFP, 10G SFP+, 25G SFP28, 40G QSFP+, 50G QSFP28, 100G QSFP28, 200G QSFP56, 400G QSFP-DD.
A nice FiberStore (FS.COM) post on the differences and backward compatibility of common form factors. E.g., SFP28 supports only one channel with 25 Gbit/s, while QSFP28 supports 4 separate lanes, each of 25 Gbps. In 100G networks, one could use QSFP28 to SFP28 breakout cables to connect a 100G switch with a 25G switch.