Mail log IP address count

The following monster will parse the mail log and report on unique host connections along with a count.

cat /var/log/mail.log | \
  grep ' connect from unknown' | \
  awk '{ print $8 }' | \
  sort | \
  sed -n 's/.*\[\([0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\)\].*/\1/p' | \
  awk '{count[$1]++} END {for (word in count) print count[word], word}' | \
  sort -n

Data formats including YAML

I finally got around to reading about YAML at Wikipedia. Worth doing if you use YAML anywhere and haven’t read the YAML page yet.

The Wikipedia article links to the official website: yaml.org, which is good fun. It’s written in YAML! :D

If you’re interested in data formats and markup languages here’s some other reading on the subject:

CRC

I watched Ben Eater’s videos How do CRCs work? and Checksums and Hamming distance (the full course is Learn about error detection) then found the Wikipedia page: Cyclic redundancy check.

Also of interest:

Keyspace on the back of an envelope

One question that always plagues me: should I use 32-bit or 64-bit keys. I can never make a firm decision on this point. But I’ve just done some back of the envelope calculations in thinking about it.

If you issued one identity per second, every second of every day, it would take 136 years to exhaust the keyspace.

If you issued one thousand identities per second, it would take 50 days to exhaust the keyspace.

Samsung 990 PRO firmware version

I’m having an issue with my Samsung 990 PRO drive. Gonna try upgrading the firmware. Actually got myself a new computer just for the purpose. Figured I should start with knowing what the firmware version is presently. ChatGPT strikes again. Firmware version is 1B2QJXD7.

-------------------
Wed May 31 11:14:03 [bash:5.2.15 jobs:0 error:0 time:0]
jj5@charm:/home/jj5
$ ll /dev/disk/by-id/
total 0
drwxr-xr-x 2 root root 1.3K May 31 03:04 .
drwxr-xr-x 9 root root  180 May 30 08:45 ..
lrwxrwxrwx 1 root root    9 May 30 08:45 ata-DRW-24D5MT_KLCL6MF4200 -> ../../sr0
lrwxrwxrwx 1 root root    9 May 31 03:04 ata-WDC_WD30EFRX-68EUZN0_WD-WMC4N0D5506W -> ../../sdb
lrwxrwxrwx 1 root root   10 May 31 03:04 ata-WDC_WD30EFRX-68EUZN0_WD-WMC4N0D5506W-part1 -> ../../sdb1
lrwxrwxrwx 1 root root   10 May 31 03:04 ata-WDC_WD30EFRX-68EUZN0_WD-WMC4N0D5506W-part9 -> ../../sdb9
lrwxrwxrwx 1 root root    9 May 31 03:04 ata-WDC_WD30EFRX-68EUZN0_WD-WMC4N0D8E3C9 -> ../../sda
lrwxrwxrwx 1 root root   10 May 31 03:04 ata-WDC_WD30EFRX-68EUZN0_WD-WMC4N0D8E3C9-part1 -> ../../sda1
lrwxrwxrwx 1 root root   10 May 31 03:04 ata-WDC_WD30EFRX-68EUZN0_WD-WMC4N0D8E3C9-part9 -> ../../sda9
lrwxrwxrwx 1 root root    9 May 30 08:45 md-name-charm:0 -> ../../md0
lrwxrwxrwx 1 root root    9 May 30 08:45 md-name-charm:1 -> ../../md1
lrwxrwxrwx 1 root root    9 May 30 08:45 md-uuid-9ef6fffa:0d37be5a:7842f20e:1319d620 -> ../../md1
lrwxrwxrwx 1 root root    9 May 30 08:45 md-uuid-f52df5c3:9aa1e0f1:8eab0df8:9808eaeb -> ../../md0
lrwxrwxrwx 1 root root   13 May 30 08:45 nvme-eui.00000000000000000026b768642eff55 -> ../../nvme1n1
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-eui.00000000000000000026b768642eff55-part1 -> ../../nvme1n1p1
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-eui.00000000000000000026b768642eff55-part9 -> ../../nvme1n1p9
lrwxrwxrwx 1 root root   13 May 30 09:57 nvme-eui.00000000000000000026b768642eff65 -> ../../nvme3n1
lrwxrwxrwx 1 root root   15 May 30 09:57 nvme-eui.00000000000000000026b768642eff65-part1 -> ../../nvme3n1p1
lrwxrwxrwx 1 root root   15 May 30 09:57 nvme-eui.00000000000000000026b768642eff65-part9 -> ../../nvme3n1p9
lrwxrwxrwx 1 root root   13 May 30 08:45 nvme-eui.00253842314123dd -> ../../nvme4n1
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-eui.00253842314123dd-part1 -> ../../nvme4n1p1
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-eui.00253842314123dd-part9 -> ../../nvme4n1p9
lrwxrwxrwx 1 root root   13 May 30 08:45 nvme-eui.6479a74fb0c004b7 -> ../../nvme2n1
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-eui.6479a74fb0c004b7-part1 -> ../../nvme2n1p1
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-eui.6479a74fb0c004b7-part2 -> ../../nvme2n1p2
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-eui.6479a74fb0c004b7-part3 -> ../../nvme2n1p3
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-eui.6479a74fb0c004b7-part4 -> ../../nvme2n1p4
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-eui.6479a74fb0c004b7-part5 -> ../../nvme2n1p5
lrwxrwxrwx 1 root root   13 May 30 08:45 nvme-eui.6479a74fb0c00509 -> ../../nvme0n1
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-eui.6479a74fb0c00509-part1 -> ../../nvme0n1p1
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-eui.6479a74fb0c00509-part2 -> ../../nvme0n1p2
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-eui.6479a74fb0c00509-part3 -> ../../nvme0n1p3
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-eui.6479a74fb0c00509-part4 -> ../../nvme0n1p4
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-eui.6479a74fb0c00509-part5 -> ../../nvme0n1p5
lrwxrwxrwx 1 root root   13 May 30 08:45 nvme-KINGSTON_SNV2S2000G_50026B768642EFF5 -> ../../nvme1n1
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-KINGSTON_SNV2S2000G_50026B768642EFF5-part1 -> ../../nvme1n1p1
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-KINGSTON_SNV2S2000G_50026B768642EFF5-part9 -> ../../nvme1n1p9
lrwxrwxrwx 1 root root   13 May 30 09:57 nvme-KINGSTON_SNV2S2000G_50026B768642EFF6 -> ../../nvme3n1
lrwxrwxrwx 1 root root   15 May 30 09:57 nvme-KINGSTON_SNV2S2000G_50026B768642EFF6-part1 -> ../../nvme3n1p1
lrwxrwxrwx 1 root root   15 May 30 09:57 nvme-KINGSTON_SNV2S2000G_50026B768642EFF6-part9 -> ../../nvme3n1p9
lrwxrwxrwx 1 root root   13 May 30 08:45 nvme-PNY_CS3140_1TB_SSD_PNY21242106180100092 -> ../../nvme2n1
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-PNY_CS3140_1TB_SSD_PNY21242106180100092-part1 -> ../../nvme2n1p1
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-PNY_CS3140_1TB_SSD_PNY21242106180100092-part2 -> ../../nvme2n1p2
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-PNY_CS3140_1TB_SSD_PNY21242106180100092-part3 -> ../../nvme2n1p3
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-PNY_CS3140_1TB_SSD_PNY21242106180100092-part4 -> ../../nvme2n1p4
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-PNY_CS3140_1TB_SSD_PNY21242106180100092-part5 -> ../../nvme2n1p5
lrwxrwxrwx 1 root root   13 May 30 08:45 nvme-PNY_CS3140_1TB_SSD_PNY21242106180100095 -> ../../nvme0n1
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-PNY_CS3140_1TB_SSD_PNY21242106180100095-part1 -> ../../nvme0n1p1
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-PNY_CS3140_1TB_SSD_PNY21242106180100095-part2 -> ../../nvme0n1p2
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-PNY_CS3140_1TB_SSD_PNY21242106180100095-part3 -> ../../nvme0n1p3
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-PNY_CS3140_1TB_SSD_PNY21242106180100095-part4 -> ../../nvme0n1p4
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-PNY_CS3140_1TB_SSD_PNY21242106180100095-part5 -> ../../nvme0n1p5
lrwxrwxrwx 1 root root   13 May 30 08:45 nvme-Samsung_SSD_990_PRO_2TB_S6Z2NJ0W215164J -> ../../nvme4n1
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-Samsung_SSD_990_PRO_2TB_S6Z2NJ0W215164J-part1 -> ../../nvme4n1p1
lrwxrwxrwx 1 root root   15 May 30 08:45 nvme-Samsung_SSD_990_PRO_2TB_S6Z2NJ0W215164J-part9 -> ../../nvme4n1p9
lrwxrwxrwx 1 root root    9 May 30 08:45 wwn-0x5001480000000000 -> ../../sr0
lrwxrwxrwx 1 root root    9 May 31 03:04 wwn-0x50014ee604f5f467 -> ../../sda
lrwxrwxrwx 1 root root   10 May 31 03:04 wwn-0x50014ee604f5f467-part1 -> ../../sda1
lrwxrwxrwx 1 root root   10 May 31 03:04 wwn-0x50014ee604f5f467-part9 -> ../../sda9
lrwxrwxrwx 1 root root    9 May 31 03:04 wwn-0x50014ee65a4af2ed -> ../../sdb
lrwxrwxrwx 1 root root   10 May 31 03:04 wwn-0x50014ee65a4af2ed-part1 -> ../../sdb1
lrwxrwxrwx 1 root root   10 May 31 03:04 wwn-0x50014ee65a4af2ed-part9 -> ../../sdb9
-------------------
Wed May 31 11:14:11 [bash:5.2.15 jobs:0 error:0 time:8]
jj5@charm:/home/jj5
$ sudo nvme smart-log /dev/nvme4
Smart Log for NVME device:nvme4 namespace-id:ffffffff
critical_warning                        : 0
temperature                             : 45°C (318 Kelvin)
available_spare                         : 100%
available_spare_threshold               : 10%
percentage_used                         : 0%
endurance group critical warning summary: 0
Data Units Read                         : 2,237,856 (1.15 TB)
Data Units Written                      : 10,957,014 (5.61 TB)
host_read_commands                      : 48,079,050
host_write_commands                     : 119,766,894
controller_busy_time                    : 282
power_cycles                            : 31
power_on_hours                          : 23
unsafe_shutdowns                        : 16
media_errors                            : 0
num_err_log_entries                     : 0
Warning Temperature Time                : 0
Critical Composite Temperature Time     : 0
Temperature Sensor 1           : 45°C (318 Kelvin)
Temperature Sensor 2           : 49°C (322 Kelvin)
Thermal Management T1 Trans Count       : 0
Thermal Management T2 Trans Count       : 0
Thermal Management T1 Total Time        : 0
Thermal Management T2 Total Time        : 0
-------------------
Wed May 31 11:36:59 [bash:5.2.15 jobs:0 error:0 time:0]
root@charm:/home/jj5
# nvme id-ctrl /dev/nvme4 -H
NVME Identify Controller:
vid       : 0x144d
ssvid     : 0x144d
sn        : S6Z2NJ0W215164J     
mn        : Samsung SSD 990 PRO 2TB                 
fr        : 1B2QJXD7
rab       : 2
ieee      : 002538
cmic      : 0
  [3:3] : 0     ANA not supported
  [2:2] : 0     PCI
  [1:1] : 0     Single Controller
  [0:0] : 0     Single Port

mdts      : 9
cntlid    : 0x1
ver       : 0x20000
rtd3r     : 0x30d40
rtd3e     : 0x989680
oaes      : 0x200
  [31:31] : 0   Discovery Log Change Notice Not Supported
  [27:27] : 0   Zone Descriptor Changed Notices Not Supported
  [15:15] : 0   Normal NSS Shutdown Event Not Supported
  [14:14] : 0   Endurance Group Event Aggregate Log Page Change Notice Not Supported
  [13:13] : 0   LBA Status Information Notices Not Supported
  [12:12] : 0   Predictable Latency Event Aggregate Log Change Notices Not Supported
  [11:11] : 0   Asymmetric Namespace Access Change Notices Not Supported
  [9:9] : 0x1   Firmware Activation Notices Supported
  [8:8] : 0     Namespace Attribute Changed Event Not Supported

ctratt    : 0x10
  [19:19] : 0   Flexible Data Placement Not Supported
  [15:15] : 0   Extended LBA Formats Not Supported
  [14:14] : 0   Delete NVM Set Not Supported
  [13:13] : 0   Delete Endurance Group Not Supported
  [12:12] : 0   Variable Capacity Management Not Supported
  [11:11] : 0   Fixed Capacity Management Not Supported
  [10:10] : 0   Multi Domain Subsystem Not Supported
  [9:9] : 0     UUID List Not Supported
  [8:8] : 0     SQ Associations Not Supported
  [7:7] : 0     Namespace Granularity Not Supported
  [6:6] : 0     Traffic Based Keep Alive Not Supported
  [5:5] : 0     Predictable Latency Mode Not Supported
  [4:4] : 0x1   Endurance Groups Supported
  [3:3] : 0     Read Recovery Levels Not Supported
  [2:2] : 0     NVM Sets Not Supported
  [1:1] : 0     Non-Operational Power State Permissive Not Supported
  [0:0] : 0     128-bit Host Identifier Not Supported

rrls      : 0
cntrltype : 1
  [7:2] : 0     Reserved
  [1:0] : 0x1   I/O Controller
fguid     : 00000000-0000-0000-0000-000000000000
crdt1     : 0
crdt2     : 0
crdt3     : 0
nvmsr     : 0
  [1:1] : 0     NVM subsystem Not part of an Enclosure
  [0:0] : 0     NVM subsystem Not part of an Storage Device

vwci      : 0
  [7:7] : 0     VPD Write Cycles Remaining field is Not valid.
  [6:0] : 0     VPD Write Cycles Remaining 

mec       : 0
  [1:1] : 0     NVM subsystem Not contains a Management Endpoint on a PCIe port
  [0:0] : 0     NVM subsystem Not contains a Management Endpoint on an SMBus/I2C port

oacs      : 0x17
  [10:10] : 0   Lockdown Command and Feature Not Supported
  [9:9] : 0     Get LBA Status Capability Not Supported
  [8:8] : 0     Doorbell Buffer Config Not Supported
  [7:7] : 0     Virtualization Management Not Supported
  [6:6] : 0     NVMe-MI Send and Receive Not Supported
  [5:5] : 0     Directives Not Supported
  [4:4] : 0x1   Device Self-test Supported
  [3:3] : 0     NS Management and Attachment Not Supported
  [2:2] : 0x1   FW Commit and Download Supported
  [1:1] : 0x1   Format NVM Supported
  [0:0] : 0x1   Security Send and Receive Supported

acl       : 7
aerl      : 3
frmw      : 0x16
  [5:5] : 0     Multiple FW or Boot Update Detection Not Supported
  [4:4] : 0x1   Firmware Activate Without Reset Supported
  [3:1] : 0x3   Number of Firmware Slots
  [0:0] : 0     Firmware Slot 1 Read/Write

lpa       : 0x2f
  [6:6] : 0     Telemetry Log Data Area 4 Not Supported
  [5:5] : 0x1   LID 0x0, Scope of each command in LID 0x5, 0x12, 0x13 Supported
  [4:4] : 0     Persistent Event log Not Supported
  [3:3] : 0x1   Telemetry host/controller initiated log page Supported
  [2:2] : 0x1   Extended data for Get Log Page Supported
  [1:1] : 0x1   Command Effects Log Page Supported
  [0:0] : 0x1   SMART/Health Log Page per NS Supported

elpe      : 63
  [7:0] : 63 (0's based)        Error Log Page Entries (ELPE)

npss      : 4
  [7:0] : 4 (0's based) Number of Power States Support (NPSS)

avscc     : 0x1
  [0:0] : 0x1   Admin Vendor Specific Commands uses NVMe Format

apsta     : 0x1
  [0:0] : 0x1   Autonomous Power State Transitions Supported

wctemp    : 355
 [15:0] : 82°C (355 Kelvin)     Warning Composite Temperature Threshold (WCTEMP)

cctemp    : 358
 [15:0] : 85°C (358 Kelvin)     Critical Composite Temperature Threshold (CCTEMP)

mtfa      : 0
hmpre     : 0
hmmin     : 0
tnvmcap   : 2,000,398,934,016
[127:0] : 2,000,398,934,016
        Total NVM Capacity (TNVMCAP)

unvmcap   : 0
[127:0] : 0
        Unallocated NVM Capacity (UNVMCAP)

rpmbs     : 0
 [31:24]: 0     Access Size
 [23:16]: 0     Total Size
  [5:3] : 0     Authentication Method
  [2:0] : 0     Number of RPMB Units

edstt     : 35
dsto      : 0
fwug      : 0
kas       : 0
hctma     : 0x1
  [0:0] : 0x1   Host Controlled Thermal Management Supported

mntmt     : 318
 [15:0] : 45°C (318 Kelvin)     Minimum Thermal Management Temperature (MNTMT)

mxtmt     : 356
 [15:0] : 83°C (356 Kelvin)     Maximum Thermal Management Temperature (MXTMT)

sanicap   : 0x60000003
  [31:30] : 0x1 Media is not additionally modified after sanitize operation completes successfully
  [29:29] : 0x1 No-Deallocate After Sanitize bit in Sanitize command Not Supported
    [2:2] : 0   Overwrite Sanitize Operation Not Supported
    [1:1] : 0x1 Block Erase Sanitize Operation Supported
    [0:0] : 0x1 Crypto Erase Sanitize Operation Supported

hmminds   : 0
hmmaxd    : 0
nsetidmax : 0
endgidmax : 1
anatt     : 0
anacap    : 0
  [7:7] : 0     Non-zero group ID Not Supported
  [6:6] : 0     Group ID does change
  [4:4] : 0     ANA Change state Not Supported
  [3:3] : 0     ANA Persistent Loss state Not Supported
  [2:2] : 0     ANA Inaccessible state Not Supported
  [1:1] : 0     ANA Non-optimized state Not Supported
  [0:0] : 0     ANA Optimized state Not Supported

anagrpmax : 0
nanagrpid : 0
pels      : 0
domainid  : 0
megcap    : 0
sqes      : 0x66
  [7:4] : 0x6   Max SQ Entry Size (64)
  [3:0] : 0x6   Min SQ Entry Size (64)

cqes      : 0x44
  [7:4] : 0x4   Max CQ Entry Size (16)
  [3:0] : 0x4   Min CQ Entry Size (16)

maxcmd    : 256
nn        : 1
oncs      : 0x55
  [8:8] : 0     Copy Not Supported
  [7:7] : 0     Verify Not Supported
  [6:6] : 0x1   Timestamp Supported
  [5:5] : 0     Reservations Not Supported
  [4:4] : 0x1   Save and Select Supported
  [3:3] : 0     Write Zeroes Not Supported
  [2:2] : 0x1   Data Set Management Supported
  [1:1] : 0     Write Uncorrectable Not Supported
  [0:0] : 0x1   Compare Supported

fuses     : 0
  [0:0] : 0     Fused Compare and Write Not Supported

fna       : 0x7
  [3:3] : 0     Format NVM Broadcast NSID (FFFFFFFFh) Supported
  [2:2] : 0x1   Crypto Erase Supported as part of Secure Erase
  [1:1] : 0x1   Crypto Erase Applies to All Namespace(s)
  [0:0] : 0x1   Format Applies to All Namespace(s)

vwc       : 0x7
  [2:1] : 0x3   The Flush command supports NSID set to FFFFFFFFh
  [0:0] : 0x1   Volatile Write Cache Present

awun      : 1023
awupf     : 0
icsvscc   : 1
  [0:0] : 0x1   NVM Vendor Specific Commands uses NVMe Format

nwpc      : 0
  [2:2] : 0     Permanent Write Protect Not Supported
  [1:1] : 0     Write Protect Until Power Supply Not Supported
  [0:0] : 0     No Write Protect and Write Protect Namespace Not Supported

acwu      : 0
ocfs      : 0
  [1:1] : 0     Controller Copy Format 1h Not Supported
  [0:0] : 0     Controller Copy Format 0h Not Supported

sgls      : 0
 [15:8] : 0     SGL Descriptor Threshold
 [1:0]  : 0     Scatter-Gather Lists Not Supported

mnan      : 0
maxdna    : 0
maxcna    : 0
subnqn    : nqn.1994-11.com.samsung:nvme:990PRO:M.2:S6Z2NJ0W215164J     
ioccsz    : 0
iorcsz    : 0
icdoff    : 0
fcatt     : 0
  [0:0] : 0     Dynamic Controller Model

msdbd     : 0
ofcs      : 0
  [0:0] : 0     Disconnect command Not Supported

ps      0 : mp:9.39W operational enlat:0 exlat:0 rrt:0 rrl:0
            rwt:0 rwl:0 idle_power:- active_power:-
            active_power_workload:-
ps      1 : mp:9.39W operational enlat:0 exlat:200 rrt:1 rrl:1
            rwt:1 rwl:1 idle_power:- active_power:-
            active_power_workload:-
ps      2 : mp:9.39W operational enlat:0 exlat:1000 rrt:2 rrl:2
            rwt:2 rwl:2 idle_power:- active_power:-
            active_power_workload:-
ps      3 : mp:0.0400W non-operational enlat:2000 exlat:1200 rrt:3 rrl:3
            rwt:3 rwl:3 idle_power:- active_power:-
            active_power_workload:-
ps      4 : mp:0.0050W non-operational enlat:500 exlat:9500 rrt:4 rrl:4
            rwt:4 rwl:4 idle_power:- active_power:-
            active_power_workload:-
-------------------

Data corruption

Oh, yuck. ZFS-8000-8A. :(

-------------------
Mon May 29 16:23:47 [bash:5.2.15 jobs:0 error:0 time:35]
root@charm:/home/jj5
# zpool status -v
  pool: fast
 state: ONLINE
status: One or more devices has experienced an error resulting in data
        corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
        entire pool from backup.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
config:

        NAME                                              STATE     READ WRITE CKSUM
        fast                                              ONLINE       0     0     0
          mirror-0                                        ONLINE       0     0     0
            nvme-Samsung_SSD_990_PRO_2TB_S6Z2NJ0W215171W  ONLINE       0     0     2
            nvme-Samsung_SSD_990_PRO_2TB_S6Z2NJ0W215164J  ONLINE       0     0     2

errors: Permanent errors have been detected in the following files:

        /fast/vbox/218-jj-wrk-8-charm-prod-vbox/218-jj-wrk-8-charm-prod-vbox.vdi
-------------------