Creating and deleting files automatically in shell scripts

I frigging love linux for this shit

appends the date to filename $date + (“%Y%m%d”)

example

tar -zcf documents.backup.$(date +”%Y%m%d”).gz Documents

locate files of five days and older find * -type f -mtime +5

So there i was creating files and then wondering about the bother in deleting them manually when i thought there must be some way and a bit of thinking meant no human interaction from me with a cron job.

You want to the aim the find at the right place with the delete syntax which i have omitted.

Only moan i now have to delete my calender reminder i set.

Microsoft windows 10 – ransomware removal

Crook

The zoo bought some pc’s without microsft windows and when i got to install linux on it I spent a good ten minutes attempting to defeat tpm,secure boot and the windows boot manager for it had a virus on it called windows 10 *.

Quite why the fuck windows 10 has to talk to you and offers no shutdown during setup made me think of ransomware may be i think in original ways that this has never occurred to you before..

The bios really REALLY did not want me to remove the windows boot manager it kept going further and further down the list until it did get disabled – this reminding me of ransomware which happened to vista pc many years ago.

My linux on usb iso image although it worked on my usual device i wrote it to did not on the new pc. so i baked a cd image and that did, Once i installed the cdrom all was well with the universe and the ransomeware was gone from the disk..

In fact to remove the ransomware on that old pc i had to use a cd as well.  Great to see microsoft learning from the crooks.

I repurposed the usb thing for the non gpl hardware drivers.

mafia run the british red cross

It is strange that harddisks come with mafia approved virusware on them automatically , however i did find out the screen was ok, and the sound card worked. Thanks to the mafia at Microsoft.  Really wanted to not know that.

Here is a top tip: So if a new pc does not have sound  you should return it to your retailer immediately after all microsoft have decided its defective.

I wonder what other spyware courtesy of crooks and governments (my blog) was on the disk until i repartitioned it.

*no key sticker from microsoft

 

How much! and adverts too

also in beige

The zoo needed pc’s as some of them are well over ten years old and showing signs of hardware issues like forgetting the time** and being a bit slow with other regular seek human movements that persuaded the zoo to flash some cash on stuff..

Naturally i go to a site that i can buy to order rather than buy hp (my blog) or another brand get what we need without the crap copy of windows which is a lot cheaper than in stock and off the shelf from online retailers.

Crook

Deselecting windows 10 (my blog) resulted in the zoo shaving £100 off the bill (converted tp usd 130.00 at time i wrote this. So next time you wonder what you could do with an spare $130 makes using open source software a lot of sense.

mafia run microsoft

Apparently I also read that Microsoft are putting adverts everywhere in windows 10 so in addition to buying mafia insurance* from microsoft they also plaster adverts over it.

I am fascinated that people pay microsoft for this software via hardware vendors who i assume do it for laziness and some very shady microsoft sales staff.

Sure we have to install something on the hardware, but even windows needs setting up.

*nice shop window there, we will use it to sell your competitors products and spy on you. ** can break ssl.as websites will refuse to load if your bios believes its 1/1/1980

Unreliable systemd (shit software in the mist)

Is it up or down ? systemd has one job although the megalomaniacs at the systemd project really want to make it linux itself after all who needs dns servers and other stuff – i mean they can do that too, systemd one day might build cars for elon musk.

So after fixing opendkim with my original systemd config it i turned my attention to postfix which works but systemd has decided is not loaded or active despite being so.  Go figure.

Starting postfix (via systemctl): postfix.serviceJob for postfix.service failed because
the control process exited with error code.
See “systemctl status postfix.service” and “journalctl -xe” for details.
failed!

So i go exploring the joys of systemd once more after debian 9.2 is released (my blog)

systemctl status postfix.service
● postfix.service – Postfix Mail Transport Agent
Loaded: loaded (/lib/systemd/system/postfix.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since x; 11min ago
Docs: man:postfix(1)
Main PID: 10259 (code=exited, status=1/FAILURE)

Oct 10 14:50:22 * postfix/postfix-script[13821]: fatal: the Postfix mail system is already running
Oct 10 14:50:24 * postfix-*/postfix-script[13828]: fatal: the Postfix mail system is already running
Oct 10 14:50:25 * postfix-*/postfix-script[13835]: fatal: the Postfix mail system is already running
Oct 10 14:50:26 * postfix-*/postfix-script[13844]: fatal: the Postfix mail system is already running
Oct 10 14:50:27 * postfix-*/postfix-script[13851]: fatal: the Postfix mail system is already running
Oct 10 14:50:28* systemd[1]: postfix.service: Control process exited, code=exited status=1
Oct 10 14:50:28 * systemd[1]: Failed to start Postfix Mail Transport Agent.
Oct 10 14:50:28 * systemd[1]: postfix.service: Unit entered failed state.
Oct 10 14:50:28 * systemd[1]: postfix.service: Failed with result ‘exit-code’.
Oct 10 15:01:00 * systemd[1]: postfix.service: Dependency Before=postfix.service dropped

systemctl start postfix.service
Job for postfix.service failed because the control process exited with error code.
See “systemctl status postfix.service” and “journalctl -xe” for details.

postmulti -p status
postfix/postfix-script: the Postfix mail system is not running
postfix-*/postfix-script: the Postfix mail system is running: PID: 16077
postfix-*/postfix-script: the Postfix mail system is running: PID: 16075
postfix-*/postfix-script: the Postfix mail system is running: PID: 15993
postfix-*/postfix-script: the Postfix mail system is running: PID: 15642

for example

So regardless of whether /usr/sbin/postfix and postmutli starting in systemd  still cannot figure out whether its running or not  – well done you systemd project geniuses.

I can still run the software because i know systemd is shit at what it does but is this really the future of init systems ?   If systemd is unreliable and with distros ignoring custom systemd files then clearly systemd is not up to the job unless you think the systemd way*.

Why it starts twice now is a mystery to me and yet another systemd fuckup (my blog) to savour. although there be others well worth lol’ing over.

I do not change postfix configurations often and feel happy with postmulti so i know its not me making these changes.

Enough of that shit software i think for today

*George Orwell’s boot on face quote seems apt.

debian stretch update 9.2 screw ups with systemd

Sisyphus is still a role model

I hate systemd – an update to postfix wiped our systemd file (our file) and the postmulti settings did not start. Another milllion reasons to hang draw and quarter Potterang and the systemd people (my blog).

Fishing about in my notes i diagnosed with postmulti,, copied yet another broken debain systemd file copied my backup to the real systemd file, daemon reloaded systemctl and then started postfix.

Postfix starts x5 but now the systemd file says it does not start even though they are – so if you still think systemd is a good idea your in need a mental health doctor.

If anybody can fuckup a start stop script these people can.  I mean if my copy of what once worked now does not how the fuck is this maintainable.  Beyond me.

How was your weekend ?

Then i had the issues with opendkim. (my blog) same old shit once more which makes systemd a nightmare.

wrong results from kali linux

crossKali is a penetration testing distro and i loaded it on a usb stick and scanned a linux host on our local lan.  Oddly it thought that many accounts that did not exist did.

So it kind of explains why people try and brute force them from our public ip’s.

It did (or nmap [my blog] did)  detect open ports and some weaknesses on the rfc1918 lan and that hpkp was not setup up.  More false flags where flagged up too referencing  software which had no hardware to do things.

The next day i went through a system report and found no evidence of intrusion even though i did the test..

I also tried to brute force our internal  routers of which was to get a wifi password hours went by and i was not the wiser even though i knew the password.  Its interface was quite nice doing those ip <if> mode commands, but it did have root permissions.

After two hours and little extra insight of the zoo’s server from an inside address i decided to call it day with kali.

Steam on debian stable 64bit

Carol Beer little britain says computer said no

I changed my distro from testing to debian stable [jesse] (my blog) some months later i decide to reinstall steam (my blog) and find that i cannot make the dependencies

It appears i will not be installing steam without enabling i386 architecture support.

Argh! it is installable just a pain to install although something features are yet to make it into linux.  More on steam in the future.

mailgraph and logwatch reporting curiosities with postmulti and some regex’es for fun.

The zoo’s mailgraph charts are not working and i have mentioned it before (my blog).

So after changing our /etc/postfix instance (we have more better instances) for a new feature to allow outbound internet mail to be sent to an address the charts began to show only that traffic.  Bounces too also appear to work (not shown).

Spam and viruses as defined by amavis do work but the received email from those other postfix instances is still not being recognised even with explicit syslog statements in the main.cf file.

So something is off

Reading the charts could give you the impression that despite receiving email that the chart does not graph bur we appear send out spam and viruses and blocked,  The bounces where something i induced and could have been dmarc related too as many dmarc reporters have problems clearing there gmail inbox..

It is a good reminder that badly made statistics may look interesting but do not reflect reality.

The logwatch config files /usr/share/logwatch/default.conf/services/postfix.conf are written as perl and at this point are beyond my comprehension

*OnlyService = “(?:post(?:fix|grey|fwd|fix-1|fix2|fix-0|fix-3|policyd-spf)(?:/[-\w]*)?”
$postfix_Syslog_Name = “(?:post(?:fix|grey|fwd)|policyd-spf)”
# POSTMULTI NOT WORK *OnlyService = “postfix\d?/[-a-zA-Z\d]*”
#$postfix_Syslog_Name = “postfix\d?”

My changes are in bold. That does not work.. /etc/postfix-1 etc is how postmulti expects its managed instances to be located (my blog).

A few days pass and with the help of a pcre debuger [https://regex101.com/] i find that

$postfix_Syslog_Name = “postfix/[\w]*”
*OnlyService = “(:postfix-1/|postfix-2/|postfix-3/|postfix-4/|policyd-spf|postfix/|post-grey|post-fwd)(?:[-\w]*)?”

Provides output from postmulti instances as well as the /etc/postfix daemon.  I might not need that last postfix on the third line but completist me me thought it worth specifying.

post-fwd and post-grey are not used here in the zoo we use postscreen  The spf log part of the the section is a little unwieldy but that always was and i could turn it off,

I find with postmulti reporting that “postfix/lmtp” is best stated as “lmtp” if grepping unless you want to add extra grep lines to your cron jobs.

So charts are still a bit messed up.   Not the end of the world although i have cron jobs that grep for connections and sasl abusers so between the broken things and our existing zoo cron jobs we keep on top on what postfix is having to deal with.

A work in progress mailgraph.requires that the /usr/sbin/mailgraph file be changed for postmulti.

I seemed ho have some luck and you can see the switch on since the data before was sent from a non internet postfix host denoted by green and red suddenly appearing.

I changed the line for postfix (a regex again) from

if($prog =~ /^postfix\/(.*)/) {

to

if($prog =~ /:postfix|postfix-1\/(.*)|postfix-2\/(.*)|postfix-3\/(.*)|postfix-4\/(.*)/) {

Which is not very maintainable and a bit of a bodge job but gets the regex working for more than one instance..  If that reflects reality or not i will have to check with logwatch reporting although with postfix dropping more bad connections earlier (my blog) feels right so the charts now ignore a large quantity of data of bad smtp clients say.

106 Reject by IP --------
 3 49.213.57.100 unknown
 3 103.241.75.75 unknown

So mailgraph and postfix seem now not count certain items compared to before the upgrade.  So that regex might see an edit.

Mailgraph was and then was not working i was unsure of my efforts – another regex to adjust

I eventually found

/postfix-1\/(.*)|postfix-2\/(.*)|postfix-3\/(.*)|postfix-4\/(.*)|postfix/

Appears to show green / blue and red posfix lines

Fail2ban also seems to need some help – although it seems it will not trip with rate throttling controls in my experience although the odd prober does try an extract from logwatch.

smtp
10 AUTH command rate
10 110.175.112.118 110-175-112-118.tpgi.com.au
1 Connection rate
1 110.175.112.118 110-175-112-118.tpgi.com.au

Perhaps fail2ban’s postfix jails are redundant with the rate limiting feature in newer postfix. Not that fail2ban tripped that often with our non postmulti config.

As most of our email traffic is using tls (dane – my blog) (or trying to) i somehow think mailgraphs use out of the box does not reflect reality with the rate controls, bad clients getting ignore and tls traffic not shown so i suppose this graph shows genuine email traffic rather than all port 25 attempts..

Is sogo opensource ?

I have a small interest in supporting one mobile phone via some kind of webmail and Sogo (my blog) is your horse for that. Alas in Jessie i was i386, no joy there. In Stretch (debian) things went amd64 bit and to my surprise i found a deb that could install.  It felt too easy but….

Having found most of the config files i then wondered about the database schema and found no files to create the sogo tables*.

Sisyphus is still a role model

There do exist update files for schema for crap mysql and the amazing postgres

ls so/sql*
so/sql-update-1.2.2_to_1.3.0-mysql.sh
so/sql-update-1.2.2_to_1.3.0.sh
so/sql-update-1.3.11_to_1.3.12-mysql.sh
so/sql-update-1.3.11_to_1.3.12.sh
so/sql-update-1.3.3_to_1.3.4-mysql.sh
so/sql-update-1.3.3_to_1.3.4.sh
so/sql-update-2.0.4b_to_2.0.5-mysql.sh
so/sql-update-2.2.17_to_2.3.0-mysql.sh
so/sql-update-2.2.17_to_2.3.0.sh

But how can i adjust nothing to something ?  eg

echo “Step 1 – Converting c_content from VARCHAR to TEXT in table sogo_folder_info” >&2
tables=`psql -t -U $username -h $hostname $database -c “select split_part(c_location, ‘/’, 5) from $indextable;”`

for table in $tables;
do
convVCtoText
done

So my adventure in Sogo came to an abrupt halt.

It appears to be a compile job.  Close but not useable. Its good that open source still retains mystery to it.   After all it keeps me in daily blog posts. apt remove sogo.  Maybe the third install attempt will  be the one?

*i have other databases.

 

mod_defensible in Debian stetch

I have reported that it does not log.(my blog) although i thought it was working.

I noticed

LogLevel alert

added to the bottom of apache.conf rather than warn  – I never changed the apache config during the upgrade as apache2 came out of the experience pretty much working apart from defenisble.

Commenting it  meant the value of warn set way up in the file worked after a restart of the process.  So it still works and now logs.