Running debian stretch thoughts

As a server (my blog not as workstation (my blog)

The improvements first.

Postfix with postmulti

I had this ‘forced’ on me by systemd (my blog) not running the simple one instance that supposedly everybody else does and is ok with.  While i am still a bit newbish with the postfix command replacement it feels a better solution especially with logging which my older extra instances did not offer – as to what instance was doing.

There is no real config change but seems a more modern config despite that not changing and already having scripts that ise multiple postfix configs.

Postfix is a lot less tolerant of clients such as

connect from unknown[12.130.172.232]

Who don’t do much but like to test things and despite our config from before denying such activity [ddos,spammers,clueless,bot nets] it feels as if they are discarded more quickly

Tls probers (waste of time clients) have sslv3 requests redirected to something more modern tls. Milters where odd to debug.  See the mailgraph heading below for a downside.

Dnssec

My manual method of signing zones still works – the daemon software in jessie i could not figure out – mostly because most people never used it or compiled a better version and never used the debian package being out of date.  It is something i might look at in the future

Signing zones currently takes a minute and is something i spend twelve minutes a year doing.

Not sure of and probably blame me department

cyrus imap

I am on the fence on this upgrade and feel the config still needs looking at despite working it seems a little no there et after i did seem to lose a lot of config commands to get it to work along with those bloody sockets i had to delete..

Hard to fix and runtime issues with solutions.

mod_defensible

Appears to work but now does not log – kind of/maybe situation – apache says it wrote 403 errors, but the confirmation from before means i take one softwares fact as a unknown.

Hairy eyeball those emails from root to to get these – you did read them right ?.

Mailgraph

Is not updating its png images, html updates with the date and time – think this is not a systemd thing once again not /etc/default or init.d issue.- not an urgent issue and might be something due to postmulti since spam is logged in the chart and the mail.log has stuff in it.

I get

Well it recorded the one spam

Which is not an accurate representation of activity – files do update but no postfix activity in charts.   Not working.i guess due to incompatible parsing of:

postfix-instancex/smtpd[*]: disconnect from unknown[14.161.40.101]
 helo=1 auth=0/1 quit=1 commands=2/3

My theory anyhow

logwatch

I had to adjust the cron job to get the more detailed report once again for stretch to reflect as jessie. not hard.

p0f

Has had a redesign  – we only used for port 25 scanning with amavis and now does not work with the old config.   With postfix improvements i might let this fall in to non use.

X Logout leaves running processes

Don’t ask me why but it does cache cleaner, and hplip and systemd seem unhappy with something.

rkhunter

Need –pkgmgr DPKG in cron.daily as extra parameters

logrotate

I need

TMPDIR=/var/tmp
export TMPDIR
/usr/sbin/logrotate /etc/logrotate.conf
TMPDIR=/tmp
export TMPDIR

Or i get mail errors

Yara

Can now be ‘easily’ installed without’breaking’ debian.so i might be adding to the virus scanner which gets little cause for use with our mail config refusing bad content before it gets to a scanner.

Opendmarc

reporting (my blog) command appears to now send reports from more than just the first reporting host who got on with it and did not complain.  I guess the atps and other dns lines might be deemed redundant.

apt

The package maanager does ttry o do more than it should trying to restart things – an easy example yesterday apt upgraded apache2 documents and tried to add the module and failed when the conf file it tried to read did not exist where debian assumed it was.

So i had to restart apache.manually.

Overall not a disaster.   Something i might think as a good idea at some point but then again it as a server does a lot of jobs.I guess upgrading one server with one task would be less stressful.

apt-xapian-index

I have decided to disable it since this shit software has done absolutely nothing for years except produce a weekly stack trace.

/etc/cron.weekly/apt-xapian-index:
Traceback (most recent call last):
File “/usr/sbin/update-apt-xapian-index”, line 97, in <module>
if not indexer.setupIndexing(force=opts.force, system=opts.pkgfile is None):
File “/usr/lib/python2.7/dist-packages/axi/indexer.py”, line 518, in setupIndexing
addon.obj.init(dict(values=self.values), self.progress)
File “/usr/share/apt-xapian-index/plugins/translated-desc.py”, line 105, in init
self.indexers.append(Indexer(lang, file))
File “/usr/share/apt-xapian-index/plugins/translated-desc.py”, line 41, in __init__
for pkg in deb822.Deb822.iter_paragraphs(open(file)):
File “/usr/lib/python2.7/dist-packages/debian/deb822.py”, line 388, in iter_paragraphs
x = cls(iterable, fields, encoding=encoding)
File “/usr/lib/python2.7/dist-packages/debian/deb822.py”, line 336, in __init__
self._internal_parser(sequence, fields)
File “/usr/lib/python2.7/dist-packages/debian/deb822.py”, line 441, in _internal_parser
line = self._detect_encoding(line)
File “/usr/lib/python2.7/dist-packages/debian/deb822.py”, line 217, in _detect_encoding
return value.decode(result[‘encoding’])
TypeError: decode() argument 1 must be string, not None
run-parts: /etc/cron.weekly/apt-xapian-index exited with return code 1

I may have mentioned it before once or maybe twice (my blog)

Anyhow that’s one piece of software that will not be bothering the zoo in debain stretch .

upsetting hplip when reinstalling a printer queue

One of the zoos printers was not working via hplip after i compiled it so i deleted it and set it up again for the new printer (my blog). One has real ink and the other has non hp toner (my blog) in it – hplip the linux driver for printing then refused to tell me any other ink levels -genuine or not regardless of the sorry state of guessing ink quantity by this hp software which is best ignored imho.

I feel so chastised*. – still see no purpose in buying hp ‘genuine’ ink after all .

*when the ink does not print it probably means it needs toner.

‘genuine’ toner

An older printer here in the zoo needed toner so i obtained one for it at a very reasonable price. It works and a couple of days later i decided to look up what ‘genuine’ toner costs.

I nearly fell out of the tree when i discovered a 545% difference in price for 1500 pages. That would have been expensive per page.  The non genuine one i bought is apparently rated for 2000* pages – honestly speaking as long as it lasts about the same time as usual i think that genuine toner has lost the argument.

Perhaps ‘posh’ ink should have it written on every document to impress people – a somewhat incorrect footer might say we paid 500% more to print this page.

Nice to know. *extra toner in the container

Debian Postfix v2 to v3 notes – including postmulti setup

Upgrading postfix configurations from Jessie to Stretch was ‘challenging‘  (my blog) it works but required manual startup rather than auto start on boot. Systemd being an annoyance and with the zoos config deemed bad or not as trendy as some newer configs i had to setup postmulti and learn systems syntax to auto start it

postfix upgrade-configuration resulted in these changes to already working postfix configurations (one per directory)

Upgrading Postfix

Editing /etc/postfix/master.cf, adding missing entry for postscreen TCP service
Editing /etc/postfix/master.cf, adding missing entry for smtpd unix-domain service
Editing /etc/postfix/master.cf, adding missing entry for dnsblog unix-domain service
Editing /etc/postfix/master.cf, adding missing entry for tlsproxy unix-domain service

Note: the following files or directories still exist but are no
longer part of Postfix:

/etc/postfix/postfix-script /etc/postfix/post-install
/usr/share/doc/postfix/QMQP_README

COMPATIBILITY: editing /etc/postfix/main.cf, setting
inet_protocols=ipv4. Specify inet_protocols explicitly if you want
to enable IPv6. In a future release

Version 2 issues

chroot issues your be doing a lot of as – does not mean n

submission inet n – – – – smtpd

to

submission inet n – n – – smtpd

I left my unrooted as i did not want to fight battles with sasl sockets and milters.

New features for v3

Quick Mail Queueing Protocol is i think something to do with 628 setting in master.cf that has been commented for years.  Quite what it does is still a mystery.

Postmulti

Meant copying directories and moving them as postmulti likes /etc/postfix-1 /etc/postfix-2  rather than /etc/postfix/1.  As an obliging ape did that inited the settings in /etc/postfix (different to postfix-1 etc) and imported with postmulti -I postfox1 -G mta

postmult works to start postmulti -p start|stop|reload|status

the systemd config changed on debian stretch to do it via group rather than the broken example in postfix@.service

i used a variant of

postmutli -g mta -p start

Rather than postmutli  -i %i -p eatbanana

I got friendly with postmulti first rather than wonder why the fuck systemd was doing what it was doing.

So it kind of works – i really should recreate my postfix config of over ten years but it is a lot of work and is spammer proof and a lot of other features that a new instance of postmutli mostly have.

systemctl status postfix
● postfix.service – Postfix Mail Transport Agent (instance )
Loaded: loaded (/lib/systemd/system/postfix.service; enabled; vendor preset:
Active: active (running) since x BST; x ago
Docs: man:postfix(1)
Process: 15310 ExecStop=/usr/sbin/postmulti -g mta -p stop (code=exited, status=0/SUCCESS)
Process: 15669 ExecStart=/usr/sbin/postmulti -g mta -p start (code=exited, status=0/SUCCESS)
Process: 15609 ExecStartPre=/usr/lib/postfix/configure-instance.sh (code=exited, status=0/SUCCESS)
Main PID: 2255 (code=exited, status=0/SUCCESS)
Tasks: 14 (limit: 4915)
CGroup: /system.slice/postfix.service
├─15753 /usr/lib/postfix/sbin/master -w
├─15755 pickup -l -t fifo -u
├─15756 qmgr -l -t fifo -u
├─15836 /usr/lib/postfix/sbin/master -w
├─15837 pickup -l -t fifo -u
├─15838 qmgr -l -t fifo -u
├─15916 /usr/lib/postfix/sbin/master -w
├─15917 pickup -l -t fifo -u
├─15918 qmgr -l -t fifo -u
├─15996 /usr/lib/postfix/sbin/master -w
├─15997 pickup -l -t fifo -u
├─15998 qmgr -l -t fifo -u
├─16151 tlsmgr -l -t unix -u
└─16307 tlsmgr -l -t unix -u

Jul 11 11:03:01 mail2 postfix-x/smtpd[*]:

I managed to send mail to gmail and the existing config plus upgrades appears to sign and validate n dkim and spf.

Being bananas in the falklands some wit from systemd will probably overwrite my systemd posfix service file in the future just to make my life enjoyable as i am no expert with this limiting software and put it in the wrong place.

On a plus note i have a backup of my older postfix configs – who says systemd has good points*#

*this is called sarcasm

Further Debian Stretch as a server notes

rounding up the fairies

Following on from this (my blog) i continue my bug upgrade hunt.  Its not over.

I have mentioned many of these items before in this blog, it is not my job to tell you what they are.

Apache/Perl

Rkhunter say:

Warning: The command ‘/usr/bin/lwp-request’ has been replaced by a script: /usr/bin/lwp-request:
Perl script text executable

Might explain why perl did not exec via my ‘old’ cgi scripts as Jessie

Opendkim /Postfix

I ‘needed’ an extra line (also in /etc/default/opendkim)

PidFile /var/run/opendkim/opendkim.pid

in opendkim.conf – mail was being sent without dkim

I appear to not have dkim signatures in outbound email., opendkim-testkey thinks its config is good  i think it might be easier to reconfigure postfix from scratch.  It is not milter_protocol= 6 and 2 does not work.  Um no idea.   Opendkim seems up but not connected.

Opendkim was not working. Eventually this clued (not here) me in that the openkim config files where fine but the systemd script was buggered

So if your config files are right but the daemon refuses to follow orders try this

edit /lib/systemd/system/opendkim.service

from this

[Unit]
Description=OpenDKIM DomainKeys Identified Mail (DKIM) Milter
Documentation=man:opendkim(8) man:opendkim.conf(5) man:opendkim-genkey(8) man:opendkim-genzone(8) man:opendkim-testadsp(8) man:opendkim-testkey http://www.opendkim.org/docs.html
After=network.target nss-lookup.target

[Service]
Type=forking
PIDFile=/var/run/opendkim/opendkim.pid
User=opendkim
UMask=0007
ExecStart=/usr/sbin/opendkim -P /var/run/opendkim/opendkim.pid -p local:/var/run/opendkim/opendkim.sock
Restart=on-failure
ExecReload=/bin/kill -USR1 $MAINPID

[Install]
WantedBy=multi-user.target

to

[Unit]
Description=OpenDKIM DomainKeys Identified Mail (DKIM) Milter
Documentation=man:opendkim(8) man:opendkim.conf(5) man:opendkim-genkey(8) man:opendkim-genzone(8) man:opendkim-testadsp(8) man:opendkim-testkey http://www.opendkim.org/docs.html
After=network.target nss-lookup.target[Service]
Type=forking
PIDFile=/var/run/opendkim/opendkim.pid
User=opendkim
UMask=0007
ExecStart=/usr/sbin/opendkim -P /var/run/opendkim/opendkim.pid -p local:/var/run/opendkim/opendkim.sock -p inet:8891:localhost
Restart=on-failure
ExecReload=/bin/kill -USR1 $MAINPID

[Install]
WantedBy=multi-user.target

run (as root)

  • systemctl daemon-reload
  • /etc/init.d/opendkim restart

I hate systemd – that caused me six days of bug hunting it is limiting

Postfix needs a blog post on its own.

Opendmarc

I needed to re-enable it to start on boot oh the joys of systemd where init.d is thought as an unreliable forgetful moron and systemd knows best when clearly it is as fucked up (my blog)

It still did connect so it is a journey in systemd to fix (see opendkim magic above)

dmarc reports does not like interval and day together which appeared ok in Jessie

It is still a bit broken so nobody is being sent reports – not that many dmarc enabled domains who ‘specailise’ in just that really care about (my blog).  HistoryFile does not record data – why – no idea

-rw-rw-r– 1 opendmarc opendmarc 0 Jul 10 10:08 opendmarc.log

So a headscratcher. – and not something i can fix.

Postgres 9.4

I chowned a snakeoiil key – tested more cosmetic this than a issue which continues from Jessie..

Logwatch

Is a use full thing in my opinion although a little lacking in places moves from 7am to midnight for timing

Bind

Stops telling you if you do not have a specific spf record even though i have text records containing spf for the benefit of all the mostly retarded who run microsoft windows servers who have issues

mod_defensibile

Jury is out on if this is broken or the dns is bad. Or alternatively no rbl listed ip’s visited.

To fix

opendmarc loging, postfix startup, mod_defensible

Would i recommend the upgrade – at this point no.

Newer hp officejet printers weird setup on Linux

a printer image

The zoo’s hp officejet printer dating from 2007 died when it decided not to print 75% of the things it was asked to do of recent it made some cool grinding sounds indicating something or other. Unwillingly i opted for a newer model (under a hundred in incremental model versiion number) as a quick replacement of which several hours later i am still setting up.

Oddly though the setup was geared up for windows and mac users – no 123.hp.com dont work, us linux folk had to use cups to setup basic printing. The printer does not come with a brick to power it just a cable from a plug which from a firm that love there power bricks is different hp power bricks can be a horrible quality.

When the printer is off it is ‘sealed’ inside whether that preserves the ink is a interesting question and an interesting view of a possible design defect solution from hp

The rest of the features like instant ink connecting to its wifi and andriod i gave up on as not important to start with after all printing from our linux pc’s is more important than that.I will come back to that

Hplip (the official linux driver from hp) does not support the printer (and does not show) despite cups getting the printer to do stuff. – is hplip err legacy ? no so i had to compile a newer version from source, ignore qt4 or it will never compile. Then hplip shows the printer that cups can see and scan things.

Perhaps as linux users hp.com do not want any more money from us special types ? and I have no idea how you change its default wifi password or join it to our wifi network until i figured out the touch screen with abc as one key and you have to touch the key twice to get a b – not explained in the paper manual but something i eventually figured out wth guessing.

The printer prints in duplex and adds yet another unopened rj11 cable to the zoo’s collection of unused rj11 cables and adaptors for the phone socket since nobody faxes any more.

The paper documentation is thoroughly shit but as long it lasts a long time i guess the zoo will remain happy.

Android printing does work although the eprint app seems very ‘cloudy’* and desiring of optional things not essential if in local range.

Instant ink is another guessing game from hp for us linux folk to figure out and a google link to another link told me that hplip has sod all to do instant ink and is handled by a website that hp.com never decide to mention  (i would love to be a mind reader) however within thirty minutes provided you cloud enabled your printer beforehand i had ink setup.  See how that goes in a future post.

hp printer with shit tls

The ink is not as exotic this time and easier to buy outright too.

I suppose i need to find a real manual.  Ho hum it is not to be found on the cdrom disk for macs and windows users.  Sigh. What are hp thinking ?

Despite not being on the disk it is a pdf very much for windows and mac users.  If your me then most of it is useless.

Something also to guess is setting a password for the printers ip address done via a browser with the user name ‘admin’ this feels ripe for shodan.io to scan and abuse as is there way.

The tls needs an browser exception or no password set.

Five hours later and i think its setup, but not for google print cloud. Oh well.

*not in the sky but internet clouds

Debian jessie to stretch server upgrade notes

I did basic workstations here (my blog) and there (my blog) note the networking issues which is also pertinent to servers Some workstation issues of help discovered early on where

virtual box

Needs help form the incompetent fools at oracle (see wiki) as the kernel modules are now outside the remit of debian support – speaks volumes of oracle.whom generally turn most things into a disaster like java (my blog).- can you wean me off virtualbox with some other manager suggestions welcome in comments.

systemd

needs a grub config line and a TTYVTDisallocat=no in otherwise you have no idea if it works or not.when loading you get some messages but without systemd config you will know about a few things see the wiki to set up [not hard]

The server entailed lots of backups and copies of old data all over the place just in case thing go wrong.

Day 1:

After backups change your sources file to stretch, update and apt -f full-update..2784 packages later (3 hours) i had a debian stretch os installed, cannot really call it a server though as fail2ban, postfix, imap and apache barely work.

dns,postgresql and ssh kind of worked though

Sisyphus is still a role model

I think upgrading from i386 to i686 caused the zoo a lot of issues, apt autoremove did not help and i had to remove over a two hundred remains of jessie packages.via aptitude.

Apache2 – cant do cgi and my cgi files did work in jessie

Fail2ban – honestly no idea what is happening here, deinstalled it

Postfix – missing loading four other postfix instances

Opendmarc – is mia

Cyrus – the jump from 2.4 to 2.5 means foo becomes foo_bar – your config files need massive changes, need to reconstuct databases too.   if you know what a DBERROR db5 is then your doing better than i am

Good news printing (cups) works and networking [see above link] survived.  I considered that a win

Day 2

With a fresh pair of eyes, i ‘fix’ crappy virtual box and discover to my delight that the zoo’s cron jobs still work.   I need to remove that trash for something better that does a virtual memory space when i test things.

Cyrus Imap

Gets weirder and werieder

  • it listens on http port 8008.(REALLY)
  • mboxlist and deliver commands seem to be not used
  • sasl logins are from the twilight  zone

I got a paired down (brand new config) that kind of works although three zoo domains cannot open the mail.

poledancing

Postfix

goes to version three expect to use one of these (not here)  If like the zoo you have more than more postfix instance then your need systemd to start it as the init.d scipt is dancing with the faeries and now only loads /etc/postfix.

spf

defaultseedonly becomes testonly – spf has to have some kind of issue and alert you about with debian upgrades or you never know your doing one.

Day 2 was kind of a success.  Even if the mail was flowing in via my actions rather than a systemd startup action and postfix and cyrus kind of work i think.

I began to feel that debain might work rather than simply tell me that estortic_command_lines may have changed.

Day 3:

sasl

Issues are caused by old sockets in cyrus.  Go to your imap mail store directory and the sockets directory and delere.  I did not have to delete *.db’s but even after a reconstruction its not explictly something advertised.   – I appear to be able to receive and process inbound mail

postfix systemd

With the magic of a console i started other instances of postfix and it appears to work my additional systemd scripts dont work one shows a bash shell and the others no bash shell – i hate systemd.- i might need postfix-multi but do not like the idea of it with my existing config,

Day 4

Good news first – amavis seems to work no issues, and now back to problems

apache cgi/mailgraph

I have weird apache error codes but not a meaning as to what they mean i think

  • ah000128 start
  • ah000169 restart
  • ah001909  ssl mismatch (warn)
  • ah002811 script alias issue ?
  • ah000094 ?

google searching for those is a miss they like 404 error codes  – cgi is well broken but that seems down to perl -i had to get rid of perl -wT and run perl -w so getting there.

Moving mailgraph.cgi to cgi-bin fixes the issue (we just need the images which are called via javascript url method).  I gave up /usr/lib/cgi-bin and did cgi-bin my own way.

tls/apache

appears to work unhindered like Jessie not a fuckup

fail2ban

Apparently does work – just reconfigure from scratch

Bind

version 9.10 apparently means it now do caa records without encoding, it has a geoip feature that it loves to advertise.

Conclusion

worst thing: cyrus imap

less worst thing perl ‘changes’ (cgi)

stuff to still fix

  • clean out etc old entries
  • postfix start up ‘issues’
  • remove on disk backups
  • opendmarc reporting is not working
  • check email sending with dkim (works locally)
  • postgresql 9.4 refuses to load but the 9,6 version meens i do not have load it twice – a bit botched but progress

notable mentions to spf – good to see that i still had to change something.

Hope that has not put you off but that was my rather fraught upgrade experience.  Perhaps i should have gone from Jessie i386 to 686 and then to Stretch.

I can work on the issues at a more leisurely pace now

Opendmarc final thoughts (no knock knock jokes)

Carol Beer little britain says computer said no

Carol Beer little britain says computer said no

Generally works although some dmarc records do have errors in (my blog) and since its computer to computer chat nothing much we can do about it expect to bounce more email due to not your problem..

adsp (my blog) fixed reports getting sent out and a feedback loop can exist between hosts.  I send reports every two days.

import of records to sql is needed if you restart the daemon on say a reboot is a once a month occurrence here, otherwise the history is reset, expire is a waste of time since your waiting 180 days (six months ish)  for it to start so that’s a cron job but runs nearer the time of when an expiry is needed otherwise it just wastes cpu and memory if your new to reporting.

Eventually i renamed our dmarc socket from 54321 to 8893

Dmarc is missing an opensource report analyser – not something i really want and if you have a gmail account then you too have dmarc silently sending stuff back to the mailer.

scumbag spammer Robert Soloway

scumbag spammer Robert Soloway

Many marketing sites have dmarc setup so it is quite creepy – however if the zoo flags email as spam it is still unseen by a carbon based human so delivery should not automatically mean seen and acted upon hitting the correct address does not mean that dmarc can confirm it hit the users mailbox.. It hit the mail server but got rejected by it too.

Dmarc does confirm mailbox user seen, only it was logged

Seems to work, although rare to get reports.I like dmarc.

Retards with dmarc pure360.com, and mailinblue.com (sendinblue.com) are deserving of having the reporter from ignoring them since it takes a week for some scumbag to clear (or not bother to) there gmail reporting address.

Here is my ignore retards with dmarc list

opendmarc-reports –nodomain=pure360.com,quantumaccountingservices.net, x157.902sale.com,telegraph.co.uk,thamesonline.in.net,leadexec.co.uk, best2017.site,rac.co.uk,waitmode.com,acumenaccountants.net, mailinblue.com,pure360.com,aztecevents.co.uk, confused.com,email.inboxpro.net

Do you have other domains worth blocking ? please comment

I may return to dmarc when i upgrade debian.  and i found some sql import packages  too.