Don't get me wrong - I use and advocate the use and support of closed source software any time I feel it is appropriate to do so. But it's not that often these days.
But I don't think the following scenario would have been possible using closed source equivalents, and I am damn sure that the cost of using them would have blown my budget out of the water!
I run a network. It has over 400 users, many of whom are mobile, work abroad for extended periods of time, and work all the hours God send us. Planned downtime is a rarity. Unplanned downtime is happening more frequently, but due to outside problems (power outage, internet congestion etc) rather than internal problems - although we have our fair share of those too!
I use Open Source software wherever possible and I do so because it is generally a better "fit" for the network tasks I have than some proprietary software. And I can usually bend it to fit what I want - I can't do that with closed source.
So when I get an OS solution that works, I tend to generally leave it alone. Oh, I apply security patches, but rarely do I update anything that's working unless I need the new feature(s) or they come with a security update.
That's why you can find installations of Apache 1.3 still working on intranet machines, why I still have working Slackware 11 installations and why some un-maintained programs are still doing the business on the network - they work and they are on internal machines with no security implications.
So when a power outage along with a faulty UPS takes out a machine that has been working steadily for the last 5 years as a dhcp server, a nat box, a wireless sign-on web page, a transparent proxy and a router for several private IP ranges, I take the opportunity to upgrade the hardware and software with thanks. When it happens on the Friday of a long weekend ( Friday through to Tuesday ), I am even more thankful for the opportunity to work on it uninterrupted.
Here is the setup:
Hardware: 4 disk rack mount 1U box with dual Athlon processors and 2 gigabytes of RAM ( A bit light these days, but should be enough) and 2 disks only installed
Software: Slackware64 13.1, standard full install. Main packages are Squid, Apache, dhcpd, dnsmasq, and some custom start up scripts for adding addresses to ethernet cards and starting iptables with the nat table entries and port redirects for the transparent proxy.
The process went something like this:
Install Slackware. ( 30 Minutes )
Get dnsmasq working as DNS server only
Get dhcpd (installed version) working. (15 Minutes )
Get Apache in default mode working then configure for my defaults. ( 15 Minutes )
Get Squid. Get Slackbuild script for Squid. Compile Squid. Install Squid ( 45 Minutes )
Read Squid documentation (BIG package, lots of changes since I last used Squid in anger!) (4 Hours )
Implement necessary changes to Squid configuration, test, and repeat. ( 12 hours, including Internet searches, reading Blogs, Wikis etc. )
( Transparent proxying using Squid was a hack in Squid 2, that has been elevated to "built-in" in Squid 3 - but judging by the Blog pages and wiki's it is problematic in Squid 3...)
Curse Squid (5 Minutes)
Get a copy of Squid 2, try to compile in 64 bits on target box. (2 hours, failed )
Curse Slackware ( 30 Seconds )
Find, install, configure and test an alternative to Squid for transparent proxying (TinyProxy )( 1 hour )
Install, test, debug and eventually modify the PHP pages for the wireless page signon. ( 3 hours)
Test all functions from various areas of the building ( 4 Hours )
Total time taken: ~ 28 hours on the software, spread over 2 days.
All the software was available, free and easily downloadable - no feature crippled demos, no limit on the number of connections/users/CPU's, nobody upselling, nobody bombarding me with phone calls/emails for stuff I don't want/don't need and am quite capable of finding for myself if or when I do, and no expiry date where they get a chance to do it all again in 12 months time.
And that is why I will put up with the occasional failure (looking at you, Squid*) in the Open Source model - they don't market this stuff, they just make it useful!
(* By the way, I am quite happy for Squid users to prove me wrong - it is a BIG package, and has over 170+ options, so there is every chance that I screwed up and not Squid - but TinyProxy went in, I did a minimal config and it just worked...)
Showing posts with label Linux. Show all posts
Showing posts with label Linux. Show all posts
Monday, 2 May 2011
Tuesday, 16 February 2010
It's a jungle out there...
For my sins, I work as a System Admin for a reasonable size Department in large tertiary Institution (well, for now - but that's another story!).
One part of my job involves specifying and commissioning new servers and storage. Lately we have been using iSCSI storage servers attached to computational servers for the likes of geophysical and engineering applications such as high-rate GPS data processing. These servers are typically 8 core 48 megabyte machines and the storage attached them is measured in 10's of terabytes.
Normally, on receipt of a new machine the installed operating system is wiped, our preferred OS is put on it and it is run through our check procedures before it goes anywhere near our network. This time, I bought an appliance storage server - one controlled not from the command line, but from a web interface. Consequently, it wasn't wiped on arrival (and it had a 16 terabyte array on it that would have taken days to re-initialise). The upshot of the matter was that this appliance was not vetted as thoroughly as it should have been - I accept total blame for this, and can only plead advancing age and senility.
When the email from our security team arrived some 40 hours later to say that this machine was port scanning , I slapped my forehead, pulled the network cable and started a console session to confirm how stupid I had been.
The rootkit was, fortunately, not very smart. But you don't need to be smart when the box you infect has the root account activated with a password of 123456 - that I didn't change on arrival. The scripted login contacted a website in Rumania, downloaded a tarball, moved it into the /var/tmp directory, expanded it, then proceeded to run various commands designed to start ssh scanning sessions from this machine across the internet.
Careful checking on the contents of the rootkit and some careful monitoring of the rootkit in a virtual machine revealed that no modified binaries were installed, and the damage was limited to the ssh scanning sessions.
The machine is now on a private network, attached only to the computational server, with firewall rules to stop any errant ssh sessions - not that there has been any attempt to start any more.
The moral of the story is that any computer that is prepared for connection to the internet must be treated as if it could be infected from minute one.
Right, back to writing out 10000 times " I must change the root password on all new machines before attaching a network cable"
One part of my job involves specifying and commissioning new servers and storage. Lately we have been using iSCSI storage servers attached to computational servers for the likes of geophysical and engineering applications such as high-rate GPS data processing. These servers are typically 8 core 48 megabyte machines and the storage attached them is measured in 10's of terabytes.
Normally, on receipt of a new machine the installed operating system is wiped, our preferred OS is put on it and it is run through our check procedures before it goes anywhere near our network. This time, I bought an appliance storage server - one controlled not from the command line, but from a web interface. Consequently, it wasn't wiped on arrival (and it had a 16 terabyte array on it that would have taken days to re-initialise). The upshot of the matter was that this appliance was not vetted as thoroughly as it should have been - I accept total blame for this, and can only plead advancing age and senility.
When the email from our security team arrived some 40 hours later to say that this machine was port scanning , I slapped my forehead, pulled the network cable and started a console session to confirm how stupid I had been.
The rootkit was, fortunately, not very smart. But you don't need to be smart when the box you infect has the root account activated with a password of 123456 - that I didn't change on arrival. The scripted login contacted a website in Rumania, downloaded a tarball, moved it into the /var/tmp directory, expanded it, then proceeded to run various commands designed to start ssh scanning sessions from this machine across the internet.
Careful checking on the contents of the rootkit and some careful monitoring of the rootkit in a virtual machine revealed that no modified binaries were installed, and the damage was limited to the ssh scanning sessions.
The machine is now on a private network, attached only to the computational server, with firewall rules to stop any errant ssh sessions - not that there has been any attempt to start any more.
The moral of the story is that any computer that is prepared for connection to the internet must be treated as if it could be infected from minute one.
Right, back to writing out 10000 times " I must change the root password on all new machines before attaching a network cable"
Subscribe to:
Comments (Atom)