A friend found a 2 GB SanDisk Cruzer left in a Walmart shopping cart. There wasn’t anything fun on it, but it did have an annoying auto-run program on it called U3.

I’m trying this method from a post at:
http://noisetheatre.blogspot.com/2006/08/uninstall-u3-and-free-your-usb-drive.html

—–

Peter wrote at 21 January, 2009 05:21…
So hopefully someone will find this useful. After lots of googling I found that there weren’t any instructions for removing U3 under linux. Truth be told, it’s really easy, but the solution is as obscure as it is easy.

1)Mount the U3 “cd” partition
2)Run Mount to find out the name of the device that U3 is on. It should be some thing like scd#, the important part is the number there.
2.5) Just to be sure you’ve got the right device check that /dev/sr# is a symlink to /dev/scd# that you just found.
3) Now that you know which device you’re looking for you can start the actual removal. cd to /sys/class/block/sr#/device/
4)In this directory is a file named delete, it’s write only by root, and if you write to it (I’ve only ever tried with “1″) the U3 partition will be removed. With root privileges ‘echo “1″ > delete’ removes it quite nicely.

——
Here was my results:

1) I just plugged it in, it automounted
2) jim@mickey:~$ mount
… /dev/sdb1 on /media/disk type vfat (rw,nosuid,nodev,shortname=mixed,uid=1000,utf8,umask=077)
2.5) jim@mickey:~$ ls -l /dev/sr0
lrwxrwxrwx 1 root root 4 2009-05-12 07:04 /dev/sr0 -> scd0
3) jim@mickey:~$ cd /sys/block/sr0/device/
jim@mickey:/sys/block/sr0/device$ ls -l delete
–w——- 1 root root 4096 2009-05-14 11:32 delete
jim@mickey:/sys/block/sr0/device$ sudo -i
root@mickey:~# cd /sys/block/sr0/device
root@mickey:/sys/block/sr0/device# echo “1″ > delete
root@mickey:/sys/block/sr0/device# exit

After all that, it didn’t appear to do anything. I must be missing a step. I plugged it into a Win2k machine, and the U3 launchpad came up. I removed the software using the uninstall feature of U3.

So I guess that was a bust, but I’ll have more drives in the future to try this with.

I wrote these comments in response to Søren Hansen’s post about the future direction of Ubuntu Server development. I’m thankful for the recent discussion started by Thierry Carrez.

My wishlist:

Automatic installation. I’m using network preseeding to install servers and desktops, and it works like a charm. I’ve got apt-cacher in the mix, and when I install a new machine, it pulls only the updates from the Internet. I keep a custom repository, and have been successful in setting things like /etc/network/interfaces by making a small .deb packages to make these customizations. Setting up and adding to the custom repository is a pain, because I don’t understand the tools well enough yet — especially signing the packages so I don’t get security warnings.

Ability to review updates. I’d like a simple way, either through a chroot or virtualization, to subject updates to a vetting process before making it available for installation. Again, repository management tools are needed.

High Availability. Even in a small office, downtime is expensive. I’d like to see packages like drbd-primary and drbd-secondary, where you install each on a different machine, and you have simple redundant storage. Same for essential services like dns, dhcp, internet gateways, font servers, etc.

Guided networking schemes. When setting up a local network, there should be sensible defaults and alternatives to a standardized addressing scheme. IP addresses to use 192.168… or 10.0…? Internal services available at gateway.lan, dns.lan, printers.lan?

File-based LAN configuration. The underpinnings of configuration could be based on files, which can be altered and administered through custom .debs. A database-driven front-end could be used to help define the network, which spews out custom .debs to be distributed through the dpkg updating already in place. If you want to migrate to LDAP or some other “registry” type scheme, have it read from existing files like dnsmasq reads /etc/hosts as a starting point. Don’t force complexity on the network, offer it as alternatives to a simple, robust default.

I’m migrating from a hosted Zimbra email setup to Gmail. When importing my contact data I had some glitches, and wanted an easy way to delete all the Gmail contacts. But no! Gmail only allows you to delete 500 at a time. And to do that, you have to click-click 500 times to select the contacts you want deleted. That wasn’t good for me.

After searching the net and trying a couple of things from posts, I was going to do some major bash scripting. I went out and plowed the driveway, and got a good idea. I’ll just make a little GreaseMonkey script to check off the boxes for me.

I signed into the Gmail account, and clicked the link at the bottom of the page marked “basic HTML”. Then I clicked “Contacts”, then “All Contacts”. I loaded the greasemonkey script shown below, and refreshed the page. The first 400 checkboxes were checked off, so all I had to do was press the Delete button. The page reloads automatically with the next 400 boxes checked. In about a minute, all the contacts were deleted!

Note: I’m using Google Apps for Gmail, your “namespace” below might be different for regular Gmail, I’m not sure.

// ==UserScript==
// @name           Select 400 Checkboxes
// @namespace      https://mail.google.com/a/
// @description    Checks off the first 400 checkboxes; good for removing Gmail contacts
// @include        https://addons.mozilla.org/en-US/firefox/addon/748
// ==/UserScript==

var allCheckboxes, thisCheckbox;
allCheckboxes = document.evaluate(
    "//input[@type='checkbox']",
    document,
    null,
    XPathResult.UNORDERED_NODE_SNAPSHOT_TYPE,
    null);

for (var i = 0; i < 400; i++) {
    thisCheckbox = allCheckboxes.snapshotItem(i);
    thisCheckbox.checked=true
}

Over the past few days I’ve been playing with software called Monit.

Monit is a utility for managing and monitoring, processes, files, directories and filesystems on a UNIX system. Monit conducts automatic maintenance and repair and can execute meaningful causal actions in error situations.

Translated to a simpler phrasing, Monit sits in the background and runs tests that you tell it to on your computer, and sends you an email about the results of those tests. Optionally, it can restart programs that stop working, or do any kind of trick you can dream up based on the results of the tests.

Monit comes with it’s own email sender, so you don’t have to set up anything extra to get it to send you an alert. You will need to specify an email server, though.

Getting Monit to run is very simple. Thanks to no-names.biz, I’ve modified their howto posting to show you how to just get it running on Ubuntu 8.04 (Hardy), and I’ve used nano instead of vim as an easy-to-use editor for the configuration files. Before using this, get familiar with nano. I’ve highlighted any portion where you need to substitute anything unique to you, like your email address:

#sudo aptitude install monit

#sudo cp /etc/monit/monitrc /etc/monit/monitrc_orginal

#sudo nano -w /etc/default/monit

startup=1
CHECK_INTERVALS=60

Ctrl-O to save the file, Ctrl-X to exit nano.

#sudo nano -w /etc/monit/monitrc

set daemon 60
set logfile syslog facility log_daemon

# If you run your own mailserver (use this or the next entry):
set mailserver mail.mycompany.com

#For gmail instead of your own mailserver (all on one line):
set mailserver smtp.gmail.com port 587 username “you@gmail.com” password “password” using tlsv1 with timeout 30 seconds

set mail-format { from: monit@$HOST.mycompany.com }
set alert you@mycompany.com
set httpd port 2812
use address localhost
allow localhost
allow you:password
## Services
## You put your tests here.

Ctrl-O to save the file, Ctrl-X to exit nano.

#sudo invoke-rc.d monit start

———–
If all goes right, you should get an email shortly with the subject “monit alert — Monit instance changed localhost”. Because we used the $HOST variable in the mail-format section, you can tell which computer sent you this by looking at the from: address of the email. If you don’t get an email within a few minutes, well, the aggravation can start now while you fix the /etc/monit/monitrc file, probably by monkeying with the mailserver line.

# tail /var/log/daemon.log

The above command will give you some clues if it’s not working right, as monit will log the errors.

Now the fun begins, as we add tests to the end of the /etc/monit/monitrc file.

#sudo nano -w /etc/monit/monitrc
Scroll down to the end of the file, you can just mash the down-arrow button until you get there.
## Services
## You put your tests here.
check host mycompany.com with address mycompany.com
if failed port 80 proto http for 3 times within 5 cycles then alert
#
check host example.com with address example.com
if failed port 80 proto http for 3 times within 5 cycles then alert

Ctrl-O to save the file, Ctrl-X to exit nano.

#sudo invoke-rc.d monit restart
——
What this will do is check your remotely-hosted website, as well as the little website at example.com. If your website isn’t up in three out of five minutes, monit will email you an alert. I’m also including a check against example.com, because there’s the possibility that your computer might not be connecting to the internet properly. So if you get an email that both are failing, then it’s a good chance your website is still up, but your internal network’s got a boo-boo.

A huge amount of tests are available, and many different technologies have tests written for them. By playing these tests and researching what they do, you will get a huge dose of technology learning across many different topics. Guaranteed.

Configuration examples from the monit wiki
Service test documentation

I’m currently running this one and trying to figure out how best to tweak it to my in-house server:

## Check the general system resources such as load average,
## cpu and memory usage. Each rule specifies the tested resource,
## the limit and the action which will be performed in the case
## that the test failed.
#
check system localhost
if loadavg (1min) > 4 then alert
if loadavg (5min) > 2 then alert
if memory usage > 75% then alert
if cpu usage (user) > 70% then alert
if cpu usage (system) > 30% then alert
if cpu usage (wait) > 20% then alert

I’m excited to have my blog listed on Ubuntu Weblogs, aka Planet Ubuntu Users. I haven’t missed a day of reading your entries for a few months now, after I discovered my favorite newsreader, Liferea.

I’m a private accountant in Maine U.S.A., and have been working with computers since 1986. I have many interests, and am enthusiastic about Ubuntu — especially the LTS releases. I run Ubuntu exclusively at home, and leverage it just as much as possible at work, though I do have to use Windows for several applications.

My past blogging has been long diatribes about the experiences I’ve had with projects I’ve taken on. I’ll try to put a few special, shorter tips in for you from time to time as I run across them. You’ll see me around on Ubuntu Forums and IRC as jimcooncat.

My current project at work is setting up a two-server system with a shared data partition (drbd), and I’ll be adding a healthy dose of KVM virtualization.

At home, I’m learning to build associative databases using SqLite and Tcl/tk to use for automated publishing. I plan on opening a very small business called “Cooncat Publishing”. I’ll be mostly repurposing publicly available data in a wide variety of formats and media.

Here’s some of my earlier Ubuntu related entries I’ve made before getting on this Planet:

Almost everthing has purpose, including Microsoft Windows

August 19, 2008 at 5:09 am

I work with two computer operating systems every day, Ubuntu Linux and Microsoft Windows. I post new things I find on web forums, both problems I’m having and tips for others. Whenever I get help from someone for my problems, I try to help at least one other. Once in a while, I hang out in IRC channels for a quicker fix of my community addiction….

A grand computing scheme

September 23, 2008 at 5:33 am

The plan

A few years in the planning, with several false starts, the dream of a smarter computing environment is taking shape. Like many small businesses, ours has a central point of data that is crucial to the business’s survival. Financial data, correspondence, photos, the publications we generate, and contact databases are our office’s lifeblood, and reside on a shared drive on a Windows computer….

Agonizing’s over, accept success when you can

November 16, 2008 at 6:26 am

When you don’t know where to start, attack everything at once.

I’d say the heading speaks for itself, yet a new project of this magnitude always seems to boggle me. Other things it my personal life were also in flux, as much as they are now I’m settled. So instead of setting real goals, I had to come to terms with all the relationships that are involved in a reliable, robust computer network….

—-

Thanks for listing my blog!

When you don’t know where to start, attack everything at once.

I’d say the heading speaks for itself, yet a new project of this magnitude always seems to boggle me. Other things it my personal life were also in flux, as much as they are now I’m settled. So instead of setting real goals, I had to come to terms with all the relationships that are involved in a reliable, robust computer network.

So, mulling over what I’ve learned about technology over the last umpteen years, I agonized. Over stupid stuff: users, groups, hard drives, network cards, virtualization, file systems, updating, customizations, preferences, and so many more things.

Don’t bite off more than you can chew at a time. Especially if it’s not appropriate to spit chunks.

Well, I rechecked the preseed setup I had installed on my computer to start with. Now preseeding, when it’s set up well, is a magical beast. But like most magic, it has some serious problems with fixes when things don’t happen as you expect. To explain, preseeding involves setting up a computer automatically, so if you have to reinstall everything on the computer you can do so quickly without remembering complicated steps. Or referring to half-assed notes you took before.

Therefore I set up two of the big computers to automatically load the Ubuntu Hardy Server operating system. In it’s current state, it’s a very open, stable, practical system with the option of easily adding fancier features that make it robust. But the most advantageous feature is what it inherited from the Debian project; the apt package management system.

So I got the computers to be able to be wiped clean and reloaded with a basic operating system in about seven minutes. I played with this over and over again, adding a few tweaks as I went. When things went well, I’d hit the backup button I installed to make sure my work wouldn’t be in vain if a hard drive failed. Looks like I was in sight of my goal, to be able to install a new server with a minimum of steps that a non-tech would be able to follow by purchasing the right equipment, plugging it in, and not have to make any decisions to get to a working state.

I hit a big snag when I installed two more network cards into the computer, which only had the one attached to the motherboard. I had searched for hours trying to find good cards that would work with Linux and give me the most bang I could out of the office wiring. It was my plan that I was to use one of these to install from, yet during the installation process there was a problem. The card tries to connect to the network twice during the process, and the second time through it couldn’t connect. It was maddening!

Network card Russian Roulette.

Fortunately, the Ubuntu installer has some very nice features, and I found that pressing Alt-F2 I could log into the system at the point it was hung up; and Alt-F4 to see all the messages the installer had spit out. It took me a while, but I realized that it was switching network cards on me between the first and second connection. I googled for hours to find the right answers — I couldn’t have the computer switching network cards on me in some random manner. If I rebooted, or the power went out so long it had to shut down, then I would be the only one able to get it back running correctly. So much for reliability — I felt I was falling back into the hole I’d been spending all this time, and the boss’s money, to dig out of.

After many reboots, I discovered that the computer would assign the network cards at random. Usually, the one on the motherboard would get named “eth0″, but occasionally it would end up as “eth2″. And the other two cards posed a problem as well, in that all I could see was their make and model; since I’d bought them at the same time from the same place, they were indistinguishable.

So as I googled my fortieth page to find some answers, I discovered that the nice folks that provide Ubuntu had solved this problem with some more magic called udev, but it only kicks in after the system is installed. That’s great, but I do wish that it had been more obvious; the configuration file is buried deep in the system. I do have to remember that if I change out cards that I have to find that file again, or the new card will be mounted in a new place, rather than the system automatically replacing the one I’ve taken out.

As much as I hated to, I resigned myself to installing from the original network card that came with the machine. It did have me rethink my security plan, and after agonizing over that for a while I realized that I just had to document plugging a cable in here to start, there when it’s ready to become a second router. No biggie, and it’s embarrassing to worry too much about such a minor thing. But if you know me, that’s not my nature; I cogitate on puzzles until they are solved or something else forces me to abandon it. Then I realized that this whole situation was forcing me to unplug the computer from the jack that goes to the internet while the installation was taking place.

Computers have no soul, and much as you’d like to be friends with them and assign human characteristics, they’ll never watch your back. Yet, this whole project has developed some life of its own; or it seems that way to me. Resolving this problem forced me back to what was smart, and made me realize that the one thing I’d always done during Windows installations — unplugging from the public network — was always a smart thing to do. Even for an operating system that we may feel secure with once it’s up and running.

Hacking is fun, and will cause you much heartache.

So now I had a good basic system going, and it was time to decide what my next move would be. Well, slick as a new Ubuntu installation is, it doesn’t make much of a server out of the box. That’s because any software that talks to the network, that is any process that uses outgoing ports, is disabled even after installation. That’s a very smart move on their part, as the liability of opening a port to the outside world can potentially be huge — or at least screw up your week.

But server software isn’t very useful if it’s not serving, and it’s left to the admin to figure and configure it out. Normally, the process goes like this: you install software, configure it and all the related pieces, it doesn’t work as you expect, you try some features in it you think you may want, you find out you don’t need all that junk, you get it working, and you make some comments along the way if you think you need to do it again.

It works for now. A year later, you install a new computer or some software that changes how things are set up. You go back to what you’ve done before, and remember how clever you were at the time. When you look again, you find you can’t get back to the mindset you were at before. What seemed obvious at the time when you had originally installed the software you’d left out of your comments is now coming back to bite you. And some of the relationships to other software are no longer applicable. Almost always, it ends up feeling simpler just to start from scratch again, so you do.

You know that putting in good comments and achieving a consistent state for others to follow is the right thing to do, but at that point you feel as if you’ve wasted your time. If you’re taking over someone else’s work, its much worse unless they wrote an entire step-by-step how-to document. It’s much better to configure software as an installable package, and keep consistency with the rest of the system. Keep commenting and provide a list of changes you’ve made to it over time, but you really only need to fully explain what makes your customized version different from the default. So, when it comes time for someone else (or you at a later stage in your life) to consider whether to use this version or go back to the drawing board, they only need to evaluate if your package does what they want it to do; and you make it easy for them to make that decision by keeping your notes very simple.

Enjoy the nice things that are right in front of you.

Ubuntu inherited something very nice from the Debian project; it’s not only an operating system, it’s a software publishing platform. Much thought and care has been put into the publishing tools, and hundreds of little helper programs are available to round out your setup. You can list these programs as “dependencies” to your software, and call on them to provide services so you don’t have to write the functionality yourself. That’s great if you’re the official maintainer of the software package, and it works extremely well for their developers.

Now I want to make packages for myself, to set up this new network automatically to a custom configuration. It turns out that the huge effort to develop the operating system does not support this well with how-to documents; or rather that these documents are huge and unwieldy as they address things such as how to submit new work to the society for inclusion in the distribution.

Here’s the choice I was facing: either do things the old fashioned hacking way, or learn how to apply what was already available. I thought some more about how things are normally done by system administrators to maintain and distribute custom configurations. I was looking at developing my own way of pulling down a custom configuration, applying it to the software, restarting the software if needed, etc. Or I could use tools others had developed to do similar things, having to learn their arcane nomenclature and understand their mindset. As I looked at this hard, I could see very clearly that a different system would require much more work on my part than just learning how to do it right in the first place.

Old dog learns new tricks.

So after poring over more how-to documents and bugging the nice people at the Freenode’s #ubuntu-motu chat room, I’ve realized a very simple way to make and distribute software configured to my needs. Note that this is just a starting point, I’ll be refining and better documenting this method as I learn more.

• Start with a clean working system.
• Install the official package you want to customize.
• Tweak the configuration as desired, making comments along the way.
• Test, re-tweak, and test again until all bugs are resolved or acceptable.
• In an empty directory, run “sudo dpkg-repack –generate”. This will make the guts of a .deb package with all your configuration changes intact.
• Edit the DEBIAN/control file to bump the version number up higher than the original software.
• Run “sudo dpkg-deb -b <your directory>”. This makes a file called “.deb”, which is a hidden file.
• Rename the .deb file to a standardized package name with your new version.
• Copy the renamed file to your repository.
• In your repository, run “dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz”.
• On the computers you want to install or update, make sure your repository is listed in “/etc/apt/sources.list” or “/etc/apt/sources.list.d/”.
• Run “sudo aptitude update”. This will make your new package available.
• Run “sudo aptitude purge <your package>” if there’s an older version of the software installed. This wipes out any configuration files on the target system as well.
• Run “sudo aptitude install <your package>”. That’s it!

If this looks complex, of course it is. The method has many advantages to it though:

• You can install the custom configuration on as many computers as you want simply once this process is done.
• If you want to share your customized package with others, you make a publicly assessable repository and upload/update it.
• You can include your customized package in a preseeded, hands-off install.

I’m missing some steps I’ll want to include once I learn how, mainly how to proper document the changes. That looks very simple, I just haven’t gotten that far yet. And I’ll be automating this process along the way so I won’t have to copy and paste the somewhat arcane commands. Thank you for reading, I’ll look forward to writing another overly-long post on some irregular date in the future.

The plan

A few years in the planning, with several false starts, the dream of a smarter computing environment is taking shape. Like many small businesses, ours has a central point of data that is crucial to the business’s survival. Financial data, correspondence, photos, the publications we generate, and contact databases are our office’s lifeblood, and reside on a shared drive on a Windows computer.

Backing up this drive was initially a nightmare. A consultant installed a tape drive with terrible software, and it was my job to change the tapes daily. This wouldn’t have been so bad, but restoring any file would take a matter of two hours — if I was lucky. I was having a 50% rate of recovery. I suppose that if I had spent another $500 of the company’s cash and gone to a training session on the software, I may have improved my efficiency somewhat.

A couple years later, I switched my PC to Ubuntu Linux. After some research, I settled on a backup tool called rsnapshot. Once I started using that software, my nightmare went away. I was able not only to back up our drive, but in a fashion that was automatic and very easy to live with. It not only stores the files, but also the changes to them, so I can easily go back to previous edits. The backups are on my hard disk, which I trust much more than tapes that stretch and wind and grind. Restoring is super-simple — I just go into the appropriate directory and copy a file, no need to run any special software to restore.

As time wore on, our computers started wearing out. A power supply here, a hard drive there, entropy was starting to catch up with us. Our boss was reluctant to shell out for new hardware, and I was reluctant to keep installing software onto new computers. I’d spend two days reinstalling Windows and all the software we use, only to wind up with an environment that wasn’t an exact clone of what the worker had before — so they’d spend at least a couple days worth of work getting used to it.

I started again to think about that shared drive, and realized that if the machine it was on went south, it would mean some hours of downtime to restore it. Many times in the office it would not affect our bottom line much; but there are hectic moment where downtime would be disastrous. At some point, and I’m not sure when, computers turned from an efficiency tool into a production mechanism. And we need a production-grade environment.

Doing some heavy research on disaster recovery and high availability scenarios, I found that small businesses are greatly under-served by commercial computing vendors. The marketing teams generate huge amounts of hype for bad products that don’t work correctly, or try to take a product made for larger business and shoehorn it into cheap hardware. They didn’t seem to take value into account at all.

Like many in my situation, I played with the idea of getting a big box and using RAID1 or RAID5; that is multiple drives that would allow for one to crash and still be able to work. It seemed workable to me, and then we lost a computer entirely to a power spike. Not only the power supply burned out, but the drive fried as well. A few weeks later, our huge web hosting provider went down. The reason? A RAID controller in one of their servers burned out, and they had to wait for delivery from the other side of the state to get back up and running. All the while, they had been touting the safety and redundancy of their service. I was disgusted.

It was apparent that we needed to not only protect the drive itself, but the whole environment. Single points of failure are not a permanent option for a business that provides my livelihood. Back to scouring the internet for solutions, I came across the Linux-HA project, which provides tools called drbd and heartbeat. These tools make two drives on two separate computers appear as one drive to the rest of the machines. If one of the computers goes bad, the other takes over in about 15 seconds. The only downtime is that the worker would probably have to restart their machine. This was what I was looking for!

After making a few sales pitches to my boss, he agreed to allow me to construct a new server setup. I purchased three new identical machines, with additional large hard drives and filled with 4 Gigabytes of RAM. So there they were, sitting on my newly-cleared-off workbench in the basement — and then I had to wait. Other projects screamed for my attention, and the staff was going through some changes. So there it sat for a full year almost untouched. I am very thankful for the patience my life experiences have trained me for; yet stuck in my mind was this project I needed to build and knew that distraction wouldn’t cut it.

Setting up my computer

Last week I finally was able to start on the project. Ubuntu Linux has progressed to the point where I will be able to have support for the operating system without massive changes for another four years. The hardware purchased has proven to be wonderful, as I had taken one of the boxes and used it to run two virtualized copies of Windows with fair success. And I had done plenty of research, so I understood to a management level what I wanted out of this — but now for the implementation.

The primary focus of this project is simplicity. Taking complex situations and boiling them down to their underlying substance is the kind of puzzle I thrive on. So what I want to do with this setup is to turn each piece into a appliance, which can be fixed and maintained with a minimum of instruction.

So I’ve started by turning my work computer into a “netboot installer”. This is probably the most magical thing I’ve ever seen a computer do in all the years I’ve worked with them. I take one of the new computers, plug it into the network, and reboot it. When it starts up, I press a key (the F8 key on these) and tell it to boot from the network. It finds my work computer, and automatically wipes out everything on the drive, installs the operating system (plus any additional software I want loaded), and reboots. When it’s done, it’s completely loaded with the most current versions of everything; I don’t even need to go through another long round of updates!

Setting up this kind of environment wasn’t simple, of course. There are four pieces of software that I had to install and configure the hell out of on my work computer: dhcpd3 (the network magic), tftpd-hpa (serves configuration files and kernel for the installation software), apt-cacher (temporarily store the operating system files) and apache2 (a web server for the operating system files).

So we take the new machine and plug it into the network. Start it up, and when it first beeps mash the F8 key and tell it to boot from the network card. It’s hands-off from then on if the configuration is right. The new machine looks for the DHCP server that’s broadcasting on a special address. When it finds it, it downloads the installer operating system and starts it up. This temporary installer then looks again at the DHCP server, and gets its first instructions from it based on the new machine’s network hardware address. It then downloads the “preseed”; a series of answers to the questions it normally asks during installation.

I have the preseed set up to reformat the hard drive and set up new partitions. It also does many other things automatically, like telling the computer to use English, a US keyboard, and set the time zone.

It then starts to download the packages of the operating system from the internet. Since this normally takes more than an hour for a server (even longer for a desktop), we have it download through apt-cacher. What this does is to automatically store each package locally on my work computer, so it won’t have to be downloaded again unless it gets a new update. Once you’ve done one install, subsequent installs take only a few minutes; I’m down to seven minutes for a subsequent server install, and I believe a desktop install (with graphics) will take about twenty based on a test I did last year. So far, I’ve only done the first desktop install on this new setup, as I realize this isn’t the current project — but I may need to use it for another computer at some point.

Once the new computer has been installed, it reboots itself and loads the new operating system from the hard drive. So if I had to talk the boss through re-installation of a computer over the phone, it would go like this:

1. Plug the computer into the network jack.
2. Start it up, and press F8 (several times if you want) at the first beep.
3. Press the down arrow until you reach the menu item “NVIDIA Boot Age”
4. Watch for half a minute to make sure nothing gets stuck. If it does within that time, restart.
5. Go do something else for a while. If you hear another beep from the computer, it’s done and will show a login screen.

Customizing

Well, the workbench is in the basement with the new computers on it, but my office is on the upstairs floor. The first day I was working, I was running up and down two flights of stairs to reboot, adding to the distraction of my co-worker at the front desk. Well, I did move the hardware upstairs on the second day, so that’s much better.

My next step was to check out the new operating system environment to see how the installation went. So I added a remote-control program called openssh-server to the setup. Now once I hear the second beep (much easier now that it’s upstairs) I can log into the new computer and check it out.

The installer configuration lets me do some of the customization I need to do the operating system, but not everything. I can add packages to the setup, like I did with openssh-server. But beyond some very simple options, it installs everything to the default, which in some cases means that a program I want started automatically is not set up to. Well, I need to customize, and with that comes a problem.

In the many years I’ve worked with computers, I’ve seen a lot of installations. The tech setting them up would load the operating system, then play around with the settings until they get it the way they want. Most times, they would only document what they believe wasn’t obvious, invariably leaving out an important detail or two. Subsequent installs meant doing the same thing, relying on notes or memory from previous times.

This is no way to ensure quality installation every time. But we have another option, which is to customize the software packages. With the high quality package management that Ubuntu inherited from the Debian project, I can make my own repository of customized packages that will seamlessly integrate with the setup. So one package at a time, I’ll unpack it, tweak it, package it back up, and test it. If I hose the whole computer doing so, oh well — it will cost me another twenty minutes (or less) to reinstall the whole machine. Since I have two machines right now to test the installation, I just work on the other one while the reinstall is going on.

Time to back up

As I was driving home after the second day, I was quite pleased I got the bugs out of the installer setup (there had been many, almost all my fault for not understanding the manuals) and was looking forward to starting the package customization portion. Then it dawned on my that my setup on my work computer wasn’t backed up! First thing the next workday, I installed a simple backup program. Now after I test changes to the setup, I can press a button and make a new backup of my installer setup. Whew!

The third day at this I needed to get some other work done, but I was able to research how to make a simple “software repository” for my customized packaging. It turns out that this is a deceptively simple thing to do, but there was very little available on the internet to point me to the right documentation; so the research took me some time that I didn’t expect.

Next steps

Now I have two major tasks ahead of me, to set up the drbd and heartbeat on the two computers to test it out; and clone my installer setup to the new computers. These are both fairly major undertakings since it’s new-to-me software; but I believe using the methods I have so far will let me plug away at it easily while doing other work as well during the day. I’m also going to be installing two monitoring tools; a generalized package called “monit” that can test for all sorts of things, and a specialized “smartmon-tools” that will notify me of potential hard drive problems before anything actually goes bad with them.

I’m so looking forward to this work!

I work with two computer operating systems every day, Ubuntu Linux and Microsoft Windows. I post new things I find on web forums, both problems I’m having and tips for others. Whenever I get help from someone for my problems, I try to help at least one other. Once in a while, I hang out in IRC channels for a quicker fix of my community addiction.

A good number of people I give help and get help from aren’t blinded by people who detract from what others like to use for an operating system. It’s difficult for us practical people though, because there are many others out there that shout to promote their favourite toys and disparage those who use other, competing wares. It seems that it should be hard to tune out this noise, yet an interesting article will have huge strings of posts fuelled by these “fanbois”; making the few interesting comments that practical people offer difficult to find.

In my view, an operating system is only a means to an end — tools to manipulate some bits into, hopefully, doing what you want. At this point, and for what I can see will be for the next few years, there is no one affordable operating system that can do everything a person needs. (A note to Mac fanbois, your financial priorities are much different than mine.) I use Ubuntu exclusively at home, and for the majority of my work. However, there are a few Windows programs that I can’t do without, such as Quickbooks, MS Access, MS Excel, and Irfanview. Coworkers that I support also have a few other programs that they wouldn’t want to swap out; not being wildly computer-oriented they also dislike changes in the what they see on the screen.

Microsoft Windows (and by that I mean the affordable XP Home, XP Pro, Vista Basic) is best for:

• running software made for Microsoft Windows;
• enabling the use of hardware devices that have only been on the market for a few months;

but it doesn’t:

• give a simple way to seamlessly export its windows to other machines*;
• provide a set of signed repositories that includes most software a person needs;
• allow a technician to update or configure the software without a big production of user distraction and downtime;
• stay easy for a person to keep updated and secure;
• make it simple and frugal to use in a multi-user networked environment.

So what I envision as a workable solution is a combination of machines that offer the best of both operating systems. In an office or large household, it’s quite normal for there to be several machines; and that can be a huge advantage.

Newer machines with lots of RAM can be run with virtualization software (KVM, Virtualbox, VMWare). This allows for multiple operating systems (aka “virtual machines”) to be run on one piece of hardware. For these machines, I’ve been suggesting using AMD over Intel, as it’s simpler to know in advance of purchase that vitualization is fully supported by the hardware. However, I believe most newer computers have no problems doing this or need very quick adjustments to the system.

Older machines, regardless of the amount of RAM, can be used as thin clients to the larger machine. I’m appalled by the amount of computers people discard as junk that can be used for this purpose. Also, I’ve set up and used some very nice new specialized thin client boxes that use little electricity and are super-nice to the wallet.

A combination of those machines takes care of the first requirement, running software made for Microsoft Windows.

Older machines that have a good amount of RAM can be used to meet the second requirement, enabling the use of hardware devices that have only been on the market for a few months.

* A method to seamlessly export MS Windows applications is described at https://help.ubuntu.com/community/SeamlessVirtualization, yet I don’t have the time or cash to experiment with it.

A relationship with the truth

Basis

Source

Once you come to the realization that you’ve committed a falsehood into your knowledge repository, and your basis for an assertion is junk, then it would be very helpful for you to know the source of that basis. If one thing is wrong, why would not other assertions from the same place?

When I think back
On all the crap I learned in high school
It’s a wonder
I can think at all – Rhymin’ Simon

Good enough for this purpose

Creeping Doubt

Re-evaluation

The Polluted Namespace of Database Applications

Wikipedia will tell you about relational databases with terminology that will confuse any person used to the English language. Drilling down to the source of the term “relational”, we find in the article Relational model that

“Relation” is a mathematical term for “table”, and thus “relational” roughly means “based on tables”.

You can keep drilling down through the articles until you get into set theory and start approaching the human usage of the word “relation”, which I understand to be: A logical or natural association between two or more things; relevance of one to another. (From relation.)

I don’t want to knock the work of Dr. E.F. Codd; his invention of the relational database has helped support my livelihood for half my lifetime. All I take issue with is the terminology, and understandably when the terms were coined, there wasn’t the familiarity with how this invention would be used that would lend it to better wordage.

More Questions than Answers

Anyone who has used a table of data knows that there’s a lot of knowledge being presented in a small space. The very simple table below seems at first glance to show the most basic of relationship representation:

   id    name     team
   1     Amy      Blues
   2     Bob      Reds
   3     Chuck    Blues
   4     Dick     Blues
   5     Ethel    Reds

But the apparent simplicity actually assumes a greater knowledge. Boiling it down to pieces, we find that:

• id is probably a unique integer the database uses to keep track of the link between name and team
• name is the first name of a person
• team is the name of a group of people

And so when we use the table, we have lots of questions.

• Can a person be on more than one team?
• Can a team exist if there are no people assigned to it?
• What do we do if there is more than one person with the same first name? Do we put the last name in the name field, or do we make a separate field for a last name? And if there are two people with the same first and last name, what do we do then? If a team changes its name, how does that change the assignments?
• Who is recorded this information? Did they spell everyone’s name right? Did they make any mistakes as to which team the people were assigned?
• Who actually assigned these people to a team? What was their basis for assigning these people to one team over another?
• Are there constraints to how many people are to be on a team? Do you have a minimum number in order to play? Do you have a maximum to play at a time? What do you do with additional people?

The Start of a Good Relationship

One thing that’s obvious to start with is that there are two teams. Those teams could, and might very well want to, change their names to something more sporty — lets say the Lobsters and Clams. So there is a relationship here between the name of the team and the team itself.

So far in this series, I’ve been describing a scheme for Unique ID’s. We’ve gone through the mental gyrations of what the parts of an ID should consist of, but haven’t come to a conclusion yet of the final format. As you’ve seen by now, the more characters we have available, the more comprehensive our ID format can be. With registration, we can also cut down the size to a short string.

Before we cut it down to it’s minimum, though, let’s look at values. I’d like to format to not only be able to store unique ID’s, but also values so that we can map relationships against a number. Once we find a minimal length for useful numbers, then we can determine how to wedge our ID scenario into that space.

Through the many years I’ve been crunching number, I’ve been bitten so many times by software, especially Microsoft Excel, using floating point values. For the unaware, when a program stores a value, it has a choice — it can either store it as an integer, or use much less space by storing an approximation. The software very seldom tells you this, and if you rely on your numbers, your computer lies to you. I just can’t describe how wrong that situation is. By the time you notice the tiny bit off you are, you’re already layers deep in hundreds of calculations. Good accounting programs store numbers as integers, and even in MS Access currency is normally stored as an integer with four decimal places.

Hey, I don’t mind if a computer program can’t store pi accurately, but I want it to tell me that what it displays is an approximation if I need to know that for my purposes. So if it’s an approximation, how was it done? Was it truncated, rounded up, rounded down, an exact half? If a computer makes a mess of things on my behalf, I want to know about it.

When computers give you a number, such as 5,000, there is no information as to how many significant digits there are in that number. If this represents 5 boxes of 1000 paper clips, you can be pretty sure that each box is almost exact, maybe have one or two extra clips in it. But I could have derived this from the number of feet I guessed was equal to one inch I measured on a map. If I make a mess of things, I want to be able to document it.

If I know that a value is wonky, I can, if needed, use techniques such as significance arithmetic to make known how much an end result is off. It makes a big difference if the basis of the number is measured or counted.

Now we can start to enumerate the relationship of a value to a stored number in a computer: the basis of how it was derived, the number of significant digits, the accuracy of the final digit, it’s sign, and exponent. Some of these can be encapsulated in the way most computers store a floating point number; sign, exponent, and fraction are encapsulated in 32 or 64 bits of binary data.

Time pervades all things. I remember in my first Geometry class, the teacher talked about three dimensions, and time being the fourth. This concept bothered me for months, it just didn’t seem right to me. A line, for instance, cannot be if it doesn’t exist for a period of time. And when old teach was trying to describe to us what a point was, the geometric concept of it, my confusion cleared up. A point, he said, had no length, breadth, or height, it was only a point. At which one of my little imaginary friends whispered in my ear, “But it has time. It is a point in time.”

I digress from the subject at hand to make my own point. Mankind’s understanding of the true nature of time is weak. We may think we can go back and forth in time if we can just find a De Lorean and build a flux capacitor out of junk parts in the garage. Or we can accept Stephen King’s explanation of the Langoliers, that eat away the past after the present is gone.

In my previous missive on Unique ID’s, I went through all the mental hoops I could to describe how an ID generator could be itself identified by thread, process, computer, and public parts. Now, we’re only required a series of numbers to finish off the Unique ID. That could be as simple as a counting loop within the thread of execution, and we might want to keep that in mind.

But usually, computer programs have a very simple way to get a time value, and that can come to our aid. At the risk of defying the You Ain’t Gonna Need It (YAGNI) concept, we’ll go on to figure out how we can incorporate time into our Unique ID’s, or at least leave room for implentation if we want.

Suppose we have gone through all the hoops to figure out “the best thing to do” for our computer process, and we have two or more items of equal weight (whatever we’ve defined that to be) in contention to be the winner. We could pick a random choice, or as random as our machine will give us, and hope that luck will lead us clearly to our goals. But if we keep arriving at this same decision, we could end up that a value is never chosen, that our luck has gone sour and karma has caught up with us. So, one way out of this mess is to choose the oldest value.

Having a timestamp at this point in the consideration process is extremely handy, in that we don’t need to implement another decision level. We just pick the ID with the smallest time value. Next time we’re at this juncture, our old value will have been gone, and we’ll pick the oldest ID now available.

One of the downsides of doing this is that we’re now shifting our decision process from a comparison of whole ID’s to a relation of embedded values. That will add additional complexity to our program thread, and we’ll only be able to determine its effectiveness when we finally get to implementation.

If we look back at the original spec for UUID’s, we see that time is captured at 100 nanosecond intervals, or 10 million of these values per second is our granularity. When programming, though, we’re very lucky if we can get anything better than granularity to the second without using an external library or weird procedure. I’ll suggest here, then, we keep our programming simple and call seconds as “close enough” for our purpose. This will probably bite me in the butt later as well, because YAGNI.

So, let’s look at the value that gets returned when we ask for a POSIX time value. It is “the number of seconds elapsed since midnight UTC of January 1, 1970, not counting leap seconds.” If we store this value in 32 bits of information, we’ll run out of space on January 19th, 2038. But this doesn’t bother us, as we’re adamant to store our ID in base 32, and that will get us through the rest of this millennium with seven characters.

Something cropped up when I was originally researching UUID’s for this project a few years back, what do you do when someone sets the clock backwards? Well, we could just ignore it if we trusted our system to check for duplicate values. That’s not in my nature, though, as we should try to do always be doing right thing when dealing with lower level functionality.

The UUID spec says they would use a Clock ID set at a random value, then increment the ID by one if a reset was detected. Well, that’s fine, it keeps the ID unique. But it doesn’t help with the sorting, because you’d have to intersperse the clock ID into the right spot in the series somehow. Thankfully, these resets are few and far between, and hopefully the clock resets on a fairly regular basis.

That just about wipes me out as far as using time as part of our Unique ID scheme. I ain’t gonna use it, mostly because of the shifting context problem. We’ll think about keeping space for it in our final version, as we can capture it with eight characters.

Is a UUID really unique? This post seems to think there are problems. http://blog.joeware.net/2005/06/19/42/

There are several types of UUID’s, the most common being (supposedly) randomly generated, and one based on time and MAC addresses. There are problems with each of these.

The above link has good opinions on the fallacy of randomly generated addresses. Besides bad implementations, there is also a great reason for not using them — they are so damn unique, there is no way to look them up unless you have a database of all which are in use. You can’t specify ranges where you can divide the database into segments.

MAC addresses have nothing to do with Apple Macintosh computers, although the computers have MAC addresses in them. MAC addresses are unique identifiers brazed in read-only on your computer’s network card. Ranges of MAC addresses are doled out by a central authority to network card manufacturers.

There is something obviously wrong with that scheme in that we’re expecting the network card manufacturers to police themselves to ensure that addresses aren’t duplicated. When you pay sixteen dollars for a piece of hardware, made in the poorest manufacturing environment in the world, do you really feel that good that the manufacturer has paid their dues and been in compliance with this directive, that isn’t even backed by law? OK, but just be sure you take that painted wooden play block out of the baby’s mouth.

If that wasn’t enough, there’s a real possibility that someone has set up duplicate MAC addresses on their LAN intentionally. Some ISP’s (Internet Service Providers) and wireless configurations use MAC addresses to allow access to the next network over. Of course, you and I wouldn’t come up with a scheme like that since it relies only on the obscurity of the address, and not a shared secret or hashed password. But that’s the nature of the market.

So what happens is that the customer registers his MAC address with the ISP, and is now allowed into the network as long as he’s using that ethernet card. Later on, the customer trades in his computer, or replaces the ethernet card that’s gone bum, or sticks a router in between his computer and the internet. Whatever he’s done, he needs to change is MAC address; either register his new one with the ISP, or change the new hardware back to the old MAC address. Given the choice, he will probably not want to spend quality time listening to muzak while waiting for customer service to answer the phone and then find someone at the ISP that has a clue.

So, especially with the “insert a router in the mix” scenario, there’s a good possibility that there is a duplicate MAC address on the LAN. Intentionally done, with good intentions. The intentions don’t even have to be good, as one could use this scheme to leech access from the ISP from multiple locations.

Finally, MAC addresses are a lousy way to do generate unique ID’s that represent real-world data. That’s because in order to look up an unknown piece of information, you need a map of MAC addresses to computers. When collaborating with others outside your LAN, you’d have to replace your UUID’s to some central MAC address.

You also have a problem within one computer itself. You may want multiple programs to be able to generate unique ID’s, but there is only one MAC address per computer. So, you would have to have some locking mechanism so that more than one process can’t have access to the ID generator at a time. This scenario is even specified in the UUID specification document, www.opengroup.org/dce/info/draft-leach-uuids-guids-01.txt

Providing a system-wide lock makes programming a tool much harder; harder than it has to be. At a particular time, a process has a unique process id, which the computer uses to keep track and schedule each thing that’s going on. Well, with Linux 2.6 (the current version), there are a maximum of 1,073,741,824 process id’s. Encoding that many numbers in base32 takes six characters. And in the future, Linus and crew might up that number — you would think a million things going on with a computer at once would be enough for anybody!

So it’s probable that when implementing a unique ID with a process ID incorporated, we’ll need to cut this number down to size. That would mean some extra programming in order to have a registry of processes allowed to give out an ID. So we’re back to a locking mechanism, but arguably more effective, since you only need to register the process one time, and provide some way to clean up the mess after you’re done.

But let’s say we didn’t want to make a registry, and just confine our ID’s to the computer we’re using right now. After all, we don’t need to come up with an ID for the computer until we’re ready to share the information outside of it, right? So we can flag the ID with a code that says we’re only going to have it valid on the current computer, then list the process ID after it. We could even have a multithreaded program be able to give a short ID to each thread; creating a in-memory mapping within a single program would be fairly easy, you’d just map the thread id when creating the thread, if mapping is really needed.

Now, if we’re not using MAC addresses as the primary way to identify the computer, what’s the best way to go about it? More registries, please! We’ll want to make some kind of global registry so that an unknown value can be looked up, and if the author wants to share that information, then you’d be able to get its definition.

At the public side, we may want to consider a range of five characters, which our base32 converter tells us will provide over 35 million identifiers. So that’s how many public domains would be available, and is still in the realm of a Sqlite database lookup without making a small computer go dizzy. Five’s a good number, since a sequence of characters can be read off over the phone and copied down without the recipient losing track of the characters spoken.

Now we have to have some layers of redirection from the public identifier to the individual computer. For one, this gives us some sense of privacy. But even more important, this allows a large organization to give a level of autonomy to its divisions.

Two base32 characters provide up to 1024 values, while one character is 32. So with five more characters, we can allocate one character to the process id registration (32 processes per computer), 1024 computers per division, then 1024 divisions per public identifier. Now, I’m curious if that would be a good spread. What do you think?

Another way would be to have 1024 process id’s per computer, using two characters, then let the public ID have 32,768 computer id’s to dole out as they see fit. Any more than that, they’d just need to get another public ID. Now I feel better about using that spread.

If we wanted to go whole hog, then five characters would be enough to directly convert internal IP addresses to computer id’s, no matter if they used the 10.x.x.x, 172.x.x.x or 192.161.x.x scheme, or even a combination of those. But now that we’re at our five-character maximum, we have to allocate even more characters for our process id. And if the company started using sub-sub-nets, which may be common with virtualized OS’s, internal IP addresses can very possibly be duplicated, as I’ve seen with VirtualBox.

For giggles, the full range of IPv4 address space comes to 4000000 in base 32. Some expect exhaustions of address ranges as early as two years from now; but I just don’t think it’ll happen for a while. But if the world moves to IPv6, that’s over ten with 38 zeroes after it — way to big for us to make a usable ID format out of anyway.

Well, I think that about covers all the scenarios we’d want to identify who generated the ID. Note that I am not making any effort to conceal privacy, if one wants to keep the data anonymous then they will have to arrange for an anonymous computer to do their dirty work. I would rather forsake simple anonymity than to make our ID’s so that they are incompatible with any type of accountancy.

Now we need a series of unique numbers for our ID generator to use. Which will bring me to my next post, about the strange concept we call time ….

The ultimate flexible program starts out totally ignorant. It uses a database to determine “the best thing to do at a particular time.”

The database returns: what to do (action), and how to do it (method).

How to do it, also know as the method:

• Do it and report success or failure.
• Do it and report what happened at each step.
• Do it and report how long it took, along with success or failure.

Success or Failure reports include:

• I don’t know how to do that action.
• I don’t know the method in which you want me to do that.
• I don’t know how to do that action, nor do I know the method in which you want me to do that.
• I tried to do that action, but it was unsuccessful. The method was successful.
• I tried to do that action using the method you wanted me to, but the method was unsuccessful.
• I tried to do that action using that method, but something went wrong. I don’t know if it was the action, or the method used.
• I did it. Success.

Over the many years I’ve been using computer programs, I’ve been disheartened by how limiting most databases are. A programmer’s job is to make a useful tool, and if that tool is supposed to model the physical world, it needs to be able to capture more than just enough to supposedly do the work it set out. Even if the tool doesn’t use the data, the user always has a need to record their thoughts as they are working on something, and it’s not very expensive to at least allow a “comments” area for anything they may happen to dream up while using the tool.

It mounts the frustration of people using a program to find its inadequate for their needs after they’ve already invested a load of data entry into it. We’ll use a contact database for an example. Let’s say that one of the co-workers is going on vacation, and some new vacation contact information will only be good for a set duration, perhaps a phone number they use. What would be wonderful is if they wanted to call someone during that period the database would give the vacation number, but after the vacation’s over it would revert back to the old number. But if putting in that functionality is so difficult, why not give the database the tools to capture a message, and possibly incorporate an email reminder system?

As needs get more complex, we wind up with multiple records of the same thing across many databases. A person’s records in our office is in our contacts database, and in our accounting systems (up to three times, if they have multiple relationships to the business, such as customer, vendor, and employee). They might also be a member of one of the organizations we do staff support for, hence a record in yet another database that provides functionality our accounting system doesn’t do. When that person calls in to be helped, there are far too many places that we have to look to retrieve information on them, and it is a burden to update if they have a change.

The solution is to recognize that these databases don’t have a record of the person themselves; they have a record of the relationship between that person and “yourself”, as defined by the program. In the physical plane, your needs are greater than one programmer’s vision of what their tool is going to provide. They don’t recognize that, and ego plays a lot in it — they feel that somehow their tool is going to give you everything you need to manage what you have going on. Yet most all programs have very little capture of your own information, for some reason they objectify you as the luser, and some even punish you for all the trauma that programming for your needs has cost them as a programmer.

I’m trying to come up with the ultimate simple programming environment. I’d like to be able to support most any type of user interface, such as CLI (command line), ncurses (maybe via dialog), zenity, tk (I think it’s ugly, but might make a good fallback) and wxwidgets. And I’d like to be able to daemonize it simply and use it as a web server, or to be able to called by one.

I’ve been debating TCL vs. Python. Python is very nice, but it seems that I would have trouble making small executables with it. One of my main goals is for the deliverables to remain small as possible. A program call freewrap is advertised as making small executables, for which I could make TCL programs, and (optionally) a TK interface. Unfortunately, it seems freewrap doesn’t appear to work with the newest 8.5 Tcl/Tk, so therefore ugly GUI without Tk’s new theming. But maybe it’s still possible, or will be by the time I need it.

Since I want to make a generic view (from the Model, View, Controller pattern), I might still be able to use wxwidgets (and probably Python) if I want to deliver a beautiful interface — at the expense of deliverables size. How I’d wrap this into an installable deliverable I don’t know yet. That’s later, anyway, as well.

Sqlite and Tcl seem to be a very nice fit for what I want. D. Richard Hipp has some wonderful ideas about using Sqlite for a generic storage backend that would replace use of arrays; becoming part of the code itself. This goes along with my philosophy for a super-generalized computer program.

The program itself would know very little, not even its own name. It’s function is to run a loop: What’s the best thing to do, then do it. It would provide error messages if it couldn’t initialize. And that’s about it. The database would load code as needed.

I spent a good part of this morning reviewing Expect, as it has a signal handling functionality. It’s important that anything I write (and should be anything I do) maintain its cleanliness. So if there’s something dirty that should be cleaned up before exiting, Expect can trap SIGTERM (hopefully on Windows too) and run a cleanup function. It looks now as if there’s nothing dirty about opening the database, even with a table that’s only in memory. If so, I could load the cleanup functionality from the database, and keep it very simple.

Data would be stored as either a unique ID or as a value. From the past research I’ve done (which I seem to have misplaced at the moment), I came up with a scheme using 32 numbers and letters, which I called ZILO encoding. I named it that, since those would be the letters I would omit, since they look like other letters or numbers. Using 128 bits plus a few more, I’d end up with (if I remember right) 30 digits that could be handwritten and reread without error. This would store a GUID, a big integer, or just raw data.

Part of the ZILO encoding is the ability to sort data, which the default GUID format doesn’t lend itself to. Important data that’s part of the GUID should be available: where the ID came from, and what time it was when formed. Note I don’t subscribe to random GUID’s as MS devs do, as they are not truly unique.

The basic table format I described came in three parts: an ID, a subject, and an object. A representation of any unique or commodity object would get a GUID. Values stored in big integers would play a big part. Every object would be a class, an individual being a class with one object.

Subsequent entries would compound upon another, and a freeform database (within a database) would emerge. Every assumption made would be linked to a basis, which can be revised. Revisions would stay within the database, and bogus assumptions would be culled by a garbage collection routine. So there are no Updates or Deletes involved, just Inserts and Selects. The most valid truth would just come from the relationship which has the most recent good basis.

The concept.

An independent web hosting agency doesn’t look out for the hosting company’s best interest, they look out for yours. With Cooncat Publishing as your web hosting agent, your web presence is insulated from vendor lock-in, outages are minimized, your name is protected, and publishing procedures are customized to match your workflow. For businesses that maintain multiple websites, the cost of using Cooncat Publishing to arrange for web hosting costs less than if they purchased hosting themselves.

The risks.

It’s no secret that web hosting is a volatile business. Companies fold frequently, often leaving their customers with no easy access to their own data. Hardware fails after years of use, and they won’t tell you that they don’t keep a spare “RAID controller” in the house. Servers are loaded, often overloaded, with several customers’ data, and popularity of (or network attacks against) one companies website may cause your own to go down.

Sometime we’re own own worst enemy, especially when it comes to computers. We might delete a necessary file inadvertently, and can’t find the backup for it. We could update a web page on our main site, and forget to copy it to your backup server. Or we might get someone else to write our web page, and find out what they publish wasn’t at all what we wanted — after several other people have already seen it!

The benefits.

Uptime

Monitoring software installed in several locations around the internet keeps track of whether your site is accessable, and switches automatically to a backup server if there’s a problem. You’ll be notified along with easy-to-understand details as to what happened. Redundancy is built into each part of the setup. Image files are kept on separate machines, so in case the traffic on your site becomes overwhelming, the visitors will still get your message. Our monitoring software will pick up on this, and adjust for the increased traffic in minutes.

Workflow

Editing your website is always done on a draft copy. Publishing the draft is a separate, one-click step that’s done after the draft is approved, and multiple levels of approval can be set up. Once approved, copies of the draft are automatically published not only to the main site, but to all backups as well.

Two types of backups are used. A mirror is a full copy of your website, ready to serve if your main site goes down. An archive is also kept, that keeps the details of the changes you’ve made to the site, in case you ever need to retrieve deleted pages, or parts of pages.

Ownership

Cooncat Publishing’s hosting agency ensures that you always have ownership of the media you entrust with us. We share all the information with you about the services we arrange on your behalf in clear, concise documentation. We provide you the tools for you to automatically include your own computers as a place to store your backups. We even provide you with everything you’d need to do your own arrangements in case you no longer wish to use our services. Our philosophy is that we abhor vendor lock-in; service providers should compete for your business by providing excellent value.

No silver bullet exists.

Unfortunately, the very nature of the World Wide Web keeps us from being able to achieve full reliability. This is because there is no way to quickly switch a computer, whether a name server or web hosting server for everyone who may visit your website.

The problem is that your visitors’ internet provider may keep stale information in their own computers. So they visit your site at www.foobar.com, and get an error message. The monitoring software has already told “the internet” that you’ve moved the site to a host that works, but the visitor’s computer doesn’t get that information for hours later.

While this problem might not be completely solvable — mostly because some internet providers don’t follow the standards — there are steps that can be taken to minimize disruption. That involves recurring testing, and informing you if you have visitors who may not be able to visit your web site; as well as what to tell them so they can fix the problem. These simple changes on their end will also fix their setup so they will see many other websites they’ve been having problems viewing as well.

We understand the web, and give you the tools you need to get your message out.

Related articles:

Impartiality Key in Hosting Brokerage Business

A blogging app should be immediately available.

A typical session now.

1. Wake up with a great idea for your blog.
2. Turn on the coffee pot.
3. Open blogging app and wait for it to initiate.
4. Connect to blog.
5. Jot down a quick outline of what you’re going to write about.
6. Make up a filename and save it.
7. What does the advanced toolbar do?
8. An operating system update is flashing. Guess I should run that while I’m just blogging.
9. Switch back to the blog, and write down the part of the main idea that I remember right now.
10. I’m in my browser, and can’t think of anything this second; would it hurt to check my news site? I promise myself I’ll only read one headline.
11. Oh, coffee’s ready. My throat’s dry. One more line and I’ll get a cup.
12. Switch back to the blog, and write down anything just so I can get my cup of coffee.
13. She comes in. “While you’re on the computer, dear, why don’t you go through this stack of bills?”
14. Open online banking site so she can see you’re being helpful. While it’s loading, start ripping open envelopes.
15. She went to get a cup of coffee. I can finally get back to blogging!
16. Switch to blog. Write down the punch line of the blog post.
17. Hear large crashing sounds coming from the kitchen. Jump up, “Are you alright?”
18. Find out she’s knocked over the sugar bowl you’d left hanging precariously off the counter. Help pick up glass and sugar mixture.
19. Back to the bills, she wants to know how much electric you used last month. Try to justify the usage increase.
20. Check bank balance, and find out you’ll have to make a decision between electricity and food. Put off decision until you open rest of bills.
21. OS update’s done, according to the annoying popup. Dismiss the popup, and leave the nagging icon on the screen. You’ll restart your browser later, right now you have work to do!
22. Switch back to your blog. Start rereading what you’ve written so far to refresh your own memory.
23. Hear commotion from the cellar. Run downstairs and find angry cat in the water of sump pump hole. Bring wet cat upstairs and wipe him down in the sink with the dishrag. Get hand scratched before done.
24. Wipe up water and blood from around sink area. See you have five minutes before you have to start getting ready to go to work. Hang dishrag up to dry in laundry room and find replacement.
25. Get back to computer. Your blog’s there awaiting attention. Mindful of the time, you switch back to the banking site. You’ve timed out, so you have to go back to the front page of the banking site to log in.
26. Before the bank site lets you in, it asks you to pick a security question and come up with an answer. You choose “what’s your pet’s name?” and write down “scratching little sump pump bastard”. The phishers won’t guess that!
27. Choose bill pay, and throw the electric company a thirty dollar bone. Hope they don’t shut you off before next payday, or you finish your blog post, whichever comes first.
28. Close bank website, and your news page shows. Do a quick scan of the headlines.
29. Check the time. You should be in the shower right now.
30. Write down the partial payment on your electric bill.
31. Close news site, and see your unfinished blog entry.
32. Save your blog without starting to read it again.
33. Open checkbook spreadsheet and type in the payment.
34. Close browser and tell the computer to reboot for good measure.
35. Take shower, get dressed, all that good stuff. While in the shower you remember some major thing you wanted in your blog post.
36. Get back to computer. Dismiss the “Can’t shut down right now” popup and hit save for the checkbook spreadsheet.
37. Spend a disappointing few seconds realizing your computer is going to reboot without giving you the chance to type your idea.
38. Put some oil in the car so you can make it to work. She’s in the window waiting impatiently. Run back in and kiss her, acting like you didn’t forget about that at all.
39. Cat give you a dirty, accusing look. Hiss at the cat while walking out the door.
40. Drive to work, holding throbbing scratched hand, trying to remember what you were blogging about.
41. Money worries pop up in your thought process, or rather that you’ll be giving up beer and lunchtime snacks until next payday.
42. Make a mental note that you need to make up with the cat or life will become unbearable.
43. Get to work late and walk into a large staff meeting you’d forgotton about. Everyone looks tired, sad, scared or angry.
44. Realize you’re not going to get back to that blog post today.

Help develop the future of blogging apps. Please leave your comments on how to avoid the problems.

A decision has to be accompanied by a basis. Why did you make the choice for action? If this isn’t recorded, and you forget, you won’t be able to defend your actions.

If those who are affected by your choice wonder if they are benefitted, they should be able to research your rationale without having to interrupt your life to recall the basis of your decision. That is, if you’ve recorded the basis of your decision understandably, and haven’t changed your mind since then without updating your basis. In that latter case, you’re in trouble.

By not sticking with the program, your crappy decision is about to affect someone else. If they are more cognizant of the situation than you were at the time of your original decision, they will call you on it. But if they take faith in your basis, take action on the same conclusion you did, they will go happily down a false path.

Depending on the potentials along the way, you’ve set them up for disaster. Now what will you do, plead ignorance?