Author: francis

MySQL

Decided to move my project to MySQL because it seems to be more of a standard compared with Postgres. This is a pity because I like Postgres a lot, but you have to go with the flow and if I’m going to work on some non-Oracle skills I should at least go with the market leader.

So, got it installed on XP (which was a breeze). Read the manual through a couple of times (you can download a compressed HTM help file). Persuaded PHP to talk to it, which mainly consisted of placing the MySQL bin directory in the path and rebooting.

Worked out that I needed the INNODB table types so that foreign keys would work (you can default this using the excellent configuration utlility). Also discovered that (assume table jim exists for the sake of this discussion):

create table fred
( x varchar(2) references jim(x) ) ;

Does not work:- it accepts the syntax but does not create the foreign key. I had to hack my SQL generator to get rid of the Oracle shorthand and to not use domains (as I was using with Postgres) but translate them into raw types. Now it’s:

create table fred
( x varchar(2) ) ;
alter table fred add constraint fred_jim_fk (x) references jim(x) ;

Which should work in Oracle and Postgres anyway. I’ve decided to eschew using domains (although my software lets you define them) so that it can be portable around many SQL engines. Need to also drop the Oracle-ism varchar2, as it is now a synonym for varchar anyway. I believe that when Oracle first created the type it wasn’t done right so they had another go; hence the 2 on the end for compatibility reasons and then it stuck. I wonder how many petabytes of storage are taken up with all of those 2’s? Someone pointed out to me that a single-character field should be char(1), because you’re storing extra space for the length, which is pointless when you only have one character …

All in all fairly painless.

If you’re an Oracle supporter, read this:

I found it interesting that the Census Bureau had an Oracle site licence and chose not to use it. One of my buddies uses MySQL a lot and is looking at using the text extensions to replace Oracle’s context (or whatever they are calling it now) becuase Oracle want loads of cash and context is a pain in the proverbial. MySQL is also a bit quicker for complex queries and doesn’t have so many common(ish) words that are used as keywords that you have to pre-process into a form that isn’t interpreted as one, how lovely and user-friendly- those context guys really put some thought into it, didn’t they? Apparently every term added to an Oracle query increases the time taken for the query, where MySQL has what seems to be a fairly fixed cost. Don’t quote me on this I haven’t validated this for myself (but it doesn’t surprise me). Don’t say the words “memory leak” or “restart your database every morning” either. (I think this is 8i – maybe fixed?)

I’m quite curious about how their spatial database stacks up against Oracle’s but don’t have the time to look into it. The beauty of it is, of course, that you could add things in if you wanted to, with it being open source. Hmm … who the has the time, though?

Every database vendor other than Oracle allow you to define auto increment columns and then programatically get back the last autoincrement created in that session. Oracle sequence thing is a poor cludge. I suspect you could simulate the autoincrement with a packaged procedure (caching the last sequence number and messing with the insert statements or some horrible trigger), but why don’t Oracle do this?

MySQL doesn’t let you create views with subqueries in them. Not sure why, I’d have thought it fairly easy to implement. Views are just strings you run together with whatever you’re joining them to; or am I being naive?

MySQL 5 (which isn’t stable yet according to the website) has procedures and triggers. Postgres already has them. I just wish there was a standard for these languages so I don’t have to learn them all. I think MySQL’s look Postgresy but don’t want to do in-database stuff at the moment because it isn’t portable so don’t care about this. But I do like triggers for policing complex relationships and auditing.

Python

Sort of keeping a watching brief here. Was going to look at using Ruby as my scripting language of choice but it’s not mainstream enough to want on a CV. I like Python’s clean syntax. I like Ruby’s power and the syntax is OK. They have a lot in common as well (PHP has too). I suspect that when the Open Source guys see a good idea in another language they nick it, starting with variants of a lot of the good things in PERL. On first inspection I think Python’s object model is poor, but maybe what I’m seeing is flexibility. PHP 5 has a very strong one, as does Ruby (everything is an object in Ruby). It does have lambda functions (see http://diveintopython.org/power_of_introspection/lambda_functions.html). Lambda functions are very useful and give you a lot less clutter and overhead, if you use them carefully. Sharp tools, sharp tools. Love ‘em.

Professional Development

Have decided to spend 3 months on my PHP project and then go back to the Java Certification route. Java may be a boring thing to work with but at least it gives me options. Options are where it’s at. I think that Python or PHP or Ruby are probably a lot faster than Java in terms of bangs per developer buck and getting the job done with less fuss (I haven’t the space here to critique Java) but Java’s what the IT manager types want because it’s safe. Probably not as safe as .Net, but pretty safe.

Sheds and bookcases

Finally got my bike shed built in the back yard. Annoyingly it will only take 3 bikes and we have 4. Ho hum. I bought a new bookcase on Friday to replace one that fell to pieces one day. Guess what? I thought I’d got a wide but low one and in fact got a narrow low one that is about a quarter of the capacity I needed. Ah well back to IKEA and get another. Ho hum …

Onward … blessings all. I need some sleep.

Lots of competent people, who think carefully about the problem, making small contributions is how it works. Don’t knock it. I’m a bit jealous of people who have made a contribution, which I know is weird of me. At times I feel like I’m trying to do a Michelangelo on the ubiquitous slab of marble, and not being allowed to find David, more like a badly-executed urinal without a drain hole.

(ClamAV) I need to get some up to date AV on my home machines. Was going to look at the OS stuff partly ‘cos I’m a tight wad and mostly ‘cos I want to support the OS movement.

I intend to make the Pharmarketeer stuff OS (collaborative evidence-based marketing – something like that). I’m trying to keep myself under control at the moment and do all of the work on pencil and paper until I’ve really thought through what’s required. Then I could make it into a book and get some residual from the book and consultancy work, just a thought. The move to MySQL is going slowly because I can’t connect to the database. Should have that cracked soon.

Ah well, back to me slab…

In response to http://www.it-director.com/article.php?articleid=12700

This seems a very bad caricature of the open source view. The way I understand it is that the ideas behind the software can be patented rather than just the software itself. So, for example, you come up with a means for managing some resource that is very efficient and write some software to do it, no-one else can use that idea without paying you, even if they implement it completely differently and do a much better job.

There are a whole ruck of ideas (linked lists, relational databases, hashed indexes for example) that are fundamental to working in a modern IT infrastructure. Where would Oracle (or MySQL) be if IBM had fought a patent over SQL – it could never have become the standard it is. How would this make the world a better place?

We cannot identify what ideas around today are as fundamental for the next wave of technology. It is totally naive to support patenting things like algorithms and good ideas. Profit comes from being better and quicker than your competitors. Profit is at the edge of things, where you are ahead of your competition, where you use sharper and better tools than they do.

Say you are trying to validate that at least one element in a radio group has been selected.

This hacking is because if a radio group only has one element you can’t treat it as an array so if the length is === (points to the the same object) as an undefined variable, numRowSelects, then there must only be one of them otherwise we use numRowSelects to walk the array of radio buttons, total nonsense.

Assume that theForm is the form with the radio group.
Assume that there is at least one element in the array or it all goes pear shaped.
When you get to the end of this oneSelected will be true if one has been picked or false otherwise.
As a side effect, if there is only one, it will be automatically selected.
This was developed using Firefox and also works with IE.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

var oneSelected = false ;
var numRowSelects;
if ( theForm.pi_row_select.length === numRowSelects ) numRowSelects = 1 ; // numRowSelects was undefined - hack
else numRowSelects = theForm.pi_row_select.length;
if ( numRowSelects == 1 )
{
  oneSelected = true
  theForm.pi_row_select.checked = true;
  theForm.pi_row_select.onclick();
}
else
{
 for( var i = 0; i < numRowSelects ; i ++ )
 {
    oneSelected = theForm.pi_row_select[i].checked
//  alert("loop " + oneSelected);
    if( oneSelected ) break
 }
}

What’s the matter with people

Today I remembered why I hate driving.

Incident 1:

Coming up to a set of lights just before a 30 zone. I’m in the outside lane, doing about 40. Car on inside wobbles part-way into the outside lane, sees me and wobbles back. I carry on. I’m about a foot from him (or her) and they start coming again. No indicators, nothing. Fortunately I’m able to use a right-turn lane to go round him. We then stop at the lights.

Horn beeping at me. I look in the mirror and shake my head. Then the abuse and shouting starts.

Let’s get this straight: I managed to avoid an accident, I manage to stop this person spreading me and my son all over the road, and somehow it’s my fault.

I indicated right (as in look, this is what indicators look like) and waved at the indicators. More beeping and abuse.

The lights change and we move off.

For some reason this is still winding me up at 1 o’clock in the morning and I bet the idiot concerned is sleeping soundly. There ain’t no justice.

Incidents 2, 3 and 4

All at roundabouts, people pulling out at me. I’m not speeding, I’m obeying the rules about signalling properly. I just don’t get it. If you’re going to take a risk because the traffic’s heavy you should get out of the oncoming person’s way, not make them have to brake to miss you.

Incident 5

I’m coasting to a stop at the lights and someone swaps lanes when I’m maybe six inches from them. Fortunatly they’re moving and can get away from me before I hit them. I just wave hello, completely speechless.

All this for the two or three seconds you save at 2 o’clock on a Saturday. Not even in a busy part of town. Why risk your neck for this? I don’t get it, not at all. All of these incidents happened in the same hour or so.

Sad thing is, this happens to me all the time when I drive to work. No wonder I love going there so much. I wish people would chill and calm down a little, just think through how little life we all have left, before they start taking dumb risks in cars made of kitchen foil.

Family stuff

Took Jon to the kid’s club at Awesome Walls. He manged to get his first award. We’re going there tomorrow (today, actually) so we can all climb. I then went and got him a new controller for is PS/1, because the old one’s broken. This was a reward for getting something called a silver book award at school for having a good attitude.

Programming Projects & books

Just finishing reading Advanced PHP Programming by George Schlossnagle (0-672-32561-6). Very good book, even if you’re not interested in PHP he has a lot to say about cookies, caches, security, reverse proxies, using design patterns (as in getting some value from the academic view), as well as a host of other useful stuff. PHP differs from Java in that it doesn’t use threads: the persistent thing is in fact the interpreter. This makes it hard to crash but perhaps slower because it’s difficult to share things across sessions. Not an issue for my project but worth remembering.

I also really like the __get() and __put() object methods, where you can add arbitrary fields to objects that you can access using standard syntax rather than a method call. I think this is going to be a very useful facility for my database abstraction layer.

Motivational books

If you’re struggling with motivating people or yourself I think you should read Whale Done by Ken Blanchard (I think of One Minute Manager fame). Essentially:

1. Catch people doing things right and sincerely thank them for it.
2. If they are doing things wrong, take the blame and then redirect their energy where you want it to go.
3. Praise improvement, however small.
4. Don’t forget to catch yourself doing things right.

It revolves around positive feedback. The metaphor is based on how they train the killer whales at the Sea Life Center in Florida. You can’t use coercion on an animal that is a top predator and weighs 1000 pounds. You have to become its friend and find out what it wants as a reward. People are even harder than this and yet we all persist in saying GOTcha.

I think this is why I’ve been so fed up recently. I have no idea what the value is of anything I do, and I get no feedback; either positive or negative. How can I improve or fix things without a benchmark? Also being told that your work is good when you know you are demotivated and underperforming calls your integrity into question and adds to the stress. Brains are weird.

Ah well, just finished some camomile tea.

In response to this:

http://www.theregister.co.uk/2005/03/24/ingres_and_open_source/

Hi Philip,

Speaking as someone who has jumped from Oracle to Postgres, mostly for cost reasons, I’m curious about what the relationship is between Ingres and Postgres. I thought Postres was Ingres. I obviously haven’t been following it too well.

The other thing about this database is that it is a much better implementation of the relational model. I can create proper domains that describe what a thing is instead of some techie crap like varchar(27) maybe or maybe not being a soundex code (or whatever). I like this because you can see what a table is supposed to do when you look at its description. Oracle still haven’t understood this.

Apropos Oracle, a lot of my friends say that they’d be very happy with an open-source (or just virtually free) version of the old Oracle 7.3.4, perhaps with the 8.1.5 PL/SQL engine. I would say that most of us just want a reliable data store and don’t give a stuff about all of the high-end stuff. 9i RAC and 10g – who gives a toss, really, apart from the half-dozen big corps (mostly telcos) that need big databases. I don’t care about Java in the database either, quite happy running it in the middle tier where it belongs, or just using PHP because it takes a tenth of the time to do something useful without fity squnitillion XML files that are sometimes ignored in some middle-tier framework that’s too hard to use.

Don’t get me started on their Applicaton Server – was a nice usable thing and is now a pregnant elephant in a temper with more layers than an alpine climber. In the light of which I’d like to see an article looking at the relative merits of JBOSS, OAS and WebSphere in the real world. Are there any plans to do anything like that?

Somewhat off the point but never mind.

Regards

Thought I might make some notes about this to help the next person along.

Firstly, PHP5 on windows just has the XML SAX parser installed by default so, even though there is a PEAR XML library, you don’t need it.

Attributes

The Java classes have a org.xml.sax.Attributes class. When you get passed this you can call a method to get values you are expecting, e.g.:

1
2
3
4
5
6
7
8

  void handleTable( final Attributes att )
  {
    firstColumn = true ;
    currentTable = att.getValue("name");
    os.println("create table " + currentTable );
    // Set up markers to create the tags table later
    if ( "true".equals(att.getValue("hasTag") )) tagTables.put(currentTable,currentTable) ;
  }

In Java there is an idiom where if there isn’t a value it will return null (which is why I have the string to test against the constant string first in the equals test, so that you don’t get a null pointer error if the attribute is null). PHP, on the other hand, uses the very useful associative arrays and the function became:

1
2
3
4
5
6

    function handleDomain( $att )
    {
      os::doprint("create domain " . $att["name"] . " " . $att["type"]);
      if ( array_key_exists( "nullable", $att ) && $att["nullable"] == "" ) os::doprintln(" not null");
      os::doprintln(";");
    }

Notice that I now have to use array_key_exists to test for something’s existence before I can do anything with it.
In this class, instead of having a local variable that is set to a print stream I have created a static class that I can use to do the printing instead. I may change this again, not sure.

A lot of the O-O junk isn’t there in PHP, a class is static if it doesn’t have any methods that need to use the this variable. Purists won’t like this but I don’t care, to be honest.

Method calls

One thing to be aware of is that PHP insists on the $this→; operator in front of any class methods. If you don’t put it there it will think you are calling an internal function but won’t complain until you try and call it.

Simplification using associative arrays

This:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

  void handleTags()
  {
    String table = null
    ,      column = null ;
    Object [] theKeys = tagTables.keySet().toArray();
    for (int i = 0; i < theKeys.length; i++) 
    {
      table = (String)theKeys[i] ;
      column = (String)tagTables.get(theKeys[i]) ;
      os.println("create table " + table + "_tag" );
      os.println("( " + column + " id references " + table + "(" + column + ") not null" ) ;
      os.println(", tag_id id references tag(tag_id) not null" );
      os.println(");");
    }
  }

Becomes:

1
2
3
4
5
6
7
8
9
10

    function handleTags()
    {
      foreach ($this->;tagTables as $table =>; $column) 
      {
        os::doprintln("create table " . $table . "_tag" );
        os::doprintln("( " . $column . " id references " . $table . "(" . $column . ") not null" ) ;
        os::doprintln(", tag_id id references tag(tag_id) not null" );
        os::doprintln(");");
      }
    }

Which I think is quite good. A ton less code to get to what you want to do. I think there is a similar thing to foreach in java 1.5 but nothing as easy to use as this.

All in all, for getting the job done, I think PHP wins hands down. Java is very low level and has to do everything through a class and chuck arrays of Object around. The people who developed PHP weren’t afraid of putting things like associative arrays in the core language and this saves a whole ton of code. If you want to override methods and go the whole O-O hog you can, but you have to explicitly say you are going to do so.

PHP5 has most of the Java-like constructs apart from this see http://www.developer.com/lang/php/article.php/10941_3302171_1

If you want to see a very coherent and logical explanation of why software patents are such ridiculous nonsense see http://www.kuro5hin.org/story/2005/3/9/93357/91614.

Can you imagine a world where the particular genius of your favorite author’s rendering of an archetypal story would violate a patent? Quite often I read a novel for the characterisation and the inner life of the characters, but if the plot was patented then it wouldn’t exist. All those fantasy novels

1. Problem – evil in the good land
2. Gathering of the good guys
3. Gathering of the forces of evil
4. The impossible quest
5. The struggle to complete the quest (sub plots, party gets split, is betrayed etc.)
6. Quest is complete and good triumphs (always at the last second)

I read them if the characters are interesting and well-written. I know what the plot is, the interest is in the execution! Maybe I should copyright this, only problem is I don’t want to get any money from the really bad ones because it would be too embarassing.

Imagine if physics was patented. You would be paying money to the patent holders when you read a book explaining it, instead of which the author has to acknowledge who did the work, had the idea.

I mean, what if they patented breathing?

See how silly it is?

Has Knuth patented his fundamental algorithms (well he gathered them together and validated them, but you know what I mean)? There’s a scary thought.

Got talked into looking at PHP rather than Python.

I really like it: it’s got a lot of the lovely features of the unix shell (heredocument and `), you can have nice structures like

if ( cond ) :
expr
endif

So you can block your code properly. I have always hated the C and Java thing about the closing brace not telling you what’s being closed; this can be a source of bugs that only becomes apparent when you run the code through a formatter. Of course, it also supports the annoying {} as well, which makes it nice and flexible and easy for the Java/C heads to get into. I know you can do stuff like

if ( cond )
{
// 20 pages of “stuff” (Should only be about 20 lines if you structure it properly)
} // If

I hate this though. Just looks ugly and you have to remember to do it or enforce some really annoying coding standards that people will ignore because they think you’re being too pedantic. By the way, I always place the opening brace under the if so that if you have a mountain of conditions in the if statement you can see where the executable expressions begin. In PL/SQL I do this for the same reason

if cond – no brackets, irritiating things; why do a lot of modern languages pretend to be C?
then
– Stuff
end if ;

I’m skim reading the tutorials at the moment to get a feel of the language’s capabilities.

I am going to rearchitect my system away from Struts and the like: I think I can just get a move on and get the thing finished quckly in PHP. Like Paul Graham says: the power of the abstractions of a language help you to get the job done and think PHP is ideal for what I need. And there are tons of exellent tools and large well-written projects to look at. There are also lots of nice free editors (including my all-time favourite VIM).

My plan currently is to redo the XML parsing I’ve been doing in PHP (a good learning exercise). Then I will create some self-describing objects that will define how the site is put together. Then PHP will be used to deliver these objects up to the end user. Make a change, regenerate (or maybe just change the XML descriptors), site stays up. That has always been the goal. I plan to use Open LDAP and suchlike for user and resorce management.

It looks loads less hassle than Struts for a small project like mine. I’ve been trying off and on for weeks just to get an HTML form that allows me to update some data, haven’t even got to the XML rewrite capabilities yet. Time to change to more powerful tools that can help me build the tools I need before I die of old age.

Postgres vs MySQL

I prefer Postgres as a platform, even to Oracle, because it is a purer implementation of the relational model. Particularly the ability to create domains, which is central to relational theory, but Oracle doesn’t do it. Types in Oracle don’t cut it, sorry, too fiddly and hard to insert and retrieve out again. Besides, they answer a different problem about composite data types. Problem is I need to run PG inside Cygwin on my XP machine and it hasn’t worked for a while, I discoverd that I need to run something called cygserver but it won’t play. There is now a native Windows version (have a look on the site here) but I need some time to investigate it. I’ve downladed MySQL as a temporary stopgap but don’t like it. Transactions and other stuff you need are relatively new: Postgres and Oracle have had them for as long as I can remember. MySQL is (I believe) mostly in-memory which makes it fast, but what happens when things fall over? Maybe it’s just my old-fashoined, data-centric training making me want stronger medicine than MySQL, I know it works very well for lots of people. I just want something stronger that suits my purist leanings, describe a table and you can see that a given column has a type of ID or Adress_line, you know what it’s for; what use is integer or varchar2(256), where are the semantics? Also change the domain and you know all of the related tables are OK. That is the way it should be done, not by relying on a tool like Designer or some external XML document that is used to generate scripts to align domain-related columns – rubbish, a waste of time and energy.

Amazon

Whacked 25 old computing books up on Amazon, I’ll give them a couple of months to sell and then bin them. I was holding on to a load of old Unix, Prolog, C programming, Smalltalk, Forth (from 1985!). I also inherited my mother’s poetry collection, which I will sell on too, but suspect that a ruck of paperback poetry won’t sell.

Ah well, off to bed. G’night all.

PS – I suspect I’ll be lookng at Python too, maybe to replace Awk as my text-processing “quickie” tool. Not sure yet.

In response to this

OK Paul

You’ve persuaded me to look at Python. I must admit I use Gawk a lot and I’ve made my windows XP machine into a Linux one by installing Cyngwin. I use Vim all the time, I find the way the vi navigation lets me get to the end of a word, or delete up to an underscore Just Works. This is by far more productive for me, even though I spend a lot of time writing PL/SQL and use the TOAD tool. I find I use TOAD for browsing stuff but need a decent editor with RE’s and callouts to Awk to be productive. I will probably migrate my Awking to Python. PL/SQL is Ada without the interesting bits.

Anyway, I half agree with what you say about Java being “for the rest of us”, but the interesting thing is that there are a lot of cool(ish) projects, particularly Apache Struts and Jakarta, that give you a lot of leverage if you are looking at solving some kind of web-based problem quickly, and it’s all open source. If I have a problem in Java I can usually find someone else has solved it in about 10 minutes using Google.

One of the guiding principles of J2EE was that people wouldn’t need to know how to program with threads, for example, and I don’t think this is necessarily a Bad Thing in and of itself. Most of us have to solve boring business problems for a living and we want well-defined architectures that will take the weight and let us get rid of them as fast as possible. I confess I’m still trying to see the point of EJB’s, other than being able to talk intelligently about them at job interviews.

So, I think that when you are being a craftsman it is far better to use more expressive languages (I’ve even worked my way through your book on ANSI Lisp and loved it) but when you are assembling stuff for boring work applications what you want is components you can assemble quickly and others will be able to follow when you’ve gone. That’s the problem, I think.