I still earn my living with Rails, but I’ve become a little cynical about running it for personal projects. This is mainly because there’s a minimum amount of resources you need to run a Rails app of any complexity and it’s a bit expensive for hobbyist things. For example, I run my own Mastodon instance and it feels expensive for one person. I could allow others to use the instance, but then I’d have to police them and nah. I like having my own instance, some dickhead can’t call a ban hammer down on me.

I tried the Phoenix platform a few years back, talking to a JS front end, and it wasn’t quite there yet. It’s much more mature now, and more than ready for prime time. There’s also the Live Page thing, so no need to scar myself mentally trying to use some over engineered madness like React. I can just write some HTML and embed a few things. This is also how I feel about Rails these days, with Turbo and a little judicious use of JS you can just add the frills you need without several megabytes of download and a very slow response time. (However, if you include the React compiler it makes a big difference).

The project

Some years ago I became a little obsessed with word games on my phone. The ones I got into took a longish word and then used it to construct other words. The simplest one just wants a list of the other words, usually in alphabetical order, the more complex one creates a crossword style layout so you can sometimes get letters from the words you’re trying to work out.

Now, the game makers want to make some money, and I don’t object to that. So they allow you to win in game tokens and spend them on hints if you get stuck. Then you can purchase tokens if you’ve not managed to win enough to get past where you’re stuck. One game fills up a piggy bank as you solve puzzles and they charge money to open the piggy bank, we’re talking £20 or £30 here. They also want to charge as much as £30 if you want to top them up without having to earn them through the game. I kept getting stuck and won’t pay more than £2 or £3 for things like this, £30 being a relatively serious amount of money if you’re parting with it every few days. I do object to that, if you look at paying for Angry Birds (for example, he said) you can build up quite a lot of spending there, but only is relatively small amounts. They have it about right, or at least they did a couple of years ago.

Enter the Werds Gem that I created to search through a list of British and US American English words from the tool Scowl and do some matching against the source word and the pattern you might be looking for. I must have been quite pissed off to put in that much effort. I got ambitious and thought I would front the Gem with a Rails app and maybe start a web site for fellow frustrated word game fans. I even failed an interview I never had when the programmers looked at it on github and didn’t like that it probably wouldn’t work well in a web environment so wouldn’t talk to me, despite the readme never claiming it was architected for that and anything other than a play time alpha software thing. It works fine in the console when you’re sat there in your jammies trying to solve a puzzle before you go to bed.

I did attempt a Rails app and even got something primitive going on Heroku, which is gone now, but the slowness of the search meant it was unreliable and would time out. The gem was a proof of concept, didn’t have any tests, the documentation wasn’t quite right, etm.. It scratched my frustration with the word game itch though, I didn’t play these games for a year or so because other things happened and the project went back to sleep.

I got back into them a few weeks ago and started using the gem when I was stuck. The thought of the website got me going too, but the problems with Rails I mentioned earlier made me think I’d go for something cheap and relatively easy to work with, so let’s make a web site with Phoenix and use the lovely Fly to deploy. I recommend Fly, by the way, cheap and reliable, developer focussed, and not wrecked by money grubbing like Heroku now is.

To go to Elixir I had to translate the gem to a Hex library and work out how to share the dictionary in that environment. I also embraced the Elixir approach of writing documentation strings, specs, and tests for your public functions. Asking Deep Think to translate parts of the gem into Elixir made me realise that the approach I took in Ruby land was rushed and flawed. I worked out what the Ruby code was doing, but it was obvious that it did things in a back to front way.

The Ruby code would construct a Regex to get a list of candidate words, and then apply another regex to whittle it down, then check that the letters in the source word weren’t used more than the amount of times they appear in the source word. The code was incredibly convoluted.

So, as a first pass I wanted to create a function that took the source word, any candidate mask for a word to search for, and create the regex for it.

def make_mask(source_word, match_string) do
  pre_processed_match =
    match_string
    |> String.replace(@ellipsis, "...")
    |> String.downcase()
    |> String.replace(~r/[[:space:]]/, "")

  used_chars =
    pre_processed_match
    |> String.replace(".", "")
    |> String.graphemes()

  adjusted_regex =
    Enum.reduce(used_chars, source_word, fn char, acc ->
      String.replace(acc, char, "", global: false)
    end)

  "^#{String.replace(pre_processed_match, ".", "[#{adjusted_regex}]")}$"
end

The pre_processed_match string cleans up the source word. We’re reusing the regex match that a . character matches anything, but in this case anything is a character from the source word. I’d had problems with earlier web app when three dots got changed into an ellipsis, so kept that in. Say we have a source word banana and the mask of “..ana”, the regex we want is /[ban][ban]ana/ to scan the main dictionary with.

Then used_chars becomes a string with all the characters that are in the candidate list, ignoring any dots. Then we adjust it so the first occurrence of any letter we’re looking for is removed. Only the first and not all because the search pattern may only use one letter up and it occurs more than once in the source word. This gives us back a parseable string that is of the right form.

This should really be a private method, but I found developing it using tests to be beneficial so left it public. The equivalent method in the Ruby gem is unreadable to me now.

In the main function we can use this like so:

Enum.reduce(@dictionary, [], fn str, list ->
  if Regex.match?(search_pattern, str) do
    [str | list]
  else
    list
  end
end)

This will give us a list that matches the candidate pattern.

The job is not done yet.

  1. The words found can have multiple occurrences of characters that should appear only a certain amount of times from the source word.
  2. We may have been given a mask that contains letters that aren’t in the source word, which means it will find words that don’t match properly.

I think that solving this by hacking instead of testing is what made the Ruby code so difficult to understand.

The first problem is solved by adding a function that compares character counts;

def check_word(word_char_counts, source_char_counts) do
  Map.keys(word_char_counts)
  |> Enum.reduce(true, fn char, acc ->
    acc and Map.get(source_char_counts, char) >= Map.get(word_char_counts, char)
  end)
end

This takes two maps that contain the letter and the count of times it was used. I used Enum.reduce to and together a check that each letter isn’t used more times than it should be. The word counts are obtained from a helper function not given here.

Now we can filter the previous candidates

|> Enum.reduce([], fn str, list ->
  if check_word(get_char_counts(str), source_char_counts) do
    [str | list]
  else
    list
  end
end)

The second problem is solved using these character count maps, by extracting their keys and diffing the lists you get back:

extra_letters = Map.keys(match_char_counts) -- Map.keys(source_char_counts)

if extra_letters != [] do
  {:error, "Source word does not have letters '#{extra_letters}'"}

I decided to follow the convention of returning an error tuple. For logical consistency I should really return an ok tuple, but as I will be using the library myself I didn’t want to. Again, if I recall correctly, the Ruby code does this is a much more convoluted way.

The rest of the code is in the repository in Github if you’re interested.

The next thing is to create the first pass of the word finder on the web.