ODBC (and HTTPD) now Testing via ren-c-action

I've revived the ODBC continuous integration test, at least for MySQL and Sqlite (the Firebird OBDC driver was in a bit of a problem state when I last checked, so I stopped trying to suffer through that specific one).

These scripts use the new ren-c-action on GitHub, and are fetching the built tool to test it. It's this easy:

mysql-test.yml Line 142

And you can see nice green lights on the MySQL Test Log and Sqlite Test Log

As another case of seeing it in action, here is the HTTPD server being tested in debug and release configurations on Linux, Windows, and MacOS:

6 green lights

When you look at how simple the GitHub Actions script is that does this, it's again super easy...barely an inconvenience:

test-httpd.yml

So it's definitely long overdue... Any Project You Care About Should be in a Repo With Tests Like This! That means it's time for @BlackATTR to have the QUERY dialect covered by this, for instance!

2 Likes

After about a month of a break from things Rebol-related, I'm back and mucking with things.

@gchiu wants to do ODBC-related stuff and is building a demo for a presentation, which mixes together the httpd server along with doing database requests. And of course he's doing it on Windows :angry: which has no automated test for ODBC.

The reason is that when you're dealing with the cloud, generally Windows sucks because all the install processes for things involve clicking through menus.

But after some research I did manage to figure out how to run .msi install processes headlessly and register ODBC DSNs.

So it's working for MYSQL 8, and something of this pattern should help build CI testing for any Windows application that uses ODBC.

While it all looks nice and clean in retrospect...

This Stuff Is A Pain To Figure Out

  • There's very little precedent in the modern world for finding people who are building test toolchains of hardly anything on Windows...especially when it comes to databases and servers. If there's testing code to refer to at all, it's all on Linux.

    • PyODBC was the exception. It may look like they only test on Linux because their GitHub Workflows directory only has %ubuntu-build.yml. But looking deeper they actually have a separate CI for Windows on AppVeyor. This is where I found how to list ODBC drivers and do headless installation.
  • Passwords and configurations for things are (for some amount of good reason) really complicated. But seem to change on a whim from moment to moment, and no one seems that interested in making easy loopholes for testing purposes.

    • Just looking at MySQL there was a period of time where when you start from default the user is "root" with a password of "root". But then sometimes this changes to where the root account has no password and expects to depend on superuser privileges, but if you provide a password of "root" it won't tell you that in this case there is no password.

    • Once you manage to figure out how to get into the root account doesn't mean you've got it made for your tests. Because many things can't be done as root. So instead of just being able to run your test, you have to go through the full DBA training class to invoke the right voodoo to make an account that is sufficiently privileged yet not-too-privileged to be authorized for the test.

    • Cloud systems are busy trying to tell you not to put passwords out in the open; if you're lucky they'll just print out a warning and put stars over the thing. If you're unlucky they'll just prohibit you from doing it altogether. This forces you to keep pushing the password further to the edge until you're hardcoding it into the test file.

      • (Then just wait for one database to tell you that test-password isn't secure enough and needs an uppercase letter, a digit, and five non-sequential unicode emoticons...so you have to go around and change all your configurations to please that outlier.)
  • There is more B.S. like "oh it's supposed to be -ppassword without a space, not -p password" than one should reaosnably have to deal with.

  • Everything on Windows is moving to PowerShell...deprecating everything else. I honestly can't say there seems to be anything that makes it preferable to plain old bash code.

    • My current vendetta against PowerShell is that there is no way to get its redirection to avoid corrupting the data stream with LF => CR LF (hence you cannot pipe binary data at all). They also throw in CR LF at the end of any data stream whether it was there or not...with countless examples of this messing people up trying to do piping tasks.

      • This only makes my resolve to fight the CR LF madness stronger. The default MUST be to leave data as-is, and anything else is just epicycles of corruption.

Of course, this is just a bunch of icing on top of the usual cake. You've got N operating systems multiplied by M databases with Q number of ODBC drivers, all of which have their fantastic quirks that you'll be spending a solid day sorting out for any particular instance of (n, m, q).

Anyway, Moving Forward...

Given that ODBC represents a continued investment here, we might as well make something of it. So that means interesting dialects and tools on top of it.

2 Likes

I've made a few changes to at least get the test-odbc.reb to complete though not without errors on firebird

I've gotten a hosted Windows GitHub Cloud Runner to install Firebird 3 and the Firebird ODBC driver, and it runs the ODBC test script:

https://github.com/metaeducation/rebol-odbc/runs/4381051309?check_suite_focus=true#step:15:5

This should mean there's no excuse for @gchiu to not establish a workflow that tests his database... stuff.

I've just given a talk tonight about using Rebol and Firebird to mine medical records. I should give it again as a Re-meet talk.

Definitely need a test suite on this!