Web Testing Now Takes Screenshots 📸

I don't particularly enjoy the process of making GitHub Actions...but...the time spent on making the reusable bits of code pays for itself quickly.

We had only one test of the Repl and it was rickety. But I'd said from the beginning we should be testing things like @gchiu's chess demo and power calculator...keeping them working.

But if it doesn't work on the cloud--and works locally--what do you do?

Screenshots, of course!

I made it so that you can ask for a screenshot when the run completes or times out. Just give it a name and it will be uploaded as a "GitHub Artifact" for you.

Here are four little smoke tests in the browser for today's world, and what the screen would look like when they stopped (if there was a screen). They are available on every test run as the artifacts at the bottom of the page. They expire after 7 days.

Chess


- name: Test Chess GUI Example
  uses: metaeducation/ren-c-action@release
  with:
    web: true
    timeout: 15
    screenshot: chess
    script: |
      animate-game: do @chess
      assert [
          comment [https://en.wikipedia.org/wiki/Fool%27s_mate]
          <done> = animate-game [
              f2f3 e7e6
              g2g4 d8h4
          ]
      ]

LATEST-OF


- name: LATEST-OF Smoke Test
  uses: metaeducation/ren-c-action@release
  with:
    web: true
    timeout: 15
    screenshot: latest-of
    script: |
      (url: latest-of)
      print ["Result was:" mold url]
      assert [url? url]

WATCHLIST


- name: Watchlist Smoke Test
  uses: metaeducation/ren-c-action@release
  with:
    web: true
    timeout: 15
    screenshot: watch
    script: |
      x: 10
      watch x
      assert [10 = watch 1]

REDBOL


- name: Redbol Smoke Test
  uses: metaeducation/ren-c-action@release
  with:
    web: true
    timeout: 15
    screenshot: redbol
    script: |
      redbol
      block: [b c]
      assert [[a b c d] = compose [a (block) d]]

It Really Is (Mostly) That Easy

You can use it too. So why not? Remember it can run code on GitHub Actions Linux, Windows, and Mac containers too (just don't say web: true and it will detect the platform and download the right r3).

...Web Automation Still Primitive, More Work To Do...

You can see that it makes the last thing it does:

print reverse {ETELPMOC TSET}

Then a setInterval()-based JavaScript timer kicks in every couple of seconds looking for the text "TEST COMPLETE". This is not great for the long term. We should be looking for something out of band (like the title bar of the browser, perhaps...I've done that before)

The newlines are replaced with spaces because if the console gets a newline, it starts running code...and isn't responsive to keypresses until the prompt comes back again. So characters get lost. Stripping out the newlines is bad because if you have any semicolon comments to end of line, they will screw things up by turning the subsequent lines into comments. Also it ruins multiline strings.

So a better way is needed to actually simulate hitting enter and then waiting (or the console needs to find a way to do keyboard buffering while code is running).

Deficiencies aside, it gives things more hope to have these kinds of tests.

3 Likes