Micropython – Scheduling

Yesterday I finished some draft code (if drafts are ever ‘done’…) to do scheduling in Micropython.

By ‘scheduling’ I mean having an arbitrary set of tasks for a microprocessor to do at arbitrary times, and also being able to upload these (right now, as files) and validate the files were transferred ok, and that they are ok in the filesystem. Scheduling for my current projects means things like:

  • altering the time of certain tasks based on solar power / sun position over different months
  • day-of-week specific tasks, like doing sensor log summary uploads once a week
  • time-of-day for different behaviour between day and night.

Here is what I sketched out, and where I’m at:

  1. Configurable
    • sched.json in the root holds a list of files and a schedule
    • it’s bascially human readable & editable
    • it lists external files that will be called
    • it lists the schedule of when to call them
    • it looks like this:
    • {
        "files": {
          "0": {
            "fn": "c1",
            "sh": ""
          "1": {
            "fn": "t1",
            "sh": "0fcec37295cb2b2b777598152d068c494091d2e689aa229f6c847de655910739"
          "2": {
            "fn": "x1",
            "sh": "deadbeef"
        "sched": {
          "0": {
            "File": 1,
            "Func": "run",
            "Months": "JFMAMJJASOND",
            "Days": "SMTWTFX",
            "StartH": 8,
            "StartM": 15,
            "EndH": 22,
            "EndM": 0
    • What’s going on here?
      • There are three sample files:
        • c1 is (right now) where the sched.json is read from; it’s a sort of ‘self’ entry
        • t1 is a real file, and it has a real sha256 hash that validates it
        • x1 is a missing file, and it has a dummy sha256 hash.
      • There is one schedule entry:
        • File: ‘1’ refers to the index key ‘1’ in the files section (so, file t1.py in the filesystem)
        • Func: ‘run’ means call a function named ‘run’ if this schedule is triggered
        • Months: all months are valid, none are X or x ‘d out.
        • Days: all days but Saturday are valid, so it won’t run on Saturday
        • It starts at 8:15 am, and runs until 22:00 (10:00 pm)
        • Internally it calculates times as minutes of day ( hours * 60 ) + minutes and will run the file if the current time is between those two times. It’s up to the user to NOT schedule an item to run if there isn’t time to complete it. I was thinking of including some type of ‘minimum runtime’, but not for this version.
  2. There is a ‘presha.py’ file that runs on the workstation to pre-compute the sha values for the sched.json file.
    • If the sha256’s in sched.json and the actual computed value at runtime don’t match, then any sched entry referencing that file won’t run, because calling a probably-corrupted file is BAD. These are pre-loaded at first run, right now.
    • TODO: If there are no files that pass the sha-256 check, the system should do something else as a recovery step. This is possible, but I haven’t written it yet because it’s going to depend on the project. Projects that are battery powered will have different recovery steps from those that are always-on, for example.
    • I’m using uhashlib to do the sha256 computations; which looks (without error try/except blocks) like this (note: the re-encoding is to ease string comparisons later, I’m not sure it’s the best way, but it works).
    • sha = uhashlib.sha256()
      file = open(fn)
      sha.update(file.read() )
      hash64 = bytes.decode(ubinascii.hexlify(sha.digest()))
  3. It *could* read time from an ntp server, but:
    • It seems that the times returned from either ntptime.settime() or machine.rtc.datetime()  are UTC, although I didn’t check beyond the point that they don’t return the time in my current timezone, and don’t seem to know about timezones.
    • Constantly hitting a *real* internet / public ntp server is considered bad form. Don’t do it.
    • The ESP-8266 has an RTC and it works. It’s obviously not battery-backed separately from the device (at least in the ESP-01), but it does work.
    • Rather than fuss with those it was simpler to just modify my internal not-quite-ntp service written in Go to return ‘dayofweek’ (actually, time.Now().Weekday()) in byte 8 of the response buffer. That change and the recompile took about a minute. Go is awesome. On the micropython side, it’s a matter of using socket.connect(addr), sending an ‘F’ to port 8123, and getting 16 bytes back; in the 16 bytes are the date and time.
    • So now I can get internal time plus day-of-week [0..7], correct in my chosen timezone, and pass that to IoT devices for use in things like sched.json where processes should only be run on certain days of the week.
    • TODO: Decide if I want to get fancy *now* with fallback logic if there isn’t valid time to set the local RTC. For now: No.
  4. Not-Global Variable Logic:
    • Last up, I needed a way to pass variables from the scheduler to the called function. The most direct way that doesn’t rely on more than one known variable is to pass a simple dictionary. In that dictionary keys/values can be placed that might represent state machine flags (like bad network communications), or configuration data shared in multiple bits of code.
    • In the current code it only has one key (eMin) representing the ‘end Minutes’ that the function must yield by.
    • That last point means that for now, each called function needs to respect it’s ‘eMin’ value, and start it’s own cleanup and exit routines by that time.
  5. Not Done Yet:
    • It’s not a schedule thing, but another use for JSON, since it’s so simple to read, is configuration data, like the hub IP address or where to direct logging / debugging; basically anything that might be user / installation specific.

Next: Hackaday? Instructable? GitLab?