Kaleidoscope/UPGRADING.md

As the firmware evolves, there are - and will be - APIs that we deprecate, and eventually remove. We are constantly adding new features and plugins too.

This document lists noteworthy new features for the current release, with examples of use. Another section provides a short guide for upgrading from deprecated APIs. For deprecations, their planned removal date is also listed.

If any of this does not make sense to you, or you have trouble updating your .ino sketch or custom plugins, do not hesitate to write us at help@keyboard.io, we can help you fix it.

• Upgrade notes
  • New features
    • New plugin API
    • Bidirectional communication for plugins
    • Consistent timing
  • Breaking changes
    • The RxCy macros and peeking into the keyswitch state
    • HostOS
    • MagicCombo
    • TypingBreaks
    • Redial
  • Deprecated APIs and their replacements
    • Removal of Layer.defaultLayer
    • More clarity in Layer method names
    • Finer OneShot stickability control
    • EEPROMKeymap mode
    • Source code and namespace rearrangement
• Removed APIs

Upgrade notes

New features

New plugin API

For end-users

With the next version of Kaleidoscope, we are introducing a new plugin API. It's more efficient, smaller, and uses less resources than the previous one, while being more extensible, and a lot easier to use as well. But all of this matters little when one's not all that interested in writing plugins. However, the new plugin API comes with breaking changes, and one will need to update their own sketch too.

To achieve all of the above, we had to change how plugins are initialized. Instead of using Kaleidoscope.use() in the setup() method of one's sketch, the plugins must now be initialized with KALEIDOSCOPE_INIT_PLUGINS(), outside of the setup() method. While use() was expecting pointers (&Plugin), _INIT_PLUGINS() expects references (Plugin).

The conversion should be simple, and all of the official plugins have been updated already to use the new API, so they're safe to use this way. Some third-party plugins may still use the older API, they will need to be updated.

To make things clear, here's an example of how to migrate from the old way to the new:

// Old way
void setup() {
  Kaleidoscope.use(&LEDControl,
                   &Macros,
                   &OneShot,
                   &MouseKeys,
                   &LEDOff,
                   &LEDRainbowEffect);
  Kaleidoscope.setup();

// New way
KALEIDOSCOPE_INIT_PLUGINS(LEDControl,
                          Macros,
                          OneShot,
                          MouseKeys,
                          LEDOff,
                          LEDRainbowEffect);
void setup() {
  Kaleidoscope.setup();
}

One thing to keep in mind is that with the old interface, plugins were able to automatically pull in their dependencies. This is not possible with the new interface, and one is required to initialize dependencies manually. Please consult the documentation of the plugins you use, to discover their dependencies - if any - and add them to the list if need be. You only need to add each dependency once.

For developers

Developing plugins should be considerably simpler now, there is no need to register hooks, just implement the parts of the kaleidoscope::Plugin interface that make sense for a particular plugin.

In practice, this boils down to implementing one or more of the following hook points:

• onSetup(): Called once during device bootup, at the end of the setup() method. It takes no arguments, and must return kaleidoscope::EventHandlerResult::OK.
• beforeEachCycle(): Called once, at the beginning of each cycle of the main loop. This is similar to the old "loop hook" with its post_clear argument set to false. Takes no arguments, must return kaleidoscope::EventHandlerResult::OK.
• onKeyswitchEvent: Called for every non-idle key event. This replaces the old "event handler hook". It takes a key reference, coordinates, and a key state. The key reference can be updated to change the key being processed, so that any plugin that processes it further, will see the updated key. Can return kaleidoscope::EventHandlerResult::OK to let other plugins process the event further, or kaleidoscope::EventHandlerResult::EVENT_CONSUMED to stop processing.
• onFocusEvent: Used to implement bi-directional communication. This is called whenever the firmware receives a command from the host. The only argument is the command name. Can return kaleidoscope::EventHandlerResult::OK to let other plugins process the event further, or kaleidoscope::EventHandlerResult::EVENT_CONSUMED to stop processing.
• beforeReportingState: Called without arguments, just before sending the keyboard and mouse reports to the host. Must return kaleidoscope::EventHandlerResult::OK.
• afterEachCycle: Called without arguments at the very end of each cycle. This is the replacement for the "loop hook" with its post_clear argument set.

Bidirectional communication for plugins

For end-users

Whereas one would have used Focus.addHook() to register a new focus command, with the new architecture, one needs to add the object implementing the command to their list of plugins in KALEIDOSCOPE_INIT_PLUGINS(). A number of plugins that used to provide optional Focus commands now provide them by default. Some still provide optional ones, and we must transition to the new way.

For example, where one would have written the following before:

Focus.addHook(FOCUS_HOOK_LEDCONTROL);

...we need to add the appropriate object to the plugin list:

KALEIDOSCOPE_INIT_PLUGINS(LEDControl,
                          FocusLEDCommand)

For developers

Upgrading from Focus to onFocusEvent and FocusSerial is a reasonably simple process, the interface is quite similar. Nevertheless, we present a step-by-step guide here, covering two use cases: one where we wish to always provide a Focus command when both our plugin and FocusSerial are enabled; and another where we only wish to provide the command when explicitly asked to do so.

The most trivial example

The biggest difference between Focus and onFocusEvent is that the former required explicit registering of hooks, while the latter does it automatically: every plugin that implements the onFocusEvent method will be part of the system. As a consequence, only plugins are able to supply new commands: there is no explicit registration, thus, no way to inject a command that isn't part of a plugin. This also means that these functions now return kaleidoscope::EventHandlerResult instead of bool. Furthermore, with FocusSerial, all communication is expected to go through it, instead of using Serial directly. Lets see a trivial example!

Focus

bool exampleFocusHook(const char *command) {
  if (strcmp_P(command, PSTR("example")) != 0)
    return false;

  Serial.println(F("This is an example response. Hello world!"));

  return true;
}

KALEIDOSCOPE_INIT_PLUGINS(Focus)

void setup() {
  Serial.begin(9600);
  Kaleidoscope.setup();
  Focus.addHook(FOCUS_HOOK(exampleFocusHook, "example"));
}

onFocusEvent

namespace kaleidoscope {
class FocusExampleCommand : public Plugin {
 public:
  FocusExampleCommand() {}

  EventHandlerResult onFocusEvent(const char *command) {
    if (strcmp_P(command, PSTR("example")) != 0)
      return EventHandlerResult::OK;

    ::Focus.send(F("This is an example response. Hello world!"));
    return EventHandlerResult::EVENT_CONSUMED;
  }
};
}

kaleidoscope::FocusExampleCommand FocusExampleCommand;

KALEIDOSCOPE_INIT_PLUGINS(Focus, FocusExampleCommand);

void setup() {
  Kaleidoscope.setup();
}

Summary

The new version is slightly more verbose for the trivial use case, because we have to wrap it up in an object. But other than that, the changes are minimal, and we don't need to explicitly register it!

Observe that the return values changed: with Focus, if we wanted other hooks to have a chance at processing the same command, the hook returned false; if we wanted to stop processing, and consider it consumed, it returned true. With the new system, this is more descriptive with the EventHandlerResult::OK and EventHandlerResult::EVENT_CONSUMED return values.

A stateful example

Perhaps a better example that shows the quality of life improvements the new system brings is the case where the command needs access to either plugin state, or plugin methods. With the former system, the focus hooks needed to be static methods, and needed to be public. This is not necessarily the case now, because onFocusEvent is a non-static object method. It has full access to plugin internals!

Focus

namespace kaleidoscope {
class ExamplePlugin : public Plugin {
 public:
  ExamplePlugin();

  static bool exampleToggle() {
    example_toggle_ = !example_toggle_;
    return example_toggle_;
  }

  static bool focusHook(const char *command) {
    if (strcmp_P(command, PSTR("example.toggle")) != 0)
      return false;

    ::Focus.printBool(exampleToggle());
    return true;
  }

 private:
  static bool example_toggle_;
};
}

kaleidoscope::ExamplePlugin ExamplePlugin;

KALEIDOSCOPE_PLUGIN_INIT(Focus, ExamplePlugin)

void setup() {
  Serial.begin(9600);
  Kaleidoscope.setup();

  Focus.addHook(FOCUS_HOOK(ExamplePlugin.focusHook, "example.toggle"));
}

onFocusEvent

namespace kaleidoscope {
class ExamplePlugin : public Plugin {
 public:
  ExamplePlugin();

  EventHandlerResult onFocusEvent(const char *command) {
    if (strcmp_P(command, PSTR("example.toggle")) != 0)
      return EventHandlerResult::OK;

    example_toggle_ = !example_toggle_;
    ::Focus.send(example_toggle_);

    return EventHandlerResult::EVENT_CONSUMED;
  }

 private:
  static bool example_toggle_;
};
}

kaleidoscope::ExamplePlugin ExamplePlugin;

KALEIDOSCOPE_PLUGIN_INIT(Focus, ExamplePlugin)

void setup() {
  Kaleidoscope.setup();
}

Summary

It's just another plugin, with just another event handler method implemented, nothing more. No need to explicitly register the focus hook, no need to provide access to private variables - we can just keep them private.

Optional commands

Optional commands are something that were perhaps easier with the Focus method: one just didn't register them. With onFocusEvent, we need to do a bit more than that, and move the command to a separate plugin, if we do not wish to enable it in every case. This adds a bit of overhead, but still less than Focus did.

Focus

bool exampleOptionalHook(const char *command) {
  if (strcmp_P(command, PSTR("optional")) != 0)
    return false;

  Serial.println(Layer.getLayerState(), BIN);
  return true;
}

KALEIDOSCOPE_INIT_PLUGINS(Focus)

void setup() {
  Kaleidoscope.setup();
}

Do note that we do not register the exampleOptionalHook here! As such, because it is unused code, it will get optimized out during compilation. While this is a simplistic example, the optional hook might have been part of a class, that provides other hooks.

onFocusEvent

namespace kaleidoscope {
class ExampleOptionalCommand : public Plugin {
 public:
  ExampleOptionalCommand() {}

  EventHandlerResult onFocusEvent(const char *command) {
    if (strcmp_P(command, PSTR("optional")) != 0)
      return EventHandlerResult::OK;

    ::Focus.send(Layer.getLayerState());
    return EventHandlerResult::EVENT_CONSUMED;
  }
};
}

KALEIDOSCOPE_INIT_PLUGINS(Focus)

void setup() {
  Kaleidoscope.setup();
}

Summary

The trick here is to move optional commands out into a separate plugin. It's a bit more boilerplate, but not by much.

Consistent timing

As an end-user, there's nothing one needs to do. Consistent timing helpers are primarily for plugin use.

As a developer, one can continue using millis(), but migrating to Kaleidoscope.millisAtCycleStart() is recommended. The new method will return the same value for the duration of the main loop cycle, making time-based synchronization between plugins a lot easier.

Breaking changes

The RxCy macros and peeking into the keyswitch state

The RxCy macros changed from being indexes into a per-hand bitmap to being an index across the whole keyboard. This means they can no longer be or-ed together to check against the keyswitch state of a given hand. Instead, the kaleidoscope::hid::getKeyswitchStateAtPosition() method can be used to check the state of a keyswitch at a given row and column; or at a given index.

HostOS

Prior versions of HostOS used to include a way to auto-detect the host operating system. This code was brittle, unreliable, and rather big too. For these reasons, this functionality was removed. Furthermore, HostOS now depends on Kaleidoscope-EEPROM-Settings, that plugin should be initialized first.

MagicCombo

To make MagicCombo more portable, and easier to use, we had to break the API previously provided, there was no way to maintain backwards compatibility. This document is an attempt at guiding you through the process of migrating from the earlier API to the current one.

Migration should be a straightforward process, but if you get stuck, please feel free to open an issue, or start a thread on the forums, and we'll help you with it.

The old API

void magicComboActions(uint8_t combo_index, uint32_t left_hand, uint32_t right_hand) {
  switch (combo_index) {
  case 0:
    Macros.type(PSTR("It's a kind of magic!"));
    break;
  }
}

static const kaleidoscope::MagicCombo::combo_t magic_combos[] PROGMEM = {
  {
    R3C6,  // left palm key
    R3C9   // right palm key
  },
  {0, 0}
};

void setup() {
  Kaleidoscope.setup();

  MagicCombo.magic_combos = magic_combos;
}

Previously, we used a global, overrideable function (magicComboActions) to run the actions of all magic combos, similar to how macros are set up to work. Unlike macros, magic combos can't be defined in the keymap, due to technical reasons, so we had to use a separate list - magic_combos in our example. We also needed to tell MagicCombo to use this list, which is what we've done in setup().

The new API

void kindOfMagic(uint8_t combo_index) {
  Macros.type(PSTR("It's a kind of magic!"));
}

USE_MAGIC_COMBOS({
  .action = kindOfMagic,
  .keys = {R3C6, R3C9} // Left Fn + Right Fn
});

The new API is much shorter, and is inspired by the way the Leader plugin works: instead of having a list, and a dispatching function like magicComboActions, we include the action method in the list too!

We also don't make a difference between left- and right-hand anymore, you can just list keys for either in the same list. This will be very handy for non-split keyboards.

Migration

First of all, we'll need to split up magicComboActions into separate functions. Each function should have a unique name, but their shape is always the same:

void someFunction(uint8_t combo_index) {
 // Do some action here
}

Copy the body of each case statement of magicComboActions, and copy them one by one into appropriately named functions of the above shape. You can name your functions anything you want, the only constraint is that they need to be valid C++ function names. The plugin itself does nothing with the name, we'll reference them later in the USE_MAGIC_COMBOS helper macro.

Once magicComboActions is split up, we need to migrate the magic_combos list to the new format. That list had to be terminated by a {0, 0} entry, the new method does not require such a sentinel at the end.

For each entry in magic_combos, add an entry to USE_MAGIC_COMBOS, with the following structure:

{.action = theActionFunction,
 .keys = { /* list of keys */ }}

The list of keys are the same RxCy constants you used for magic_combos, with the left- and right hands combined. The action, theActionFunction, is the function you extracted the magic combo action to. It's the function that has the same body as the case statement in magicComboActions had.

And this is all there is to it.

If your actions made use of the left_hand or right_hand arguments of magicComboActions, the same information is still available. But that's a bit more involved to get to, out of scope for this simple migration guide. Please open an issue, or ask for help on the forums, and we'll help you.

TypingBreaks

Older versions of the plugin used to provide EEPROM storage for the settings only optionally, when it was explicitly enabled via the TypingBreaks.enableEEPROM() method. Similarly, the Focus hooks were optional too.

Storing the settable settings in EEPROM makes it depend on Kaleidoscope-EEPROM-Settings, which should be initialized before this plugin is.

Redial

Older versions of the plugin required one to set up Key_Redial manually, and let the plugin know about it via Redial.key. This is no longer required, as the plugin sets up the redial key itself. As such, Redial.key was removed, and Key_Redial is defined by the plugin itself. To upgrade, simply remove your definition of Key_Redial and the Redial.key assignment from your sketch.

Deprecated APIs and their replacements

Finer OneShot stickability control

The OneShot plugin has much improved stickability control. Instead of only being able to control if one-shot layers should be stickable too, or disabling the sticky feature in general, it is now possible to control stickiness on a per-key basis with the new OneShot.enableStickability() and OneShot.disableStickablity() methods.

EEPROMKeymap mode

The EEPROM-Keymap plugin had its setup() method changed, the formerly optional method argument is now obsolete and unused. It can be safely removed. Supplying a second argument will continue to work until its scheduled removal by 2019-04-30.

Source code and namespace rearrangement

With the move towards a monorepo-based source, some headers have moved to a new location, and plenty of plugins moved to a new namespace (kaleidoscope::plugin). This means that the old headers, and some old names are deprecated. The old names no longer work.

The following headers and names have changed:

• layers.h, key_defs_keymaps.h and macro_helpers.h are obsolete, and should not be included in the first place, as Kaleidoscope.h will pull them in. In the rare case that one needs them, prefixing them with kaleidoscope/ is the way to go. Of the various headers provided under the kaleidoscope/ space, only kaleidoscope/macro_helpers.h should be included directly, and only by hardware plugins that can't pull Kaleidoscope.h in due to circular dependencies.
• LED-Off.h, provided by LEDControl is obsolete, the LEDOff LED mode is automatically provided by Kaleidoscope-LEDControl.h. The LED-Off.h includes can be safely removed.
• LEDUtils.h is automatically pulled in by Kaleiodscope-LEDControl.h, too, and there's no need to directly include it anymore.
• Plugins that implement LED modes should subclass kaleidoscope::plugin::LEDMode instead of kaleidoscope::LEDMode.
• GhostInTheFirmware had the kaleidoscope::GhostInTheFirmware::GhostKey type replaced by kaleidoscope::plugin::GhostInTheFirmware::GhostKey.
• HostOS no longer provides the Kaleidoscope/HostOS-select.h header, and there is no backwards compatibility header either.
• Leader had the kaleidoscope::Leader::dictionary_t type replaced by kaleidoscope::plugin::Leader::dictionary_t.
• LED-AlphaSquare used to provide extra symbol graphics in the kaleidoscope::alpha_square::symbols namespace. This is now replaced by kaleidoscope::plugin::alpha_square::symbols.
• LEDEffect-SolidColor replaced the base class - kaleidoscope::LEDSolidColor - with kaleidoscope::plugin::LEDSolidColor.
• Qukeys had the kaleidoscope::Qukey type replaced by kaleidoscope::plugin::Qukey.
• ShapeShifter had the kaleidoscope::ShapeShifter::dictionary_t type replaced by kaleidoscope::plugin::ShapeShifter::dictionary_t.
• SpaceCadet had the kaleidoscope::SpaceCadet::KeyBinding type replaced by kaleidoscope::plugin::SpaceCadet::KeyBinding.
• Syster had the kaleidoscope::Syster::action_t type replaced by kaleidoscope::plugin::Syster::action_t.
• TapDance had the kaleidoscope::TapDance::ActionType type replaced by kaleidoscope::plugin::TapDance::ActionType.

Removed APIs

Removed on 2019-01-18

Removal of Layer.defaultLayer

The Layer.defaultLayer() method has been deprecated, because it wasn't widely used, nor tested well, and needlessly complicated the layering logic. If one wants to set a default layer, which the keyboard switches to when booting up, EEPROMSettings.default_layer() may be of use.

Layer.defaultLayer has since been removed.

More clarity in Layer method names

A number of methods on the Layer object have been renamed, to make their intent clearer:

• Layer.on() and Layer.off() became Layer.activate() and Layer.decativate(), repsectively.
• Layer.next() and Layer.previous() became Layer.activateNext() and Layer.deactivateTop().
• Layer.isOn became Layer.isActive().

The goal was to have a method name that is a verb, because these are actions we do. The old names have since been removed.

Removed on 2019-01-17

Compat headers following the source code and namespace rearrangement

With the move towards a monorepo-based source, some headers have moved to a new location, and plenty of plugins moved to a new namespace (kaleidoscope::plugin). This means that the old headers, and some old names are deprecated. The old names no longer work.

HostOS.autoDetect()

The autoDetect() method has been formerly deprecated, and is now removed.

The old MagicCombo API

We've changed the API of the MagicCombo plugin, and while it provided a helpful error message for a while when trying to use the old API, it no longer does so, the error message has been removed.

TypingBreaks.enableEEPROM()

TypingBreaks.enableEEPROM() has been previously deprecated, and turned into a no-op, and is now removed.

OneShot.double_tap_sticky and OneShot.double_tap_layer_sticky

These were deprecated in favour of a better, finer grained API, and are now removed.

Removed on 2018-08-20

We aim at making a new release by mid-July, and APIs we deprecate now, will be removed shortly after the major release, before the next point release. We may deprecate further APIs during the next month (until mid-June), and those deprecations will share the same removal date. We will try our best to minimize deprecations, and do them as soon as possible, to give everyone at least a month to prepare and update.

Kaleidoscope.use()

Deprecated in May 2018, this method is part of the old plugin API, replaced by KALEIDOSCOPE_INIT_PLUGINS. To upgrade, you need to modify your .ino sketch file, and replace the text Kaleidoscope.use with KALEIDOSCOPE_INIT_PLUGINS, then remove the & from all of the plugins inside it, and finally, move it outside of setup().

If your current sketch looks like this:

void setup() {
  Kaleidoscope.use(&Plugin1, &Plugin2);
  Kaleidoscope.setup();
}

You should change it so that it looks like this instead:

KALEIDOSCOPE_INIT_PLUGINS(Plugin1, Plugin2);

void setup() {
  Kaleidoscope.setup();
}

The old-style (v1) plugin API

This includes using KaleidoscopePlugin, Kaleidoscope.useEventHandlerHook, Kaleidoscope.replaceEventHandlerHook, Kaleidoscope.appendEventHandlerHook, Kaleidoscope.useLoopHook, Kaleidoscope.replaceLoopHook, Kaleidoscope.appendLoopHook. They were deprecated in May 2017.

Their replacement is the new plugin API:

namespace kaleidoscope {

enum class EventHandlerResult {
  OK,
  EVENT_CONSUMED,
  ERROR,
};

class Plugin {
public:
  EventHandlerResult onSetup();
  EventHandlerResult beforeEachCycle();
  EventHandlerResult onKeyswitchEvent(Key &mapped_key, byte row, byte col, uint8_t key_state);
  EventHandlerResult beforeReportingState();
  EventHandlerResult afterEachCycle();
};

}

Plugins are supposed to implement this new API, and then be initialised via KALEIDOSCOPE_INIT_PLUGINS.

Consumer_SNapshot

A key with a typo in its name, which was left in place after fixing the typo, so as to not break any code that may be using it already, however unlikely.

Removed on 2018-06-10 (originally scheduled for 2018-05-27)

These APIs and functions have been deprecated for a long time, and as far as we can tell, aren't used by any third party or user code anymore. They were removed as of the June 10th, 2018.

Kaleidoscope.setup(KEYMAP_SIZE)

The Kaleidoscope.setup() method is still around, and is not deprecated, but the variant of it that takes a keymap size is, and has been since October 2017.

Instead, one should use the argument-less Kaleidoscope.setup(), and the new KEYMAP() macros to define a keymap.

event_handler_hook_use, loop_hook_use, and USE_PLUGINS

Deprecated in October 2017, these are old aliases that should no longer be in use. They were replaced by Kaleidoscope.useEventHandlerHook, Kaleidoscope.useLoopHook, and Kaleidoscope.use, respectively.

The replacements themselves are also deprecated - see below -, but their removal will come at a later date.

MOMENTARY_OFFSET

Deprecated in October 2017, replaced by LAYER_SHIFT_OFFSET.

This symbol was meant to be used by plugins, not user code, and as far as we know, no third party plugin ever used it.

key_was_pressed, key_is_pressed, key_toggled_on, key_toggled_off

Deprecated in July 2017, replaced by keyWasPressed, keyIsPressed, keyToggledOn, and keyToggledOff, respectively.