As part of its unwavering commitment to open source and open standards, Collabora is proud to be part of bringing the recently-released OpenXR 1.0 to life. We are pioneering the Monado open source runtime for OpenXR to ensure the future of XR is truly open and accessible to all hardware vendors. As the OpenXR specification editor, I am grateful for the diligent efforts of the working group, as well as the community feedback that shaped this release.

There have been a lot of changes since the last post about OpenXR and Monado. On the working group, we've brought the concerns of the open source and Linux communities to the working group. We have worked to improve the loader and provided API layers in both cross-platform and Linux-specific ways, together with the Monado community. As specification editor, I developed or enhanced a variety of specification-related tooling to ensure a continuous standard for consistency and high-quality in the specification text and registry.

For example, xml_consistency uses specification-specific "business logic" to check the internal consistency of the XML registry. Among other things, it compares the return codes listed for a function with those inferred from parameter types, and raises an error if an expected code is missing or an existing code seems unnecessary. The comprehensive check_spec_links tool processes the AsciiDoctor source of the specification, ensuring that the spec-specific markup macros are used correctly, that all members and parameters are documented, that all entities referred to actually exist and are spelled correctly, and more.

Over the past several months, I have worked with participants in other Khronos working groups that share a related spec toolchain, such as Vulkan, to harmonize our forks of the tooling scripts and bring these additional quality checks to their specifications as well. The first such changes were published in Vulkan 1.1.96, and have been rolling in to later releases as the tools were improved across the projects. In addition, I introduced the use of Jinja2 templating in the code generation process for Khronos specs and code, in place of pure Python string interpolation. Uptake in the OpenXR group has been promising, and it is now being used in the new-for-1.0 openxr_reflection.h header as well as in-progress conformance tests.

Once the OpenXR 0.90 specification was released, we were able to begin releasing some of our previously-internal utilities and tools. These are currently using 0.90, but are in the process of being updated. [Edit 14-August-2019: xrtraits and xr-system-tests have been updated to 1.0.] These standalone projects, maintained under the broader Monado umbrella but not tied to the Monado runtime specifically, include:

• Unit and integration tests for OpenXR runtimes: https://gitlab.freedesktop.org/monado/utilities/xr-system-tests
• A simple command-line tracker data display tool: https://gitlab.freedesktop.org/monado/demos/xr-print-tracker
• "xr-traits", a collection of C++ utilities and type traits to make interacting with OpenXR more natural and idiomatic in C++: https://gitlab.freedesktop.org/monado/utilities/xrtraits

Monado updates

With the release of OpenXR 1.0, we were able to publish the few changes required to adapt Monado to the 1.0 API, which will be our target ABI from here on.

To answer the simple question "Are we there yet?", no. There are many pieces missing to make Monado a complete OpenXR runtime. We are working hard to soon get into a state where people can try it out and have a good experience. Already, people who are very adventurous can join in and hack if they want to; please feel free to join us on IRC or Discord.

As mentioned above, as a product of running test suites on Monado, we have made lots of improvements to the OpenXR state-tracker to make it more complete and bring it closer to conformance. A fairly large chunk of work was needed to bring in the action system, and code to handle rebindings of actions to various hardware still needs to be written.

On the driver side we have merged four new drivers, plus the V4L frameserver driver. A device detection subsystem has been merged as well, for use by drivers. The new drivers are:

• PSVR driver for the HMD
• PS Move Controller driver
• OSVR HDK driver
• Razer Hydra tracked controller driver

One of the biggest chunk of work ahead of us is tracking. Broadly speaking, there are 2 main types of tracking: outside-in where sensors in the world track an object, and inside-out where sensors in the object track landmarks in the world (this is a gross simplification). In many cases, these sensors are cameras. These can then be based on either: stereo color cameras as used by PSVR; IR camera(s) in the case of Constellation; or in the case of Lighthouse IR beam and sensors. The term "inside-out tracking" is usually used to refer to markerless SLAM ("Simultaneous localization and mapping") tracking such as used on the Windows MR headsets. Note that in the case of Lighthouse, despite the light sensors being on the tracked object, it could be logically considered an outside-in tracking system because the scanning base station beam is somewhat like the scan of a camera readout: the math is more similar to camera-based outside-in systems than to SLAM.

To help with our goal of being maker/hacking friendly, we chose to first focus on a outside-in color based tracking system, more specifically the PSVR. As there are a lot of PSVR headsets out there, hacking on something vision-based requires just a webcam and some printed markers.

Right now, we have merged infrastructure for passing frames of data around into master. We are in the middle of expanding that to be able to hold various metadata needed to make sense of the images. On a branch we have experimental code that is able to track a PS Move controller in 3D space, can identify the PSVR LEDs. We also have a branch with some in-progress improved colour filters to improve tracking speed.

Where to talk to us

As mentioned above we have IRC and Discord channels. We will also be at SIGGRAPH this week, so please feel free to send a message to jakob at collabora.com if you want to meet up or come to the Khronos BoF sessions on Wendsday. We will also be attending various different open source conferences to talk about our work, including GUADEC, Akademy, and XDC. Finally, we will be hosting a new conference specifically for Open-Source XR in Amsterdam on the 26th of October.

Select changes from OpenXR 0.90 to 1.0

While the general shape of OpenXR remains the same between the provisional 0.90 release and the full 1.0, changes have been made for improved usability and consistency, as well as to resolve issues discovered during use of the provisional release. Most changes are fairly small (specifying more corner case or validity behavior, renaming, resolving contradictions, etc.) The actions system underwent larger changes, in part to make it easier to use correctly and harder to use incorrectly.

As an obligatory note, this post isn't an official product of the working group. Anything that can't be derived from a comparison of the released specs should be considered my personal opinion. While I have tried to be thorough, it is not necessarily comprehensive, and it is definitely not normative: just informative for those who have been with OpenXR longer.

Jump to a section: New helper header: openxr_reflection.h | next Chains | Return codes and valid usage | Undefined behavior | Two-call idiom | Versioning | Space Relations | Spaces | Session | Rendering and Swapchains | Input and Actions | Extensions | Misc | Public spec issues closed

---

New helper header: openxr_reflection.h

This header provides expansion macros ("X Macros") for generating data about OpenXR functions, types, and enums. The way these work is that you define a macro taking a particular number of arguments (depending on the usage), and pass the name of that macro to one of the macros defined in openxr_reflection.h. It calls your macro repeatedly with the data you want.

One of its goals is to make xrResultToString and xrStructureTypeToString obsolete, though we weren't able to get to full agreement and code revision on that in time. The header does include sample code showing how to make an "enum to string" function.

While use of openxr_reflection.h is not required for OpenXR usage, it provides useful utilities, especially to those working at a level above end-user application software, such as game engines, layers, and runtimes. I am certainly excited to have it available!

The file isn't currently all-encompassing, but I anticipate its gaps will be filled in over time.

Enum types

For each enum type, there is an X Macro #define XR_LIST_ENUM_enumname(_) that calls its argument with the name and numeric value of every enumerant of that type.

For example, the first few lines of XR_LIST_ENUM_XrResult are:

#define XR_LIST_ENUM_XrResult(_) \
    _(XR_SUCCESS, 0) \
    _(XR_TIMEOUT_EXPIRED, 1) \
    _(XR_SESSION_LOSS_PENDING, 3) \
    _(XR_EVENT_UNAVAILABLE, 4) \

Note that all extensions-defined enumerants, as well as the dummy MAX_ENUM value (which forces enum type size), are provided in addition to the core values.

Bitmask bits types

For each bitmask Flags type, there is an X Macro #define XR_LIST_BITS_flagsname(_) that calls its argument with the name and numeric value of every bit for that type (from the corresponding FlagBits enum).

For example, XR_LIST_BITS_XrSpaceVelocityFlags is defined as:

#define XR_LIST_BITS_XrSpaceVelocityFlags(_) \
    _(XR_SPACE_VELOCITY_LINEAR_VALID_BIT, 0x00000001) \
    _(XR_SPACE_VELOCITY_ANGULAR_VALID_BIT, 0x00000002)

Note that all extensions-defined bits are provided in addition to the core values.

Structure types

For each structure type, there is an X Macro #define XR_LIST_STRUCT_typename(_) that calls its argument with the name of each member of that type in order.

For example, XR_LIST_STRUCT_XrInstanceProperties

#define XR_LIST_STRUCT_XrInstanceProperties(_) \
    _(type) \
    _(next) \
    _(runtimeVersion) \
    _(runtimeName)

next Chains

• Clarified in places that the next chain isn't solely for extensions.
• This work is ongoing, but the overall text in Fundamentals is now more correct.
• Clarified that runtimes must ignore all but the first struct of a given type in a next chain, and that apps should avoid putting duplicates in. (but not "must not")
• Implicit valid usage (generated) revised/improved across the spec, and the uniqueness statement was removed from all implicit valid usage since it contradicted the Fundamentals section.
• Platform-specific XrCreateInstanceInfo-chained structs no longer break the convention by requiring blacklisting by runtimes.

Return codes and valid usage

• Addition/usage of a number of XR_ERROR_..._UNSUPPORTED return codes: For when something is valid but not acceptable in a given usage. For example, XR_ERROR_PATH_INVALID is if you pass in uninitialized memory containing garbage as XrPath. XR_ERROR_PATH_UNSUPPORTED is if you pass in /user/hand/left as an interaction profile path.
• Clarified return values of destroy functions with respect to the various lost/loss-pending return codes.
The general principle that "doing the right thing won't result in an error code" is now applied.
• Specified a 1% tolerance from unit length for quaternions for returning XR_ERROR_POSE_INVALID.
• XrTime
  • Zero or negative XrTime is invalid by default.
  • Overflowing XrTime is undefined. (64 bits ought to be enough for anybody.)
  • Highlighted that the XrTime clock may drift relative to system time: use the conversion extensions instead of trying to track an offset yourself.
• Extents must be non-negative.
• XrFovf must be in (-pi/2, pi/2), and flipping of left/right or top/bottom is described.
• Path name strings must not end in a trailing slash.
• Many implicit valid usage (generated) fixes.
  • Lots of semi-meaningless statements about something "must be a valid uint32_t value" or other scalar type removed.
  • next chain text updated to reflect the prose in Fundamentals.
  • Buffers to receive strings as output don't have to be null-terminated UTF-8 upon input.
• Functions now have a "Thread Safety" block added to them if there are external synchronization requirements, and these requirements are also summarized in a list in Fundamentals. This uses the same mechanism in the XML that Vulkan uses for their Host Synchronization requirements.
• xrResultToString and xrStructureTypeToString no longer use a loader-supplied implementation for null instances.
  • As a suggestion, I would recommend not even using these functions, but instead using the new openxr_reflection.h header to generate your own version.

Undefined behavior

There was some existing text in the spec that was very Vulkan-like about the risks of "invalid usage". However, Vulkan is much more "dangerous" than OpenXR, by design. Vulkan tries to avoid overhead at nearly any cost, because it's a high-frequency API. Therefore, incorrect usage in Vulkan without the validation layer may cause terrible things ("Undefined Behavior" in the nasal demons sense) to happen.

OpenXR is comparatively very low frequency, so most error cases are required for the runtime to catch. The various references to return codes have been clarified, most by adding/changing to "the runtime must return". The actual places where Undefined Behavior may occur are very few in OpenXR. Notably, XrPath usage is not intended to cause undefined behavior since XrPath is just an easier way to refer to a path name string. They may vary between instances, so mixing them or using random garbage might cause unexpected behavior depending on what the value maps to, but the behavior is always defined: an error code or a function working "properly" on possibly-wrong data. No nasal demons.

Two-call idiom

The "count output" parameter is now always required - it cannot be NULL. Previously, this was self-contradictory. Additionally, for the sake of one current usage, a "struct"-style two-call idiom is described, along with behavior when two things need to be enumerated in the same call. The "visibility mask" extension is the troublemaker here: it enumerates vertices and indices at once - two lists of different size.

Most two-call idiom calls have had their spec text updated to indicate if or when their results may change between calls.

Versioning

• The way of packing versions has changed - it's now into 64 bits instead of 32.
• There is a new typedef, XrVersion, that is used whenever such a packed version is expected.
• XR_MAKE_VERSION now masks its arguments, so you can specify the max value for a component with e.g. ~0.
• XR_HEADER_VERSION was removed because it was unneeded.
• Some version-related structure members had their names change, to be clearer about what they refer to.

Space Relations

The XrSpaceRelation structure was replaced with two structures: XrSpaceLocation (which has pose, usually all you need) and XrSpaceVelocity (which adds linear and angular velocity). For those cases when you really need velocity (such as when initializing the velocity of a thrown object) add an instance of XrSpaceVelocity to the next chain of XrSpaceLocation. The time associated with a relation is no longer in either of those structures. The ability to get acceleration was removed, since no compelling application use case was identified that warranted the cost in complexity. (If this changes, there are always extensions.).

Note: You should not attempt to use XrSpaceVelocity to do any kind of tracking prediction: leave that to the runtime by just asking for a pose in the future.

Spaces

Runtimes now must support local and view space. Contradictory information about space change events was fixed.

Action spaces are now unlocatable any time their action's action set was not in the most recent sync call (making them inactive), rather than unlocatable only until the first sync call.

Because of the hierarchy change, action spaces are now children of sessions, rather than of actions.

Session

The session lifecycle was further refined and clarified. The "RUNNING" state is now called "SYNCHRONIZED", for clarity, and what it means for a session to be "running" was more clearly defined: it means there has been a successful xrBeginSession but no call to xrEndSession. There is now a function to request a session exit: xrRequestExitSession, which fills in a missing transition on the state diagram.

A lot of spec text in this section has been replaced/revised, so see the text for the nitty-gritty details.

Clarification was added that if there's a "user engagement sensor" (proximity detector or other presence detect), it should be used to drive the STOPPING state.

A number of events were missing a session handle to provide them with necessary context: this has been fixed.

Rendering and Swapchains

If a swapchain create flag (static image or protected content) in XrSwapchainCreateInfo is specified but not implemented by the runtime, the runtime now must indicate this with XR_ERROR_FEATURE_UNSUPPORTED. The other fields of XrSwapchainCreateInfo now have more explicit descriptions of their valid values.

It's now explicit that trying to acquire a "static image" swapchain more than once results in an error.

Some structure parameters related to swapchains are "empty" (only type and next), included only to provide a next chain for future extension. These are now consistently noted as optional.

XrViewLocateInfo now additionally takes (as input to xrLocateViews) a XrViewConfigurationType.

Some editorial, non-normative content related to IPD was removed. Other non-normative content that was still important to understanding the spec was moved to informative "Note" boxes.

Requirements on the runtime for frame timing and waiting have been made more consistent.

XrFrameState now has a shouldRender field indicating if the application should do its heavy GPU work of rendering. This is to ensure that the frame loop is kept synchronized (begin/end frame) even if no layers should be submitted. This replaces the removed XR_SESSION_VISIBILITY_UNAVAILABLE "success code".

The composition layer flag bit XR_COMPOSITION_LAYER_UNPREMULTIPLIED_ALPHA_BIT was added to indicate if you are submitting a layer whose color has not been pre-multiplied by the alpha component. Related cleanups to blending specification were also made.

View configuration type is now consistently used in conjunction with a system ID: the system ID doesn't necessarily imply a single view configuration type. (Some systems might support multiple view config types: an HMD can support mono viewing, etc.)

Input and Actions

This where the most changes took place.

• XrActionSet is now created directly under XrInstance rather than XrSession.
• Your flow for setting up interaction should now go:
  • Create instance.
  • Create actions and action sets. (moved earlier in the process)
  • Suggest bindings with xrSuggestInteractionProfileBindings (moved earlier in the process and renamed from xrSetInteractionProfileSuggestedBindings)
    • If this is called more than once for any interaction profile, only the contents of the last call are considered.
  • Create session.
  • New step: "Attach" the action sets to the session with xrAttachSessionActionSets.
    • Note: This makes those actions and action sets immutable!
      • You cannot create new actions in an immutable action set.
      • You cannot suggest interaction profile bindings for an immutable action.
    • Actions, actions sets, and suggested bindings are all intended to be submitted roughly at once, before you start the "body" of your application. This gives the runtime the full information about your use of actions upfront, so it can make better decisions, provide complete rebinding UIs, etc.
    • Game engine editors need to tear down and re-create the session, actions, action sets, and suggested bindings in order to change them.
  • Begin your session.
  • In your loop:
    • Call xrSyncActions (formerly xrSyncActionData)
      • now takes XrActionsSyncInfo instead of an array of XrActiveActionSet
    • Get your action states and/or modify haptic feedback
      • Action state functions now take a structure XrActionStateGetInfo instead of lots of parameters, for extensibility.
      • Similarly, haptic feedback functions now take XrHapticActionInfo.
      • Both of these structures contain at most a single subaction path, rather than an array as in 0.90: you may only interact with one subaction path per call if you specify a subaction path.

  
  Relatedly, it is clarified that destroying an action set handle just invalidates your access to it: the action set continues to exist (with no way for you to reference it) until it is no longer used by any handles.

  The same applies when you destroy an action handle.

  This solves the question of what happens when you destroy an action set during a session, etc.

  Path related changes:

  • The input component indicating "hand squeezed shut" is now called "squeeze", not "grip", to free up "grip" for...
  • The pose identifier for holding something in the user's hand is now called "grip", not "palm".
  • The pose identifier for pointing somewhere in the world is now called "aim", not "pointer"

  Interaction profiles were updated accordingly.

  Other minor action changes:

  • It was clarified that localizedName for action/action set must also be unique.
  • Vector1f actions were renamed Float actions.
  • XrActionType enum values were renamed to be more consistent.
  • xrGetBoundSourcesForAction has been renamed xrEnumerateBoundSourcesForAction for consistency, and now takes a session as well as an info struct with the action handle in it.
  • The XrAction parameter for xrCreateActionSpace has moved into the info struct.
  • The source and components arguments for xrGetInputSourceLocalizedName have moved into an info struct.
  • Input source paths were clarified to reflect that the /<component> part is optional.
  • Acceptable subaction paths for interaction profiles are a whitelist: suggesting a binding for a path that isn't in a given interaction profile now returns an error, for earlier feedback about mistaken usage.
  • The interaction profile for the Valve "Knuckles" controller was renamed to its shipping name, /interaction_profiles/valve/index_controller

Extensions

XR_KHR_android_surface_swapchain

Interaction with the session lifecycle and states is specified.

XR_KHR_D3D10_enable

Removed as obsolete.

XR_KHR_headless

Pulled from release due to some issues related to the session lifecycle. Collabora is working on a temporary vendor extension (edit: XR_MND_headless now published in 1.0.1), and hopes to resolve the underlying issues soon to be able to release a KHR headless extension.

XR_KHR_composition_layer_cube

In XrCompositionLayerCubeKHR, XrVector3f offset was removed, along with a large amount of explanatory and editorial text about it. Its behavior can be provided by supplying a space created with a transform.

XR_KHR_composition_layer_equirect

Clarified that only the interior of the sphere is visible, and that it's more similar to XrCompositionLayerQuad than XrCompositionLayerProjection.

In XrCompositionLayerEquirectKHR, XrVector3f offset was removed. Its behavior can be provided by supplying a space created with a transform.

Also in XrCompositionLayerEquirectKHR, float radius was added.

XR_KHR_convert_timespec_time

XR_ERROR_TIME_INVALID is now returned if a conversion cannot be performed.

XR_KHR_opengl_enable

Clarified that an implementation is only required to support the structs associated with the platform it is running on. (Windows runtimes don't have to implement xlib structures, etc.)

Clarifies creation: app must create context, runtime creates textures for swapchain.

Clarifies multi-GPU error case on Windows if there is a mismatch: XR_ERROR_GRAPHICS_DEVICE_INVALID

In XrGraphicsBindingOpenGLXcbKHR, uint32_t screen_number was renamed screenNumber for consistency.

A copy-paste typo in the Wayland section was fixed.

XR_KHR_opengl_es_enable

Clarifies creation: app must create context.

XR_KHR_visibility_mask

The XrVisibilityMaskKHR struct was revised to use the new "struct-style" two-call idiom, with separate capacityInput and countOutput members for each array. Its associated function, xrGetVisibilityMaskKHR, was updated to link to that part of the fundamentals.

The XrEventDataVisibilityMaskChangedKHR event now has a session field.

XR_KHR_vulkan_enable

xrGetVulkanInstanceExtensionsKHR and xrGetVulkanDeviceExtensionsKHR had their two-call idiom parameters renamed for consistency:

• namesCapacityInput -> bufferCapacityInput
• namesCountOutput -> bufferCountOutput
• namesString -> buffer

XR_KHR_win32_convert_performance_counter_time

XR_ERROR_TIME_INVALID is now returned if a conversion cannot be performed.

XR_EXT_debug_utils

Reference page markup was added.

A new message type was added, for future use in conformance utilities: XR_DEBUG_UTILS_MESSAGE_TYPE_CONFORMANCE_BIT_EXT.

The additional XrResult value previously added by this extension, XR_ERROR_DEBUG_UTILS_MESSENGER_INVALID_EXT, was removed and replaced with XR_ERROR_HANDLE_INVALID to match the rest of the spec.

Thread safety/external synchronization concerns were added to the XML and generated portions of the spec for xrCreateDebugUtilsMessengerEXT and xrDestroyDebugUtilsMessengerEXT.

The sample code snippets were updated and fixed to actually compile (as tested by CI with make -C specification build-examples).

XR_EXT_performance_settings

Reference page markup was added.

XR_EXT_thermal_query

Reference page markup was added.

Vendor extensions

XR_VARJO_quad_views was added.

Misc

• Members removed:
  • XrSystemGraphicsProperties::maxViewCount
• Members added:
  • XrFrameState::shouldRender
  • XrActionSpaceCreateInfo::action
  • XrViewLocateInfo::viewConfigurationType
• The return code table in the spec is now generated from XML, so it is now and will always be in sync and up to date.
• XR_MAY_ALIAS no longer appears all over the spec in source code snippets, for clarity, but it is still put in place and used in the header.

Public spec issues closed

• #21 - questions about session lifecycle state
  • session lifecycle further cleaned up
• #20 - XrPath is excessively UB-prone
  • Contradictory verbiage about "Undefined Behavior" removed/replaced.
  • Following up with #22 to finish making sure the text is unambiguous: the intent is for XrPath to not provoke Undefined Behavior.
• A number of issues about missing/mistaken return value codes
  • Issues:
    • #16
    • #15
    • #11
  • All were audited
  • A Collabora-contributed script now enforces consistency of many return codes.
• #7 - 12.13 Wayland section of KHR_opengl_enable refers to incorrect struct
  • Copy-paste typo fixed
• #6 - Section 2.9: Do runtimes have to detect invalid handles or not?
  • Clarified self-contradictory text that no, runtimes do not have to detect invalid handles: (in the absence of the validation layer) they may rely on handles being valid.
• #5 - Section 10.1: xrEnumerateSwapchainImages images parameter inconsistency
  • Implicit valid usage generator updated to understand the various *BaseHeader types and not add an impossible requirement.
• #4 - Buffer Size Parameter elementCountOutput inconsistency
  • Clarified self-contradictory text, establish that the count output for two-call-idiom calls must always be non-null.