Loading Content

Loading content like image, video, audio or text files is typically a part of any Cornerstone application.

Since accessing content on a physical media is very slow, Cornerstone aims to provide the tools for developers to create responsive user-interfaces that do not stall or stutter when loading new images or videos to display. If proper care is not taken, then just loading a single full HD image from a hard-disk can stall the user interface for seconds. This kind of application behaviour is annoying to the user and should be avoided.

Content-loading in Cornerstone

There are two ways to load content in Cornerstone: synchronous loading and asynchronous loading. Content-loading can be manually initiated by calling a load() function on widgets that support it. It can also be indirectly initiated by CSS rules, changing attributes, etc.

Synchronous Loading

Synchronous loading means that the thread initiating the loading (the calling thread) is blocked until the loading operation has completed. In case of the main-thread, this is usually undesired behaviour, because it will stall the user-interface. However, when used in another thread, such as Radiant::Task running in the background threads, this is perfectly valid and a simple way of loading content.

auto img = MultiWidgets::create<MultiWidgets::ImageWidget>();

// This will block until the loading has finished
if (img->load("picture.png")) {
  // image loading was successful, let's add the image to the scene
  app.mainLayer()->addChild(img);
} else {
  // error handling
}

Asynchronous Loading

Asynchronous loading means that the calling thread is not blocked while the loading operation is in progress. This is the desired behaviour in the main-thread where we do not want to stall the user-interface. The drawback of asynchronous loading is that the application must synchronize when the content can actually be displayed.

auto img = MultiWidgets::create<MultiWidgets::ImageWidget>();

// This will just initialize the loading, doesn't block.
img->load("picture.png");

// When the image header has been parsed in the background task, this callback
// is called from the main-thread. Instead of capturing img pointer in the
// lambda, pass the pointer as a parameter to the callback. Otherwise it's easy
// to have cyclic dependencies with Widget smart pointers.
img->onHeaderReady<MultiWidgets::ImageWidget>([&app] (MultiWidgets::ImageWidgetPtr widget) {
  app.mainLayer()->addChild(widget);
});

To implement error-handling with asynchronous loading, the MultiWidgets::BaseMediaWidget::onError callback can be used in a similar way to the MultiWidgets::BaseMediaWidget::onHeaderReady callback in the example above.

Content State

In Cornerstone, content-loading has a state associated with it. The states are represented by Valuable::LoadingEnum. The MultiWidgets::BaseMediaWidget, from which all standard content-loading widgets inherit from, provides several utility functions to define callbacks for different loading states.

A commonly used state is Valuable::STATE_HEADER_READY which is set when the content meta-data (including size) has been loaded. This state is typically reached very quickly, since no actual data has yet been loaded from the content. With the meta-data available, it is possible to perform layout for the widgets that will eventually contain the loaded content. This allows for responsive feedback to the user when content in the user-interface is changed. The utility function to set a callback for this state is MultiWidgets::BaseMediaWidget::onHeaderReady(). For images, this state indicates that the image size, format, etc. has been decoded. For videos, the size, format, frame-rate, codecs, etc. have been decoded.

Another commonly used state is Valuable::STATE_READY which is set when enough content has been loaded that it can be rendered. Reaching this state can take a considerable time. For example, in the case of an image, the image data has been loaded and is ready to be rendered. For videos, a single frame has been decoded. The utility function to set a callback for this state is MultiWidgets::BaseMediaWidget::onReady().

This separation of content-loading into different states gives developers more control over how content is shown in their application. It is possible to load content and wait for the entire data to be loaded before displaying it on-screen. However, with the header-ready state the application can, for example, display placeholder content or other feedback to the user that something is happening and the application stays responsive and reacts quickly to user-input.

Future Boolean

All content loading in Cornerstone is implemented asynchronously and all widgets that load content in Cornerstone use a Radiant::FutureBool object as their return value. This class can be though of as a standard constant bool type. It provides a simple mechanism to synchronize the loading: when the value of the Radiant::FutureBool is accessed, the calling thread is blocked until the value is available. If the value is not accessed, the loading happens asynchronously.

auto img = MultiWidgets::create<MultiWidgets::ImageWidget>();

// This will just initialize the loading, doesn't block because we don't access
// the value of result.
auto result = img->load("picture.png");

// Do something here, executed in parallel to load() from above
doStuff();

// This will block until load() has finished.
if(result)
  app.mainLayer()->addChild(img);
else
  Radiant::error("Failed to load image.");

The accessed bool value inside the FutureBool is true if loading was successful. At that point widget default and intrisic size is also set to match the content size. If the value is false, there was an error loading the content. Usual reason for that is corrupted or missing file.

How long the Radiant::FutureBool will block is determined by a parameter to the load() function returning it. Refer to the parameters of MultiWidgets::ImageWidget::load() and MultiWidgets::VideoWidget::load().