~unity-team/unity-scopes-api/trunk

/*!
\mainpage notitle

\section overview What are scopes

\subsection intro Introduction

One of Unity’s core features on the desktop is the Dash. The Dash allows users to search for and discover virtually anything,
from local files and applications to web content and other online data. The Dash achieves this by interfacing with one or more
search plug-ins called “scopes”, such as “Apps”, “Music”, “Videos”, “Amazon”, “Wikipedia”, and “Youtube”.

On the phone and tablet, scopes make up the central user interface, as they provide everything a user needs from an operating
system. Scopes enable users to locate and launch applications, access local files, play music and videos, search the web,
manage their favourite social network, keep up with the latest news, and much more.

Each scope is a dedicated search engine for the category of data it represents. The data source could be a local database,
a web service, or even an aggregation of other scopes. (For example, the “Music” scope aggregates “Local Music” and “Online Music”
scopes). A scope is responsible for performing the actual search and returning the best possible results for each
query it receives.

This document describes how to implement, test, and package your own scope using the Unity Scopes C++ API (unity-scopes-api).

\if RemoteScopes
\subsection LvsR Local vs remote scopes

Local scopes are scopes that are located and run on the user’s device, while remote scopes (or “Smart Scopes”) are scopes that
are located and run remotely on the Ubuntu Smart Scopes Server (or “SSS”). (Note: Although local scopes execute as local
processes, they may still query online services in order to retrieve search results). A local scope usually requires local data,
and therefore, can only be run locally, while a remote scope can effectively be run both locally and remotely.

When deciding on whether to write a local or remote scope, keep the user’s privacy in mind. For security reasons, a scope should
not access the user’s personal data unless absolutely necessary (i.e. The scope requires account information or local data to
perform searches). It is only in these situations that a scope should be written to run locally. <b>By default, a scope should
be written with the intention of running remotely on the Smart Scopes Server.</b>

(For more information on how to deploy your scope to the Smart Scopes Server, or how to implement a native remote scope using
the SSS REST API see: <i>link_not_yet_available</i>)
\endif

\section develop Developing scopes

\subsection starting Getting started

A simple C++ scope template with a cmake build system is currently available as part of the Ubuntu SDK IDE. To use it, install the
packages required for scope development:

\verbatim
sudo apt-get install libunity-scopes-dev
\endverbatim

Now you are ready to explore and modify the sample code in the src/ directory.

\subsection click Click packaging

To register your scope, you must use the "scope" click hook, and point it to a directory containing your .ini file and .so file.
In the template, a manifest such as the following is used:

\code
{
  "description": "Net scope description",
  "framework": "ubuntu-sdk-14.10-dev1",
  "hooks": {
    "myscope": {
      "scope": "myscope", <-- Point to directory in build tree with .ini and .so
      "apparmor": "scope-security.json" <-- Point to AppArmor manifest in build tree
    }
  }
  "maintainer": "Some Guy <some.guy@ubuntu.com>",
  "name": "com.ubuntu.developer.username.net-scope",
  "title": "Some scope",
  "version": "0.1"
}
\endcode

\subsubsection multiarch_click Multi-arch click packages
The search path for the shared library inside a click package is as follows:

<ul>
<li>${SCOPE_DIRECTORY}/${DEB_HOST_MULTIARCH}/lib${SCOPE_NAME}.so
<li>${SCOPE_DIRECTORY}/${DEB_HOST_MULTIARCH}/${SCOPE_NAME}.so
<li>${SCOPE_DIRECTORY}/${DEB_HOST_MULTIARCH}/scope.so
<li>${SCOPE_DIRECTORY}/lib${SCOPE_NAME}.so
<li>${SCOPE_DIRECTORY}/${SCOPE_NAME}.so
<li>${SCOPE_DIRECTORY}/scope.so
</ul>

Therefore click packges can be made multi-arch aware by compiling your targets for multiple architectures, and creating, e.g. the
following directory structure:

\code
├── testscopeA.ini
├── testscopeA-settings.ini
├── x86_64-linux-gnu
|   └── libtestscopeA.so
└── arm-linux-gnueabihf
    └── libtestscopeA.so
\endcode

You must also update the manifest file 'architecture' property to enumerate the included architectures, as follows:

\code
    "architecture": ["armhf", "amd64"]
\endcode 

\subsubsection multiarch-scoperunner Multi-arch scope runner
Similarly to the shared libraries, when a relative path is provided the search path for custom scope runners is as follows:

<ul>
<li>${SCOPE_DIRECTORY}/${DEB_HOST_MULTIARCH}/${CUSTOM_SCOPERUNNER}
<li>${SCOPE_DIRECTORY}/${CUSTOM_SCOPERUNNER}
</ul>

\subsubsection apparmor Apparmor manifest

Scopes that are packaged using click are inherently untrusted and must be confined. At present, there is only a single type of scope that can be defined:
<ul>
<li>Network scope - can access the network / internet, but is not allowed to use APIs that provide access to the user's data.
</ul>

The security manifest for this type of scope should be as follows:

\code
{
    "template": "ubuntu-scope-network",
    "policy_groups": [],
    "policy_version": 1.2
}
\endcode

\subsection impl Implementing a scope

This short tutorial covers the basic steps and building blocks needed to implement a scope in C++ with unity-scopes-api.
For complete examples of various scopes, see the `demo/scopes` subdirectory of the unity-scopes-api source project.

A typical scope implementation needs to implement interfaces of the following classes from the Scopes API:
<ul>
<li>unity::scopes::ScopeBase - the main scope class and entry point for all incoming requests
<li>unity::scopes::SearchQueryBase - the handler for search requests
<li>unity::scopes::PreviewQueryBase - the handler for preview requests (only if handling previews)
<li>unity::scopes::ActivationQueryBase - the handler for activation and preview action requests (only if handling previews and activation)
<li>unity::scopes::SearchListenerBase - the handler for search replies (only in aggreagator scopes, to pull results from other scopes)
</ul>

The following sections show explaing these in more detail.

\subsubsection simplescope Case 1: A simple (non-aggregating) scope

This is the typical case: a scope that connects to a remote or local backend, such as a database,
and provides results in response to search queries coming from a
client (that is, the Unity Dash or another scope).

\paragraph scopebase Implementing ScopeBase

You must create a class that derives from \link unity::scopes::ScopeBase ScopeBase\endlink and
implement a few methods. As a minimum, you must provide implementations of the
\link unity::scopes::ScopeBase::search search()\endlink and \link unity::scopes::ScopeBase::preview preview()\endlink methods. 

\code{.cpp}
using unity::scopes;

class MyScope : public ScopeBase
{
public:
    virtual void start(std::string const& scope_id) override;  // optional, dflt impl does nothing
    virtual void stop() override;                              // optional, dflt impl does nothing
    virtual void run() override;                               // optional, dflt impl does nothing
    virtual SearchQueryBase::UPtr search(CannedQuery const& query,
                                         SearchMetadata const& metadata) override;
    virtual PreviewQueryBase::UPtr preview(Result const& result,
                                           ActionMetadata const& metadata) override;
}
\endcode

The scopes runtime calls \link unity::scopes::ScopeBase::start() start()\endlink once prior to
sending any queries. You can use it to perform one-time initialization for your scope. Note that you should
not perform any lengthy operations in `start()`. Your implementation must return as quickly as possible
(in a fraction of the second), so do not, for example, make synchronous network requests as part of `start()`.

The scope ID passed to `start()` is taken from the name your scope's `.ini` configuration file.

The scopes runtime calls \link unity::scopes::ScopeBase::stop() stop()\endlink to inform your scope
that it should shut down. You can use this method to perform any one-time clean-up.

Prior to sending any queries, the scopes runtime calls \link unity::scopes::ScopeBase::run() run()\endlink if your `start()` method
completed successfully (did not throw an exception). The `run()` method
is called by a separate thread that you can use for your own purposes, for example, to run an event loop.
The scopes runtime has no further interest in this thread, but you must ensure that, if you do not return
from `run()`, you must return from `run()` in response to a call to `stop()`.

For typical and simple cases, you can ignore `run()`.

\paragraph handlingsearch Handling search

The unity::scopes::ScopeBase::search() method is called once for each query. Its purpose is to instantiate
and return a new C++ instance that encapsulates the query, that is, `search()` is a factory method.
(Do not start execution of the query as part of `search()`; the query object has a separate method for this.)

`search()` must return an instance of an object that implements the unity::scopes::SearchQueryBase interface, for example:

\code{.cpp}

class MyQuery : public SearchQueryBase { ... };

SearchQueryBase::UPtr MyScope::search(CannedQuery const& query, SearchMetadata const& metadata)
{
    SearchQueryBase::UPtr q(new MyQuery(query, metadata));
    return q;
}
\endcode

The search() method receives two arguments: a unity::scopes::CannedQuery query object that (among other information) carries the actual
query string, and
additional parameters for the search request, passed as unity::scopes::SearchMetadata. The metadata includes information such as
the current locale string, the form factor, and the query cardinality.

Cardinality is the maximum number of results expected from a query (a value of 0 indicates no limit). For optimal performance,
do not return more results than indicated by the cardinality. If you more than the requested number of results, you are
wasting resources. (The scopes runtime ignores the additional results.)

\paragraph handlingaggregation Handling aggregation

As previously stated, SearchMetadata contains additional information about the search requests you receive, including
the methods:

<ul>
<li>\link unity::scopes::SearchMetadata::is_aggregated is_aggregated()\endlink - true if the request was initiated by
    an aggregator,
<li>and \link unity::scopes::SearchMetadata::aggregated_keywords aggregated_keywords()\endlink - the list of keywords
    used by the aggregator to find your scope.
</ul>

\note Please refer to the <a href="https://developer.ubuntu.com/en/scopes/tutorials/scope-keywords/">Scope Keywords</a>
tutorial document for more detail on using keywords in your scope.

You can use the is_aggregated() method from within
\link unity::scopes::SearchQueryBase::run SearchQueryBase::run()\endlink in order to ensure that an appropriate set of
results are returned when queried by an aggregator:

\code{.cpp}
void MyQuery::run(SearchReplyProxy const& reply)
{
    if (metadata_.is_aggregated())
    {
        auto category = reply->register_category("agg_cat",
                                                 "MyScope Featured",
                                                 agg_icon);
        do_aggregated_search(reply, category);
    }
    else
    {
        do_normal_search(reply);
    }
}
\endcode

You may notice in the code snippet above that for each aggregated search we receive, we register a specific results
category. Although aggregators may be willing to accept more than one category from its child scopes, they are only
required to accept the first.

Thereafter, an aggregator may choose to ignore any additional categories the child scope registers. It is therefore
recommended that scope authors follow the above method of handling aggregated searches. It is also recommended that
your scope provide a decent category title (e.g. "MyScope Featured"). An aggregator is likely to display this category
title as is within its result set, so try to keep it clean and descriptive.

\paragraph surfacingmode Surfacing mode

The query string may be the empty string. If so, the UI is asking your scope to produce default results
that are shown in what is known as _surfacing mode_. These are the results the UI displays if the user
navigates to your scope, but has not entered a query yet. What results to show here depends on how your
scope works. For example, for a music scope, the default results could be something like "Most Popular"
and "Recently Played"; similarly, for a weather scope, the default results could be for the weather
report for the current location. As the scope author, you need to decide what is most appropriate
to show in surfacing mode. In the interests of a good user experience, it is important to show _something_
here (if at all possible), so the user gets to see at least some results (instead of being confronted with
a blank screen).

The runtime automatically saves the results of the most recent surfacing query. If a scope cannot produce
a result for a surfacing query (presumably, due to connectivity problems), calling
\link unity::scopes::SearchReply::push_surfacing_results_from_cache() push_surfacing_results_from_cache()\endlink
pushes the results that were produced by the most recent successful
surfacing query from the cache. If your scope cannot produce surfacing results, you can call this
method to "replay" the results of the previous surfacing query. In turn, this avoids the user
being presented with an empty screen if he/she swipes to the scope while the device does not have connectivity.

`push_surfacing_results_from_cache()` has an effect only if called for a surfacing query (that is, a query with an empty query string). If called for a non-empty query, it does nothing.

You must call this method before calling \link unity::scopes::Reply::finished finished()\endlink,
otherwise no cached results will be pushed. (`push_surfacing_results_from_cache() implicitly calls `finished()`);

\paragraph querybase Implementing QueryBase

You must implement a class that derives from \link unity::scopes::SearchQueryBase SearchQueryBase\endlink and return
an instance of this class from `search()`.
Your class must implement a \link unity::scopes::SearchQueryBase::run run()\endlink method. The scopes runtime calls
`run()` to execute the query.

The \link unity::scopes::SearchReplyProxy SearchReplyProxy\endlink that is passed to `run()` is an invocation
handle that allows you to push results for the query back towards the client. (`SearchReplyProxy` is a `shared_ptr`
to a \link unity::scopes::SearchReply SearchReply\endlink object.)

Two important methods of `SerchReply` are \link unity::scopes::SearchReply::register_category register_category()\endlink
and \link unity::scopes::SearchReply::push push()\endlink.

`register_category()` is a factory method that registers new categories for the results of this query (see
\link unity::scopes::Category\endlink). You can create new categories at any point during query processing. 
Categories serve to visually group query results in some way;
when you push results for a query, you indicate which
category each particular result belongs to, and the UI renders that result in the corresponding
visual group. Categories are rendered in the order in which they are encountered by the UI
as you push your results. If you want to control the order in which categories are rendered
(for examples, such that a "Breaking News" category always appears first), you may need to
buffer the results you receive from your back-end data source until you get a result for that
category, and then push that result, plus any other buffered results.

Pre-registering categories is the preferred approach because it allows the UI to reserve space
and perform layout chores before any query results arrive. (In turn, this permits the UI to
optimize its operation.) However, for some data sources, it may not be possible to determine
all of the possible categories in advance, in which case you have no choice but to create
new categories as they arrive in the data from your scope's data source.

Do _not_ wait for all results for a query to arrive in an attempt to buffer them and order
them by category. If you do, this prevents incremental rendering, and the user sees nothing
until your scope has processed _all_ results. To create a positive user experience,
your scope should push results as soon as possible.

The UI uses categories to incrementally render the display after a refresh of search results. This relies
on categories staying the same from query to query. If your scope has, say, a "News" category, you need
to make sure that the category ID and name stay the same from query to query. In particular, do _not_
create category IDs that are artificially unique per query (such as by appending a sequence number).

When you create a category, you can provide a \link unity::scopes::CategoryRenderer \endlink instance.
The category renderer determines the visual appearance of the results in that category (such as
display in a grid or in a carousel layout).

You must wrap each actual search result inside a \link unity::scopes::CategorisedResult CategorisedResult\endlink object and
pass the result instance to \link unity::scopes::SearchReply::push push\endlink.

A typical implementation of `run()` might look like this:

\code{.cpp}
void MyQuery::run(SearchReplyProxy const& reply)
{
    if (!valid())
    {
        return;  // Query was cancelled
    }

    auto category = reply->register_category("recommended", "Recommended", icon);
    //... query a local or remote backend

    for (auto res : backend.get_results(query().query_string())) // for every result returned by a backend
    {
        ...
        CategorisedResult result(category); // create a result item in "recommended" category
        result.set_uri(...);
        result.set_title(...);
        result.set_art(...);
        result.set_dnd_uri(...);
        result["my-custom-attribute"] = Variant(...); // add arbitrary data as needed

        if (!reply->push(result)) // send result to the client
        {
            break; // false from push() means that the search was cancelled
        }
    }
}
\endcode

As far as the UI is concerned, the query is complete when `run()` returns. (While the query can potentially
return more results, the UI shows a spinner or similar, to indicate that the query is not complete yet.)

It is possible for you to return from `run()` _without_ having the query complete automatically. The life time
of the query is controlled not only by `run()`, but also by the life time of the `SearchReplyProxy` that is passed
to `run()`. The scopes runtime monitors the reply proxy and informs the UI that the query is complete when
_either_ `run()` returns _or_ the last reply proxy for the query goes out of scope. This allows you to, for example,
pass the reply proxy to a different thread that pushes results (as you might want to do if you need to run a
separate event loop). That thread can then also react to query cancellation. The important point is that, if
you keep copies of the reply proxy, the query will remain alive until you destroy all copies of the reply proxy
for that query (or explicitly call `finished()` on the reply proxy yourself, which explicitly ends the query).

\paragraph cancellation Query cancellation

It is possible for the UI to cancel a query before the query has completed and is still running in your scope, potentially
producing additional results.
Typically, this happens because the user has typed a few characters as the search term (which creates a query for
the string up to that point); shortly after this, the user might type another character or two, extending the search string.
After a short idle period, the UI cancels the original query and creates a new query for the extended search string.
However, the second query will not start until _after_ the previous query has completed.

\note _Query cancellation happens frequently, and it is important for your scope to react quickly to cancellation!_

The scopes runtime provides several ways for your implementation to react to cancellation:

<ul>
<li>A `false` return value from `SearchReply::push`. If `push` returns `false`, there is no point in continuing to
provide more results.
<li>You can poll for cancellation by calling \link unity::scopes::QueryBase::valid QueryBase::valid()\endlink. `valid()`
returns `false` once a query is cancelled or has exceeded its cardinality limit.
<li>Your query implementation class must override the \link unity::scopes::QueryBase::cancelled QueryBase::cancelled()\endlink
method. The scopes runtime calls `cancelled()` if the UI has cancelled the query. (Note that calls to `cancelled()` are
made by a separate thread.)
</ul>

Testing the return value from `push()` is reasonable only if you know that results for your scope will
arrive quickly (no more than 0.1 seconds apart). Otherwise, you should push results asynchronously from
a separate thread and arrange for the query to complete (return from `run()`) in response to the scopes
runtime calling `cancelled()`.

Note that it is possible for a call to `cancelled()` to arrive before the scopes runtime has called `run()`
(because `cancelled()` and `run()` are called by different threads and, therefore, can be dispatched out of order).


\paragraph filters Filters

Scopes API offers support for filter widgets, which provide means for filtering search results based on user input other than search query string
alone. Filter widgets need to be defined by creating appropriate filters inside the overriden SearchQueryBase::run() method, and then pushed
to the UI. It is recommended to push filters early before search results are pushed for best user experience.

Here is an example of how filters can be created:

\code{.cpp}
void run(SearchReplyProxy const& reply)
{
    OptionSelectorFilter::UPtr filter1 = OptionSelectorFilter::create("brand", "Brand");
    filter1->add_option("audi", "Audi");
    filter1->add_option("bmw", "BMW");

    RangeInputFilter::SPtr filter2 = RangeInputFilter::create("price", Variant(0.0f), Variant::null(), "Min", "", "", "Max", "");
    ValueSliderFilter::SPtr filter3 = ValueSliderFilter::create("horsepower", 1, 135, 50, ValueSliderLabels("Min", "Max"));

    Filters filters;
    filters.push_back(filter1);
    filters.push_back(filter2);
    filters.push_back(filter3);

    reply->push(filters, query().filter_state());

    // push search results here
\endcode

Scopes are free to change filters at any time - with every execution of search the scope can omit any of the previously visible filters or add new ones, if that
makes sense for particular use cases.

Filters act only as UI widgets - it is the responsibility of the scope to check their state and actually apply them to search results. The current value of a filter
becomes just another parameter of the search query that needs to be taken into account in the implementation of search handling inside run().

To examine current state of the filters, pass the instance of unity::scopes::FilterState received with search query to respective methods of
the filters. For example:

\code{.cpp}
void run(SearchReplyProxy const& reply)
{
    // filter creation code omitted here
    auto state = query().filter_state();
    int search_start = 0;
    int search_end = 1000;
    if (rangefilter->has_start_value(state)) {
        search_start = rangefilter->start_value(state);
    }
    if (rangefilter->has_end_value(state)) {
        search_end = rangefilter->end_value(state);
    }

    // apply search_start and search_end to search logic
\endcode

The scope may nominate a single filter to act as "primary navigation". This is only possible if departments are not used at the same time (in which case departments
become a primary navigation tool). An attempt to nominate a filter to be a "primary navigation" while departments are present is ignored by the UI and the
filter acts as a regular filter. Also, only a single-selection OptionSelectorFilter can currently be promoted to be primary navigation. To do this, set the display
hints to FilterBase::DisplayHints::Primary:

\code{.cpp}
OptionSelectorFilter::UPtr filter1 = OptionSelectorFilter::create("brand", "Brand");
filter1->set_display_hints(FilterBase::DisplayHints::Primary);
filter1->add_option("audi", "Audi");
\endcode

When a filter becomes a primary navigation filter, it gets displayed in the search box drop-down, below recent searches, so it's readily available for quick access.
Also, currently selected option is displayed as a "brick" in the search box, hinting the user about the context of current search. All the other filters can be
revealed via the filters panel icon.

\paragraph handlingpreview Handling previews

Your scope is responsible for handling preview requests for results it has returned; you implement this by overriding
the unity::scopes::ScopeBase::preview() method:

\code{.cpp}
class MyScope : public unity::scopes::ScopeBase
{
public:
    ...
    virtual PreviewQueryBase::UPtr preview(Result const& result, ActionMetadata const& metadata) override;
    ...
}
\endcode

This method must return an instance derived from unity::scopes::PreviewQueryBase. Like `search()`, `preview()` is a factory method;
the scopes runtime initiates the actual preview by calling \link unity::scopes::PreviewQueryBase::run run()\endlink on the instance you
return. Your `run()` method is responsible for gathering preview data (from
local or remote sources) and passing it to the UI along with the definition of the visual appearance of the preview by calling
\link unity::scopes::PreviewReply::push push()\endlink on the reply proxy that is passed to `run()`. (This is analogous to returning
results from `search()`.)

A preview consists of one or more preview widgets. Preview widgets are the basic building blocks for previews, such as a header with a title and subtitle, an image, a
gallery with multiple images, a list of audio tracks, and so on.(See unity::scopes::PreviewWidget for a list of supported widget types.)
Your implementation of \link unity::scopes::PreviewQueryBase::run run()\endlink must create and populate one or more preview widgets
and push them to the UI.

Each preview widget has a unique identifier, a type name, and a set of attributes determined by the widget's type.
For example, a widget of "image" type expects two attributes: "source" (a URI that should point at an image),
and a "zoomable" flag that determines if the image should be zoomable.
You can specify the values of these attributes explicitly, or you can arrange for the values to be taken from a result
that the corresponding query returned earlier, by referencing the corresponding \link unity::scopes::Result Result\endlink instance.
You can also push the value for a referenced attribute separately as part of your implementation of `run()`.

You provide attributes explicitly by calling
\link unity::scopes::PreviewWidget::add_attribute_value PreviewWidget::add_attribute_value()\endlink:

\code{.cpp}
PreviewWidget image_widget("myimage", "image");

image_widget.add_attribute_value("source", Variant("file:///tmp/image.jpg"));
image_widget.add_attribute_value("zoomable", Variant(false));
\endcode

To reference values from results or arbitrary values that you push separately, use
\link unity::scopes::PreviewWidget::add_attribute_mapping PreviewWidget::add_attribute_mapping()\endlink:

\code{.cpp}
PreviewWidget image_widget("myimage", "image");
image_widget.add_attribute_mapping("source", "art"); // use 'art' attribute from the result
image_widget.add_attribute_mapping("zoomable", "myzoomable"); // 'myzoomable' not specified, but pushed below
reply->push("myzoomable", Variant(true));
\endcode

To push preview widgets to the client, use \link unity::scopes::PreviewReply::push PreviewReply::push()\endlink:

\code{.cpp}
PreviewWidget image_widget("myimage", "image");
PreviewWidget header_widget("myheader", "header");
// fill in widget attributes
...
PreviewWidgetList widgets { image_widget, header_widget };
reply->push(widgets);
\endcode

\paragraph previewactions Preview actions

Previews can have actions, such as buttons that the user can press. Actions are supported by a preview widget of type "actions".
An actions widget holds one or more action button definitions, where each definition has a unique identifier, a label,
and an optional icon. For example, a widget with two buttons, "Open" and "Download", can be defined as follows
(using the \link unity::scopes::VariantBuilder VariantBuilder\endlink helper class):

\code{.cpp}

PreviewWidget buttons("mybuttons", "actions");

VariantBuilder builder;
builder.add_tuple({
    {"id", Variant("open")},
    {"label", Variant("Open")}
});
builder.add_tuple({
    {"id", Variant("download")},
    {"label", Variant("Download")}
});

buttons.add_attribute_value("actions", builder.end());
\endcode

To respond to activation of preview actions, your scope must implement
\link unity::scopes::ScopeBase::perform_action ScopeBase::perform_action\endlink:

\code{.cpp}
class MyScope : public ScopeBase
{
    ...
    virtual ActivationQueryBase::UPtr perform_action(Result const& result,
                                                     ActionMetadata const& metadata,
                                                     std::string const& widget_id,
                                                     std::string const& action_id) override
    ...
}
\endcode

Like `search()` and `preview()`, `perform_action()` is a factory method. It must return an
instance that derives from \link unity::scopes::ActivationQueryBase ActivationQueryBase\endlink.
Your derived class must implement the \link unity::scopes::ActivationQueryBase::activate activate()\endlink method,
whose job it is to respond to the activation (that is, the user pressing a button). `activate` must return
an \link unity::scopes::ActivationResponse ActivationResponse\endlink, which tells the UI how it should
behave in response to the activation. For example, your `activate()` could direct the UI to
run a new search as follows:

\code{.cpp}
class MyActivation : public ActivationQueryBase
{
    MyActivation(Result const& result, unity::scopes::ActionMetadata const& metadata) :
        ActivationQueryBase(result, metadata)
    {
    }

    virtual ActivationResponse activate() override
    {
        ...
        if (action_id() == "search-grooveshark")
        {
            CannedQuery query("com.canonical.scopes.grooveshark");
            query.set_query_string("metal");
            return ActivationResponse(query);
        }
        ...
    }
};
\endcode

\paragraph handlingactivation Handling result activation

In many cases, the user can activate search results directly, by tapping on them, provided the result's schema (such as "http://")
has a handler in the system. If this is the case, you need not do anything for activation. However, if your scope uses
schemas without a handler, the shell will ignore the activation. (Nothing happens in response to a tap by the user.)

If you want to intercept such activations (either for schemas without a handler, or to generally intercept result activation),
you must implement the \link unity::scopes::ScopeBase::activate ScopeBase::activate()\endlink method:

\code{.cpp}
class MyScope : public ScopeBase
{
    virtual ActivationQueryBase::UPtr activate(Result const& result,
                                               ActionMetadata const& metadata) override;
    ...
}
\endcode

In addition, you must call \link unity::scopes::Result::set_intercept_activation Result::set_intercept_activation()\endlink
for all results that should trigger a
call to your `activate()` method. Your implementation of `activate()` should follow the same guidelines as for
`perform_action()` (except that widget and action identifiers do not apply to result activation).

\paragraph resultaction Handling result action activation

Search results can embed simple action buttons (icons) in their cards. When the user taps an action icon, the scope gets notified and can update
the affected result card to reflect the new state. A typical use case for this is to offer "social" actions, such as thumbs up/down buttons.

The following snippet demonstrates how two actions can be added to a \link unity::scopes::CategorisedResult CategorisedResult\endlink:

\code
CategorisedResult res(cat);
res.set_uri("myuri");
res.set_title("My Title");

// Add result actions
VariantBuilder builder;
builder.add_tuple({
        {"id", Variant("thumbsup")},
        {"icon", Variant("thOff")},
        {"temporaryIcon", Variant("thOn")},
        {"label", Variant("I like it")},
});
builder.add_tuple({
        {"id", Variant("flag")},
        {"icon", Variant("flag")},
        {"temporaryIcon", Variant("flagOn")},
        {"label", Variant("Flag this result")},
});
res["social-actions"] = builder.end();
\endcode

The attributes of result actions are as follows:
<ul>
<li>id - unique action identifier that will be reported to the scope.
<li>icon - the icon for the action button.
<li>temporaryIcon (optional) - defines an icon that will be shown immediately when the user taps the button, before the scope reacts to the action.
<li>label - the text shown next to the icon.
</ul>

To respond to activation of result actions, your scope must implement
\link unity::scopes::ScopeBase::activate_result_action ScopeBase::activate_result_action\endlink:

\code{.cpp}
class MyScope : public ScopeBase
{
    ...
    ActivationQueryBase::UPtr activate_result_action(Result const& result,
            ActionMetadata const& metadata,
            std::string const& action_id) override;
    ...
}
\endcode

Like `search()` and `preview()`, `activate_result_action()` is a factory method. It must return an
instance that derives from \link unity::scopes::ActivationQueryBase ActivationQueryBase\endlink.
Your derived class must implement the \link unity::scopes::ActivationQueryBase::activate activate()\endlink method,
whose job it is to respond to the activation (that is, the user pressing action button). `activate` must return
an \link unity::scopes::ActivationResponse ActivationResponse\endlink, which tells the UI how it should
behave in response to the result action activation. For result actions the typical and recommended behavior is to update the card
for the result whose action was activated.

For example, here is how to update the actions of an affected card in response to a "thumbsup" action, so that tapping the "thumbsup" action button replaces that
button with "thumbsdown":

\code{.cpp}
class MyActivation : public ActivationQueryBase
{
    MyActivation(Result const& result, unity::scopes::ActionMetadata const& metadata, std::string const& action_id) :
        ActivationQueryBase(result, metadata, action_id)
    {
    }

    virtual ActivationResponse activate() override
    {
        if (action_id() == "thumbsup")
        {
            // ... update backend data for 'thumbs up' action ...

            // get the affect result and update it
            Result updatedRes(result());
            VariantBuilder builder;
            builder.add_tuple({
                    {"id", Variant("thumbsdown")},
                    {"icon", Variant("thOn")},
                    {"temporaryIcon", Variant("thOff")},
                    {"label", Variant("I don't like it")},
            });
            builder.add_tuple({ ... })
            updatedRes["social-actions"] = builder.end();
            return ActivationResponse(updatedRes);
        }

        if (action_id() == "thumbsdown")
        ...
    }
};
\endcode

\paragraph export Exporting a scope

Your scope must be compiled into a `.so` shared library and, to be successfully
loaded at runtime, it must provide two C functions to create and destroy it.
A typical code snippet to do this looks as follows:

\code{.cpp}
extern "C"
{
    unity::scopes::ScopeBase* UNITY_SCOPE_CREATE_FUNCTION()
    {
        return new MyScope();
    }

    void UNITY_SCOPE_DESTROY_FUNCTION(unity::scopes::ScopeBase* scope_base)
    {
        delete scope_base;
    }
}
\endcode

\paragraph inlineplayback Inline music playback

Results which represent music (songs, albums etc.) can contain an extra data about audio content and can then be played directly from the Dash.
Such results have a "play" button overlaid on them. To create results that support this functionality two conditions must be met:
<ul>
<li>Category renderer definition must contain the "quick-preview-type" key with the value of "audio" in the "template" section;
<li>Results in the respective category must contain a "quick-preview-data" attribute, each of them is a dictionary with the extra playback data described
below.
</ul>

The data assigned to "quick-preview-data" attribute of a Result needs to contain the following keys:
<ul>
<li>uri - a playable uri of a media file (path of a local file, or http uri).
<li>duration - the duration of the media file, in seconds.
<li>playlist - an array of uris of additional songs, e.g. songs from same album; they will be played in sequence when the main
song denoted by 'uri' finishes.
</ul>

Here is an example of a category renderer for inline playback, which uses component mapping to map quick-preview-data to audio-data attribute of a result:
\code{.cpp}
static const char CATEGORY_RENDERER[] = R"(
{
  "schema-version": 1,
  "template": {
    "category-layout": "grid",
    "card-size": "large",
    "card-layout" : "horizontal",
    "quick-preview-type" : "audio"
  },
  "components": {
    "title": "title",
    "art": {
      "field": "art"
    },
    "subtitle": "artist",
    "quick-preview-data": {
        "field": "audio-data"
    }
  }
}
)";
\endcode

A sample code that creates a result card representing a song and all songs from same album in a background playlist may look this way:
\code{.cpp}

CategorisedResult res(category);
res.set_uri(uri);
res.set_title(media.getTitle());
...

VariantMap inline_playback_data;
inline_playback_data["uri"] = uri;
inline_playback_data["duration"] = song_duration_in_seconds;
VariantArray playlist;
for (const std::string& song: album_songs)
{
    playlist.push_back(Variant(song.getUri()));
}
inline_playback_data["playlist"] = playlist;
res["audio-data"] = inline_playback_data;

\endcode

\subsubsection aggscope Case 2: An aggregating scope

Aggregating scopes are scopes that collect results from other scopes and possibly consolidate, modify, or re-categorise
the results in some way. In other words, for an aggregating scope, the data source(s) are other scopes rather than,
say, a remote web service.

To receive results from its child scopes, your scope must implement a class that derives from
\link unity::scopes::SearchListenerBase SearchListenerBase\endlink. You provide an instance
of this class to each sub-query; the scopes runtime invokes callback methods on this class
to let you know when a new result or status update arrives, and when a query completes.

\paragraph childscopes Finding child scopes

To send queries to its child scopes, your scope must obtain a proxy for each child scope. The scopes runtime
runs a registry process. The job of the registry (among other things) is to provide information about available
scopes (whether they are local scopes or remote scopes in the Smartscopes server).

You can obtain the proxy for a child scope by calling  \link unity::scopes::Registry::get_metadata get_metadata()\endlink
on the registry, supplying the ID of the child scope. The return value is an instance of type
\link unity::scopes::ScopeMetadata ScopeMetadata\endlink that describes the scope and also provides access to the
proxy for the scope.

You can also aggregate scopes indirectly via keyword(s). Keywords describe the type of content a scope provides (e.g.
a scope with the keyword "music" will return music results, the "video" keyword indicates video content, and so on).
You can obtain child scopes via keywords by calling \link unity::scopes::Registry::list_if list_if()\endlink on the
registry, supplying a predicate function. The return value is a map containing only those scopes for which the predicate
returns true. Therefore, your predicate function should return true for all scopes matching the keyword(s) you wish to
aggregate.

\note Please refer to the <a href="https://developer.ubuntu.com/en/scopes/tutorials/scope-keywords/">Scope Keywords</a>
tutorial document for a list of recommended keywords to use.

As an aggregator scope author you must provide an implementation of the virtual
\link unity::scopes::ScopeBase::find_child_scopes ScopeBase::find_child_scopes()\endlink method. All logic for finding
your aggregator's child scopes should be implemented within this method. The return value is of type
\link unity::scopes::ChildScopeList ChildScopeList\endlink and must contain an instance of
\link unity::scopes::ChildScope ChildScope\endlink for each scope your aggregator may collect results from.

Here is how you could implement find_child_scopes() to return all scopes in the registry that contain the keywords
"sports" and "news":

\code{.cpp}
ChildScopeList MyScope::find_child_scopes() const override
{
    auto sportsnews_scopes = registry()->list_if([](ScopeMetadata const& item)
    {
        auto keywords = item.keywords();
        return (keywords.find("sports") != keywords.end()) &&
               (keywords.find("news") != keywords.end());
    });

    ChildScopeList list;
    for (auto const& sportsnews_scope : sportsnews_scopes)
    {
        list.emplace_back(ChildScope{sportsnews_scope.first,  // Child scope ID
                                     sportsnews_scope.second, // Child scope metadata
                                     true,                    // Default enabled state (when first discovered)
                                     {"sports", "news"}});    // Keywords used to aggregate this scope
    }
    return list;
}
\endcode

\paragraph subquery Sub-queries

To send a query to another scope, use one of the `subsearch()` overloads of \link unity::scopes::SearchQueryBase\endlink
inside your implementation of
\link unity::scopes::SearchQueryBase::run SearchQueryBase::run()\endlink.
This method requires a handle to the child scope to query (either via proxy or ChildScope handle), the query details
(\link unity::scopes::CannedQuery CannedQuery\endlink), plus an instance of your `SearchListenerBase` implementation
that will receive the query results.

\note `subsearch()` is identical to
\link unity::scopes::ScopeBase::search search()\endlink but, for `subsearch()`, the scopes runtime transparently
forwards query cancellation to child scopes, so your implementation of \link unity::scopes::QueryBase::cancelled QueryBase::cancelled()\endlink
does not need to forward cancellation to its children.
(However, your query class still needs to react to cancellation and should terminate the current query
is quickly as possible in response to a cancelled message.)

You should always call \link unity::scopes::ScopeBase::child_scopes ScopeBase::child_scopes()\endlink from within your
aggregator's \link unity::scopes::ScopeBase::search search()\endlink method in order to retrieve the latest child
scopes list containing the most recent "enabled" states. You can then pass this list into your instantiation of
SearchQueryBase for later use.

\note An aggregator must respect the "enabled" states of its child scopes, querying only the child scopes that are
enabled.

Here is how you could implement an aggregating scope that passes a query to a single child scope "scope-A":

\code{.cpp}

ChildScopeList MyScope::find_child_scopes() const override
{
    auto reg = registry();  // Up-call into base class
    if (!reg)
    {
        throw ConfigException(scope_id + ": No registry available, cannot locate child scopes");
    }

    ChildScopeList list;
    try
    {
        auto meta = reg->get_metadata("scope-A");
        list.emplace_back(ChildScope{"scope-A",  meta});
    }
    catch (NotFoundException const& e)
    {
        ...
    }
    return list;
}

QueryBase::UPtr MyScope::search(CannedQuery const& query,
                                SearchMetadata const& metadata)
{
    SearchQueryBase::UPtr q(new MyQuery(query, metadata, child_scopes()));
    return q;
}

...

void MyQuery::run(SearchReplyProxy const& upstream_reply) 
{
    // Continue only if our child scope is installed AND enabled
    if (!child_scopes_.empty() && child_scopes_.front().enabled)
    {
        auto category = reply->register_category("recommended", "Recommended", icon, "");
        SearchListenerBase::SPtr reply(new MyReceiver(upstream_reply, category));
        subsearch(child_scopes_.front(), query_, reply);
    }
}
\endcode

Note that the `subsearch()` call is asynchronous and returns immediately. Despite this, your `MyQuery` instance is kept alive
because the scopes runtime does not delete it until the child query has completed. (The runtime tracks the `reply` proxy
for the query and holds the query alive until it receives a finished message from the child scope.)

\paragraph receiver Receiving sub-query results

Here is a simple implementation of a receiver that passes all child categories and results through to
its parent without change. Of course, a more realistic aggregating scope will typically aggregate from
more than one child and probably de-duplicate, collate, or otherwise modify child results before passing
them upstream.

\code{.cpp}

class MyReceiver: public SearchListenerBase
{
public:
    virtual void push(Category::SCPtr const& category) override
    {
        upstream_reply_->register_category(category);
    }

    virtual void push(CategorisedResult result) override
    {
        upstream_reply_->push(std::move(result));
    }

    MyReceiver(SearchReplyProxy const& upstream_reply) :
        upstream_reply_(upstream_reply)
    {
    }

private:
    SearchReplyProxy upstream_reply_;
};
\endcode

\paragraph aggbuffering Controlling category order

Categories are displayed in the order their results are pushed. This can pose a challenge
for aggregator scopes because results from child scopes often arrive
in random order. To control the order in which categories are rendered,
the aggregator must buffer and potentially re-order results by
category before pushing them.

\link unity::scopes::utility::BufferedResultForwarder BufferedResultForwarder\endlink makes
it easier to do this.
To use the class, you create one instance for each child scope and chain the instances together
in the desired order of categories. Each forwarder buffers results until its predecessor in
the chain indicates that it has completed its category ordering, at which point it itself pushes
any results it has buffered so far and indicates to its follower that it is ready.

By default, a forwarder indicates that it it is ready (has completed ordering) as soon as it has received
a single result. This is useful if an aggregator has child scopes that produce results
for a single category each. In this case, the order of the forwarders determines which
category (the one used by child A or by child B) appears first when the results are rendered.

If an aggregator collates results from children that each produce results for more than
one category, you can override the default implementation of
\link unity::scopes::utility::BufferedResultForwarder::push push()\endlink to change
categories for results from its child, and/or indicate
that it is ready only once the child has provided results for all expected categories.
(See \link unity::scopes::utility::BufferedResultForwarder BufferedResultForwarder\endlink
for more details.)

\paragraph aggactiv Activation and preview

If an aggregator scope simply forwards the results it receives from other scopes (possibly changing their category),
the aggregator need not do anything special for previews, preview actions, or result activation. In this case,
previews, preview actions, and result activation are sent to the scope that produced the corresponding result.

If, however, an aggregator scope changes attributes of results (or creates completely new results that "replace"
received results), you must take extra care:
<ul>
<li>If the original original scope should still handle preview (and activation) requests for a modified result, you must store
a copy of the original result in the modified (or new) result by calling \link unity::scopes::Result::store Result::store()\endlink.
Preview requests for such a result will automatically trigger the scope that created the innermost stored result.

\note Making changes to a receive result but failing to store the original result with the change can cause
in unexpected behavior: a scope could receive a modified result and, depending the exact changes, may
not be able to correctly deal with the result.

<li>If an aggregator creates a completely new result that replaces the original result but does not also store a copy of
the original result, the aggregator _must_ handle preview and activation requests (if the intercept flag is set).
The actions to take are the same as for a non-aggregating scope (see \ref handlingpreview and \ref handlingactivation).
</ul>

Here is an example `push()` implementation that modifies a result and stores a copy, so the original scope
can handle preview and activation:

\code{.cpp}
void MyReceiver::push(CategorisedResult original_result)
{
    // agg_category is a category that aggregates all results from other scopes
    CategorisedResult result(agg_category);
    result.set_uri(original_result.uri());
    result.set_title(original_result.title() + " (aggregated)");
    result.set_art(original_result.art());
    result.store(original_result);

    upstream_->push(std::move(result));
}
\endcode

\subsection threading Threading model

It is important to understand how the runtime uses threads to call methods on scopes and clients.
The runtime maintains a number of threads that each call one or more methods. Methods in the same
group are always called by the same thread. This means that methods in the same group do not run concurrently,
but methods in different groups _do_ run concurrently. If you share state between methods in
different groups, you _must_ synchronize access to that state, otherwise your code will suffer from
race conditions.

The following lists shows how methods are grouped. Each group has a single dedicated dispatch thread.

<ul>
<li>\link UNITY_SCOPE_CREATE_FUNCTION()\endlink,
\link unity::scopes::ScopeBase::start() ScopeBase::start()\endlink,
\link unity::scopes::ScopeBase::stop() ScopeBase::stop()\endlink, and
\link UNITY_SCOPE_DESTROY_FUNCTION()\endlink.
<li>\link unity::scopes::ScopeBase::search() ScopeBase::search()\endlink,
\link unity::scopes::ScopeBase::preview() ScopeBase::preview()\endlink, and
\link unity::scopes::ScopeBase::perform_action() ScopeBase::perform_action()\endlink.
<li>\link unity::scopes::SearchQueryBase::run() SearchQueryBase::run()\endlink,
\link unity::scopes::PreviewQueryBase::run() PreviewQueryBase::run()\endlink, and
\link unity::scopes::ActivationQueryBase::activate() ActivationQueryBase::activate()\endlink.
<li>\link unity::scopes::QueryBase::cancelled() QueryBase::cancelled()\endlink.
<li>\link unity::scopes::SearchListenerBase::push() SearchListenerBase::push()\endlink,
\link unity::scopes::PreviewListenerBase::push() PreviewListenerBase::push()\endlink,
\link unity::scopes::ActivationListenerBase::activated() ActivationListenerBase::activated()\endlink,
\link unity::scopes::ListenerBase::finished() ListenerBase::finished()\endlink,
\link unity::scopes::ListenerBase::info() ListenerBase::info()\endlink.
</ul>
For your scope implementation, keep in mind that `cancelled()` is _not_ called by
the same thread that called, for example, `search()` or `run()`. This means that any state you
established in `search()` or `run()` must be synchronized if you use that state in `cancelled()`.

Similar considerations apply for aggregating scopes, which act as both client and server: you must
synchronize any state that is shared between the client side and the server side. For example,
you must synchronize state established in `search()` and accessed from `push()` or `finished()`.

\subsection scopesettings Settings

A scope can provide for simple customizations, such as allowing the user to configure an email address or
select a distance unit as metric or imperial.

\subsubsection settings_definitions Defining settings

You can define such settings in a configuration file.
The file must be placed into the same directory as the scope's normal
configuration file, with the name `<scope-name>-settings.ini`. For example, for a scope with ID `com.acme.myscope`,
the normal configuration file is `com.acme.myscope.ini`, and the settings definition file is
`com.acme.myscope-settings.ini`. Both files must be installed in the same directory (together with the scope's `.so` file).

The shell constructs a user interface from the settings definitions. The user can change settings via that UI.
The scope can retrieve the actual setting values at run time (see \ref read_settings).

The following types are supported for settings:

<ul>
<li>`string` - a string value
<li>`number` - a numeric value (integer or floating point)
<li>`boolean` - `true` or `false`
<li>`list` - a list of alternatives to choose from (single-choice)
</ul>

It is possible to optionally define a default value for each setting.

Here are the contents of an example definition file:
~~~
[location]
type = string
defaultValue = London
displayName = Location

[distanceUnit]
type = list
defaultValue = 1
displayName = Distance Unit
displayName[de] = Entfernungseinheit
displayValues = Kilometers;Miles
displayValues[de] = Kilometer;Meilen

[age]
type = number
defaultValue = 23
displayName = Age

[enabled]
type = boolean
defaultValue = true
displayName = Enabled

# Setting without a default value
[color]
type = string
displayName = Color
~~~

The file must contain a group for each setting.
The order of the groups determines the display order for the user interface
that is constructed by the shell. The group name is the ID of the corresponding setting.

Each setting definition must contain at least the following mandatory definitions:
<ul>
<li> `type` - Defines the type of the setting (`string`, `number`, `boolean`, or `list`).
<li> `displayName` - Defines a display name that is shown for this setting by the shell.
</ul>

The defaultValue field is optional. If present, it defines
a default value that is provided to the scope if the user has not changed anything (or has never used the settings UI
before using the scope).
It is possible to test for settings that do not have a default value and
were never set by the user (see \ref read_settings).

For settings of type `list`, the `displayValues` field is mandatory. It must contain an array
that lists the available choices. If you provide a default value,
it must be in the range `0..max-1` (where `max` is the number of choices).

The `displayName` and `displayValues` fields can be localized by appending a locale identifier
in square brackets. If no entry can be found that matches the current locale, the non-localized value
is used.

\subsubsection read_settings Accessing settings

The settings that are currently in effect are available to a scope via the unity::scopes::ScopeBase::settings()
and unity::scopes::QueryBase::settings() methods.
These methods return a unity::scopes::VariantMap with one entry per setting. The map contains an entry for
each setting (using the group name as the key). The lookup value is a unity::scopes::Variant that holds
the current value of the setting.

If a setting has a value, the corresponding entry in the map contains a string (for settings of type `string`,
a boolean (for settings of type `boolean`), or an integer (for settings of type `number` and `list`). (If the user
did not provide a particular value, but the settings definition provided a default value, the `Variant` contains the
default value.

If a setting does not have a default value, and the user did not establish a value for the setting, the corresponding
entry is absent from the map.

When you use settings in your scope implementation, do not cache the values and re-use them
for a different query. If you do, any setting changes made by the user will not take effect until
your scope is re-started by the runtime. (Because the user cannot know when that happens, this
can be highly confusing.) Instead, call `settings()` each time you need to use the value
of a setting. That way, your scope will react to any change made by the user as soon as it receives another
query.

Here is an example of how to read the current settings values for the definition in \ref settings_definitions :

~~~
// In your `ScopeBase` or `QueryBase` implementation:

unity::scopes::VariantMap s = settings();  // The settings method is provided by the base class

cout << s["location"].get_string();        // Prints "London" unless the user changed the value

auto it = s.find("color");
if (it != s.end())                         // Setting does not have a default value, need to test
{
    cout << it->second.get_string();       // Prints the user-established value
}
~~~

\subsection directories File system access

Scopes that are installed from click packages are subject to confinement and are not allowed
to access most parts of the file system. However, a few locations are available to a scope.
You can access these paths by calling methods on \link unity::scopes::ScopeBase ScopeBase\endlink.

\note Do not call these methods from the constructor of your `ScopeBase` implementation. If you
do, these methods throw `LogicException`. Instead, call them from
\link unity::scopes::ScopeBase::start() start()\endlink or any time thereafter.

\link unity::scopes::ScopeBase::scope_directory() scope_directory()\endlink returns the path of the installation
directory of the scope. This directory contains the scope's `.so` and `.ini` files, plus whatever other
files you decide to package with your scope. The scope has read-only permission for this directory.

\link unity::scopes::ScopeBase::cache_directory() cache_directory()\endlink returns the path of a
directory that is (exclusively) writable for the scope. You can use this directory to store
persistent information, such as a cache of results.

\link unity::scopes::ScopeBase::app_directory() app_directory()\endlink returns the path of a
read-only directory. If the scope is packaged together with an app, the app has permission to write
files in this location, that is, this directory can be used make information provided by the app
available to the scope (but not vice versa).

\link unity::scopes::ScopeBase::tmp_directory() tmp_directory()\endlink returns the path of a
read-only directory that is (exclusively) writable for the scope. This directory is periodically
cleaned of unused files. The exact amount of time may vary, but is on the order of a few hours.
The directory is also cleaned during reboot.

\subsection online_accounts Online Accounts

A scope may require access to an online account in order to evaluate particular results, perform certain actions, or
perhaps even operate at all. The following section describes how to use online account services from your scope.

\subsubsection oa_step1 Step 1: Update Apparmor manifest.

Firstly, in order for your scope to be granted access to the online accounts backend, the "accounts" policy group needs
to be added to your Apparmor manifest file, as follows:

\paragraph oa_apparmor Example Apparmor manifest file:
\code
{
    "template": "ubuntu-scope-network",
    "policy_groups": [
        "accounts"
    ],
    "policy_version": 1.2
}
\endcode

\subsubsection oa_step2 Step 2: Account service configuration.

Scopes access accounts at a service level (E.g. YouTube service under a Google account, Ubuntu Store service under an
Ubuntu One account, etc.), therefore each scope must provide some config to specify its account service requirements.

There are 2 additional files that a scope must supply:
1. A .service file to specify a method of accessing its particular account provider.
2. A .application file to link one or more services to your scope.

\paragraph oa_service Example .service file:
\code
<?xml version="1.0" encoding="UTF-8"?>
<service id="com.ubuntu.scopes.youtube_youtube">
  <type>sharing</type>
  <icon>youtube</icon>
  <name>YouTube</name>
  <provider>google</provider>
  <translations>unity-scope-youtube</translations>
  <template>
    <group name="auth">
      <setting name="method">oauth2</setting>
      <setting name="mechanism">web_server</setting>
      <group name="oauth2">
        <group name="web_server">
          <setting name="Host">accounts.google.com</setting>
          <setting name="AuthPath">o/oauth2/auth?access_type=offline</setting>
          <setting name="TokenPath">o/oauth2/token</setting>
          <setting name="RedirectUri">https://wiki.ubuntu.com/</setting>
          <setting name="ResponseType">code</setting>
          <setting type="as" name="Scope">['https://www.googleapis.com/auth/youtube.readonly']</setting>
          <setting name="ClientId">xxxx</setting>
          <setting name="ClientSecret">xxxx</setting>
          <setting type="as" name="AllowedSchemes">['https','http']</setting>
        </group>
      </group>
    </group>
  </template>
</service>
\endcode

\paragraph oa_application Example .application file:
\code
<?xml version="1.0" encoding="UTF-8"?>
<application id="com.ubuntu.scopes.youtube_youtube">
  <description>YouTube</description>
  <desktop-entry>com.ubuntu.scopes.youtube_youtube.desktop</desktop-entry>
  <services>
    <service id="com.ubuntu.scopes.youtube_youtube">
      <description>Watch your favorite YouTube videos</description>
    </service>
  </services>
</application>
\endcode

\subsubsection oa_step3 Step 3: Update Click manifest.

Now that we have added the new files from the previous step to our project, we need to update our click manifest file to include
them:

\paragraph oa_click Example Click manifest file:
\code
{
    "description": "YouTube scope",
    "framework": "ubuntu-sdk-14.10-dev2",
    "architecture": "armhf",
    "hooks": {
        "youtube": {
            "scope": "youtube",
            "apparmor": "apparmor.json",
            "account-application": "youtube.application",
            "account-service": "youtube.service"
        }
    },
    "icon": "youtube/icon.png",
    "maintainer": "Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>",
    "name": "com.ubuntu.scopes.youtube",
    "title": "YouTube scope",
    "version": "1.0.12"
}
\endcode

\subsubsection oa_step4 Step 4: Utilize the OnlineAccountClient class.

Finally, we can access online account services from within our scope implementation.

The first thing we need to do is instantiate a unity::scopes::OnlineAccountClient object. On construction we must
specify our account service name, service type, and provider name (These correspond to the values of the "service_id",
"type", and "provider" entries in your .service file).

The constructor accepts a fourth optional parameter which can be used to specify a dictionary of authentication data,
whose contents will complement (or override) those authentication parameters specified in the `<template>` element of
the \link oa_service `.service` file\endlink. It can be used in those rare cases where the authentication parameters are
known only at runtime.

Via this object we can get the statuses of all account services, set a callback for status updates, and register
results and widgets that require authorization (See: unity::scopes::OnlineAccountClient class documentation for more
detail).

Here's a simple example of how one could return a "Log-in" result to the dash (Selecting this result from the dash will
trigger an authorization request to the user before executing one of the 2 post-login actions):

\paragraph oa_usage Example OnlineAccountClient usage:
\code
void Query::run(unity::scopes::SearchReplyProxy const& reply)
{
    // Instantiate a unity::scopes::OnlineAccountClient object
    unity::scopes::OnlineAccountClient oa_client("com.ubuntu.scopes.youtube_youtube", "sharing", "google");

    // Check if our service is authenticated under at least one account
    bool service_authenticated = false;
    for (auto const& status : oa_client.get_service_statuses())
    {
        if (status.service_authenticated)
        {
            service_authenticated = true;
            break;
        }
    }

    // If our service is not authenticated, return a "Log-in" result
    if (!service_authenticated)
    {
        auto cat = reply->register_category("youtube_login", "", "");
        unity::scopes::CategorisedResult res(cat);
        res.set_title("Log-in to YouTube");

        oa_client.register_account_login_item(res,
                                              query(),
                                              unity::scopes::OnlineAccountClient::InvalidateResults,
                                              unity::scopes::OnlineAccountClient::DoNothing);
        reply->push(res);
    }
}
\endcode

\subsection scopetesting Testing

The Unity Scopes API provides testing helpers based on the well-known and established testing frameworks,
<a href="https://code.google.com/p/googletest/">googletest</a> and <a href="https://code.google.com/p/googlemock/">googlemock</a>.
Please see the respective documentation of these framework for general information on how to use them.

The testing helper classes are in the unity::scopes::testing namespace. The most important ones are:
<ul>
<li>unity::scopes::testing::TypedScopeFixture - A template class that takes your scope class name as a template argument
and creates a test fixture that can be used in tests.
<li>unity::scopes::testing::MockSearchReply - A mock of unity::scopes::SearchReply that makes it possible to intercept
responses to search request sent from the scope to a client, so you can test if your scope returns the expected data.
<li>unity::scopes::testing::MockPreviewReply - A mock of unity::scopes::PreviewReply that makes it possible to
intercept and test responses to preview request sent from the scope to a client.
<li>unity::scopes::testing::Result - A simple Result class derived from unity::scopes::Result that provides a default constructor,
so you can create dummy results (without attributes) for testing purposes.
<li>unity::scopes::testing::category - A simple class derived from unity::scopes::Category that makes it possible to create
dummy categories (which otherwise would require an instance of \link unity::scopes::SearchReply SearchReply\endlink and a call to
\link unity::scopes::SearchReply::register_category register_category()\endlink).
</ul>

Here is a test that checks if `MyScope` calls appropriate methods of unity::scopes::SearchReply. Note
that the test only checks that the correct methods are called and uses `_` matchers that match any value.
For a proper test, you will need to substitute values appropriate for your scope.

\code{.cpp}
typedef unity::scopes::testing::TypedScopeFixture<MyScope> TestScopeFixture;
using namespace ::testing;

TEST_F(TestScopeFixture, search_results)
{
    const unity::scopes::CategoryRenderer renderer;

    NiceMock<unity::scopes::testing::MockSearchReply> reply;
    EXPECT_CALL(reply, register_departments(_, _)).Times(1);
    EXPECT_CALL(reply, register_category(_, _, _, _))
            .Times(1)
            .WillOnce(
                Return(
                    unity::scopes::Category::SCPtr(new unity::scopes::testing::Category("id", "title", "icon", renderer))
                )
            );
    EXPECT_CALL(reply, push(Matcher<unity::scopes::Annotation const&>(_)))
            .Times(1)
            .WillOnce(Return(true));
    EXPECT_CALL(reply, push(Matcher<unity::scopes::CategorisedResult const&>(_)))
            .Times(1)
            .WillOnce(Return(true));

    // note: this is a std::shared_ptr with empty deleter
    unity::scopes::SearchReplyProxy reply_proxy(&reply, [](unity::scopes::SearchReplyBase*) {});

    unity::scopes::CannedQuery query(scope_id, "", "");
    unity::scopes::SearchMetadata meta_data("en_EN", "phone");

    auto search_query = scope->search(query, meta_data);
    ASSERT_NE(nullptr, search_query);
    search_query->run(reply_proxy);
}
\endcode

\subsection deployment Deployment

Installing a scope is as simple as running `make install` when using the scope
template. You might need to restart the global scope registry when a new scope
is installed by running:

\verbatim
restart scope-registry
\endverbatim

Scopes are installed under one of the "scopes directories"
scanned by the scope registry.  Currently these default to:

<ul>
<li>/usr/lib/${arch}/unity-scopes</li>
<li>/custom/lib/${arch}/unity-scopes</li>
<li>$HOME/.local/share/unity-scopes</li>
</ul>

The `/usr/lib` directory is for scopes that are pre-installed by Canonical.
The `/custom/lib` directory is for scopes that pre-installed by OEMs.
The `$HOME/.local` directory is for scopes that are installed from click packages.

Individual scopes are installed into subdirectories of these
installation directories. The name of the subdirectory containing
a scope's `.ini` and `.so` files can be anything but, to avoid name clashes,
we strongly suggest something that is unique, such as `com.canonical.scopes.scopename`.
At a minimum, the directory structure must contain the following:

    -+- ${scopesdir}
     `-+- subdirectory
       |--- scopename.ini
       `--- <library>.so

That is, each subdirectory must contain a scope `.ini` file and a shared library containing the
scope code. You are free to ship additional data in this
directory, such as a settings definition file (if your scope uses settings) or icon files
and screenshots.

The name of the scope's `.ini` file _must_ be a unique ID for the scope. We _strongly_
suggest to use a unique identifier, such as `com.canonical.scopes.scopename`, to avoid
clashes with scopes created by other developers.

The name of of the scope's `.so` file can be `libscopename.so`, `scopename.so`, or simply `scope.so`. For example,
for a scope named `Fred`, the names `libFred.so`, `Fred.so`, and `scope.so` are acceptable. (No other library
names are valid.)

The scope `.ini` file uses the standard `.ini` file format, with the
following keys:

    [ScopeConfig]
    DisplayName = human readable name of scope
    Description = description of scope
    Author = Author
    Version = 1
    Icon = path to icon representing the scope
    Art = path to screenshot of the scope
    SearchHint = hint text displayed to user when viewing scope
    HotKey =
    ResultsTtlType = None, Small, Medium, or Large
    Keywords =
    IsAggregator = true or false
    IdleTimeout = idle timeout in seconds
    LocationDataNeeded = true or false
    ScopeRunner = path_to_scope_runner args... %R %S

    [Appearance]
    ForegroundColor = default text color (defaults to theme-provided foreground color)
    BackgroundColor = color of scope background (default is transparent)
    ShapeImages = whether to use Ubuntu-shape for all cards and artwork (defaults to true)
    PreviewButtonColor = color of preview buttons (defaults to theme-provided color)
    LogoOverlayColor = color for the overlay in scopes overview (defaults to semi-transparent black)
    PageHeader.Logo = image containing scope's logo
    PageHeader.ForegroundColor = default header text color (defaults to the overall foreground color)
    PageHeader.Background = background scheme of the header
    PageHeader.DividerColor = color of the header divider
    PageHeader.NavigationBackground = background scheme of the navigation bar

The `ScopeConfig` group is mandatory. The information in this group makes the scope known to the registry.
In addition, this information controls how the scope appears in the "Scopes" scope.

The `ScopeConfig` group must contain settings for at least `DisplayName`, `Description`, and `Author`.
`DisplayName` and `Description` can (and should) be localized. For example:

`Description[de_DE] = Fu&szlig;ballergebnisse`

The `Version` key is optional, but we strongly recommend that you set it.
If the behavior of your scope changes in any way that is visible to the query source
(such having added or removed a result attribute), you should increment the version number.
This allows an aggregating scope to adjust its behavior according to which version of
your scope is installed.
If not set, the default value is 0. You can set the value to any integer >= 1.

The `SearchHint` key provides text that may be shown by the UI, such as "Enter a city name".

The `Keywords` key is optional, but we recommend that you use it. Keywords are used by aggregators
to collect results from scopes of similar type (E.g. The Music scope will aggregate scopes with the
keyword "music", and so on). The value of `Keywords` should specify a list of keywords your scope
falls under. This value must be a semicolon separated list (E.g. `Keywords = music;video`).

The `IsAggregator` key must be set to `true` for aggregating scopes. The default value is `false`.

The `IdleTimeout` key controls how long a scope can remain idle before it is told to stop by the registry (or
killed if it does not stop within 4 seconds). The default idle timeout is 40 seconds, meaning that a
scope will be told to stop if no query was sent to it for that amount of time. Only values in the range 1 to
300 seconds are accepted.

`ResultTtl` determines how long results should be cached by the UI before they are considered "stale"
and should be refreshed. `None` indicates that results remain valid indefinitely; `Small` indicates
results are valid for around a minute; `Medium` indicates that results are valid for a few minutes;
`Large` indicates that results remain valid for around an hour.

`LocationDataNeeded` should be set to `true` if the scope requires location data. In that case, the
\link unity::scopes::SearchMetadata SearchMetadata\endlink provides access to
\link unity::scopes::Location Location\endlink information
(assuming the user has granted location permission to the scope). If not set, the default value is `false`.

The `Scoperunner` key defines a command line to be executed when the scope is started by the registry.
Typically, scopes do not need to change this setting. It is provided mainly to allow scopes implemented
in languages other than C++ to be started, as well as for debugging.
For example, the following setting causes a scope to be run under `gdb`:

`ScopeRunner = /usr/bin/gdb --ex run --args /usr/lib/x86_64-linux-gnu/unity-scopes/scoperunner %R %S`

The `%R` expands to the path to the `Runtime.ini` config file, and `%S` expands to the scope's `.ini` file.

The `Appearance` group and all keys within it are optional and can be used to customize the look of the scope.
Some of the `Appearance` keys (such as `PageHeader.Background`) require background scheme URIs.
Valid URIs for these keys include:

<ul>
<li>color:///\#aarrggbb</li>
<li>color:///black</li>
<li>gradient:///\#aarrggbb/\#aarrggbb</li>
<li>/absolute/path/to/image</li>
<li>http://remote-server.com/path/to/image</li>
</ul>

\note Please refer to the <a href="https://developer.ubuntu.com/en/scopes/tutorials/scope-keywords/">Scope Keywords</a>
tutorial document for more detail on using keywords in your scope.

\subsection scopetool The scope tool

The Unity Scope Tool is a stand-alone rendering tool that allows you
to see how the dash will render your scope.

You can install the tool from the Ubuntu archive using:
\verbatim
sudo apt-get install unity-scope-tool
\endverbatim

After installation, you can run the scope-tool with a parameter specifying the
path to your scope configuration file (for example `unity-scope-tool ~/dev/myscope/build/myscope.ini`).
If a binary for your scope can be found in the same
directory, the scope-tool
displays surfacing and search results provided by your scope, and it allows you to
perform searches, invoke previews, and perform actions within previews.

Note that the scope-tool uses the same rendering mechanism as Unity
itself and, therefore, what you see in the scope-tool is what you get in Unity.
The tool can also be used to fine-tune category definitions, as it allows you
to manipulate the definitions on the fly. Once you are satisfied with the result,
you can just copy the JSON definition back into your scope
(see unity::scopes::CategoryRenderer::CategoryRenderer()).

The scope-tool supports a few command line arguments:
\arg By default (without any arguments) it will communicate with all scopes
installed on the system and available on the smart scopes server.
\arg When a path to a scope configuration file is provided, only that scope is
initialized, but you can either pass multiple configuration files or
the `--include-system-scopes` / `--include-server-scopes` option to allow
development of aggregating scopes.

*/