Events

This page is a list of events that Endless OS currently records for users that have not opted out of metrics recording, as well as events recorded by previous versions of Endless OS which are not recorded by current versions.

Stored Events

class LaunchedEquivalentExistingFlatpak(payload, **kwargs)

Double-click on Windows exe or Linux package with a similar installed app.

We launch the native version of a similar application since it is already installed. We record shell command line along with any arguments as well as the app ID of the replacement application. The purpose of this metric is to determine whether transparently launching a similar application reduces bounce rate.

Table name

launched_equivalent_existing_flatpak_v3

Payload type

(sas)

UUID

00d7bc1e-ec93-4c53-ae78-a6b40450be4a

UUID name

EVENT_LAUNCHED_EQUIVALENT_EXISTING_FLATPAK in eos-gates

New in version 3.2.4.

replacement_app_id

replacement application ID

argv

argv of the executable (Windows app or Linux package) the user tried to launch

class LaunchedEquivalentInstallerForFlatpak(payload, **kwargs)

Double-click on Windows exe or Linux package with a similar but not installed app in Store.

We record shell command line along with any arguments as well as the app ID of the replacement application. The purpose of this metric is to determine whether guiding the user to the app store to install a similar application reduces bounce rate.

Table name

launched_equivalent_installer_for_flatpak_v3

Payload type

(sas)

UUID

7de69d43-5f6b-4bef-b5f3-a21295b79185

UUID name

EVENT_LAUNCHED_EQUIVALENT_INSTALLER_FOR_FLATPAK in eos-gates

New in version 3.2.4.

replacement_app_id

replacement application ID

argv

argv of the executable (Windows app or Linux package) the user tried to launch

class LaunchedExistingFlatpak(payload, **kwargs)

Double-click on Windows exe or Linux package with a native installed app.

We launch the native version of the application since it is already installed. We record shell command line along with any arguments as well as the app ID of the replacement application. The purpose of this metric is to determine whether transparently launching the same application reduces bounce rate.

Table name

launched_existing_flatpak_v3

Payload type

(sas)

UUID

192f39dd-79b3-4497-99fa-9d8aea28760c

UUID name

EVENT_LAUNCHED_EXISTING_FLATPAK in eos-gates

New in version 3.2.4.

replacement_app_id

replacement application ID

argv

argv of the executable (Windows app or Linux package) the user tried to launch

class LaunchedInstallerForFlatpak(payload, **kwargs)

Double-click on Windows exe or Linux package with the same but not installed app in Store.

We record shell command line along with any arguments as well as the app ID of the replacement application. The purpose of this metric is to determine whether guiding the user to the app store to install the same application reduces bounce rate.

Table name

launched_installer_for_flatpak_v3

Payload type

(sas)

UUID

e98bf6d9-8511-44f9-a1bd-a1d0518934b9

UUID name

EVENT_LAUNCHED_INSTALLER_FOR_FLATPAK in eos-gates

New in version 3.2.4.

replacement_app_id

replacement application ID

argv

argv of the executable (Windows app or Linux package) the user tried to launch

class LinuxPackageOpened(payload, **kwargs)

A user tries to open a .rpm or .deb file.

Table name

linux_package_opened_v3

Payload type

as

UUID

0bba3340-52e3-41a2-854f-e6ed36621379

UUID name

LINUX_PACKAGE_OPENED in eos-gates

New in version 2.1.7.

argv

argv of the launched Windows application

class ParentalControlsBlockedFlatpakInstall(payload, **kwargs)

An app prevented from being installed due to parental controls restrictions.

This can happen if using the flatpak CLI, or if a UI app fails to hide a restricted app from its interface.

See https://phabricator.endlessm.com/T28741#810046.

Table name

parental_controls_blocked_flatpak_install_v3

Payload type

s

UUID

9d03daad-f1ed-41a8-bc5a-6b532c075832

UUID name

FLATPAK_PARENTAL_CONTROLS_INSTALL_EVENT in flatpak

New in version 3.8.0.

app

flatpak reference for the app which as blocked from being installed (e.g. app/org.gnome.Totem/x86_64/stable)

class ParentalControlsBlockedFlatpakRun(payload, **kwargs)

An app prevented from being run due to parental controls restrictions.

This can happen if using the flatpak CLI, or if a UI app fails to hide a restricted app from its interface.

See https://phabricator.endlessm.com/T28741#810046.

Table name

parental_controls_blocked_flatpak_run_v3

Payload type

s

UUID

afca2515-e9ce-43aa-b355-7663c770b4b6

UUID name

FLATPAK_PARENTAL_CONTROLS_INSTALL_EVENT in flatpak

New in version 3.8.0.

app

flatpak reference for the app which as blocked from being run (e.g. app/org.gnome.Totem/x86_64/stable)

class ParentalControlsChanged(payload, **kwargs)

Parental control app is closed.

Recorded whenever malcontent-control is closed, presumably after the user has edited one or more parental controls.

The payload contains the current parental controls settings for one user on the system, including identification about whether the user is an admin. Other user details (such as username or full name) are not included. The event is submitted multiple times, once for each (non-system) user on the system.

The same event is also recorded at the end of Initial Setup if (and only if) parental controls were enabled for the main user during Initial Setup. In that case, the main user account has been set up as a child user, with no administrator privileges, and with parental controls enabled. A second user has been created as the administrator. The event contains the values of the initial parental controls settings, but no identifying information about the user (such as their username or full name). The intention is to allow comparisons of which parental controls are enabled initially by users, and which are enabled long term.

See T28741 and #101.

The fields in the payload have the same semantics as the properties in the AppFilter <https://gitlab.freedesktop.org/pwithnall/malcontent/-/blob/master/ accounts-service/com.endlessm.ParentalControls.AppFilter.xml> interface.

Table name

parental_controls_changed_v3

Payload type

a{sv}

UUID

449ec188-cb7b-45d3-a0ed-291d943b9aa6

UUID name

MCT_PARENTAL_CONTROLS_EVENT in malcontent and gnome-initial-setup

New in version 3.8.0.

app_filter_is_whitelist

boolean indicating whether the following app filter is a whitelist (true) or blacklist (false)

app_filter

list of strings containing filtered apps (either flatpak refs, absolute paths, or content types)

oars_filter

string giving the filter schema (oars-1.0 or oars-1.1 at the moment), followed by a dictionary mapping OARS category strings to filter levels (none, mild, moderate, intense)

allow_user_installation

boolean indicating whether installation of software to the user flatpak repository is allowed

allow_system_installation

boolean indicating whether installation of software to the system flatpak repository is allowed

is_administrator

boolean indicating whether the user is an administrator (optional, defaults to false)

is_initial_setup

boolean indicating whether this event is being submitted from initial setup or from an installed desktop (optional, defaults to false)

class ParentalControlsEnabled(payload, **kwargs)

Any parental controls are enabled for the current user.

Recorded every time the App Center checks for OS updates (using its eos-updater plugin). This was chosen as a convenient regular event which happens inside the user session, rather than because of some deeper link to parental controls.

No identifying details about the user or which parental controls are enabled. It’s intended to allow aggregate statistics about how widely parental controls are enabled (in any form).

See T28741.

Table name

parental_controls_enabled_v3

Payload type

b

UUID

c227a817-808c-4fcb-b797-21002d17b69a

UUID name

GS_PARENTAL_CONTROLS_USAGE_EVENT in gnome-software

New in version 3.8.0.

enabled

whether parental controls are enabled for this user

class ProgramDumpedCore(payload, **kwargs)

A program crashes and systemd-coredump catches.

We include the name of the program that crashed and the ostree commits of ostree repos on the system. We do not include programs that crashed within /home or /sysroot/home.

See T18444.

Table name

program_dumped_core_v3

Payload type

a{sv}

UUID

ed57b607-4a56-47f1-b1e4-5dc3e74335ec

UUID name

PROGRAM_DUMPED_CORE_EVENT in eos-metrics-instrumentation

New in version 3.2.5.

info

Dictionary of strign-to-variant containing the following keys:

  • binary: the path to the binary on the system

  • signal: what signal caused the program to crash

  • timestamp: the timestamp the kernel reported the crash at

  • ostree_commit: the hash of the OSTree commit

  • ostree_url: the URL of the OSTree repository

  • app_ref: optionally, the full Flatpak app ref, e.g. app/net.sourceforge.ExtremeTuxRacer/x86_64/stable

  • app_commit: optionally, the hash of the OSTree commit for the crashed Flatpak app

  • app_url: optionally, the URL of the Flatpak repository for the crashed app

  • runtime_ref: optionally, the full Flatpak runtime ref used by the crashed app

  • runtime_commit: optionally, the hash of the OSTree commit for the Flatpak runtime used by the crashed app

  • runtime_url: optionally, the URL of the Flatpak repository for the runtime used by the crashed app

class UpdaterFailure(payload, **kwargs)

Failure of eos-updater or eos-updater-flatpak-installer for whatever reason.

This can happen if an upgrade fails, or if installing required flatpaks fails.

See T29247.

Table name

updater_failure_v3

Payload type

(ss)

UUID

927d0f61-4890-4912-a513-b2cb0205908f

UUID name

EOS_UPDATER_METRIC_FAILURE in eos-updater

New in version 2.6.0.

class WindowsAppOpened(payload, **kwargs)

A user tries to open a .exe or .msi file.

Table name

windows_app_opened_v3

Payload type

as

UUID

cf09194a-3090-4782-ab03-87b2f1515aed

UUID name

WINDOWS_APP_OPENED in eos-gates

New in version 2.1.5.

argv

argv of the launched Windows application

class StartupFinished(payload, **kwargs)

Computer startup finishes.

We send this event when startup finishes with a breakdown of how long was spent in each of several different phases of startup.

The value comes directly from systemd’s StartupFinished signal.

Table name

startup_finished_v3

Payload type

(tttttt)

UUID

bf7e8aed-2932-455c-a28e-d407cfd5aaba

UUID name

STARTUP_FINISHED in eos-metrics-instrumentation

New in version 2.1.2.

firmware

time spent in the firmware (if known) in µsec

loader

time spent in the boot loader (if known) in µsec

kernel

time spent in the kernel initialization phase in µsec

initrd

time spent in the initrd (if known) in µsec

userspace

time spent in userspace in µsec

total

total time spent to boot in µsec

class ComputerInformation(payload, **kwargs)

Information about the computer RAM, disk and CPU.

Sent at most once a day.

Table name

computer_information_v3

Payload type

(uuuua(sqd))

UUID

81f303aa-448d-443d-97f9-8d8a9169321c

UUID name

COMPUTER_INFORMATION

New in version 4.0.0.

total_ram

total RAM size in mebibytes (2^20)

total_disk

total disk space in gibibytes (2^30)

used_disk

used disk space in gibibytes (2^30)

free_disk

free disk space in gibibytes (2^30)

info

array of CPU model (e.g. Intel(R) Core(TM) i7-5500U CPU @ 2.40GHz), number of cores (e.g. 4) and maximum CPU speed in MHz or current CPU speed if maximum can’t be determined (e.g. 3000.0)

class SplitFlatpakRepoStats(payload, **kwargs)

Statistics for splitting the Flatpak repo from the OSTree repo.

Prior to Endless OS 4, both the OS and Flatpaks were stored in a single, shared OSTree repository. On the first boot into Endless OS 4 or later, these are split into two repositories, one for the OS and one for Flatpaks. This can take a long time, so we report this event to quantify how long it takes and why.

This is a one-time operation which only runs if needed, so this event is sent at most once per Endless OS system.

Table name

split_flatpak_repo_stats_v3

Payload type

(uuuuuuutut)

UUID

880fd091-2e68-4bd2-baa1-f6810fabcc1c

UUID name

SPLIT_FLATPAK_REPO_STATS in eos-boot-helper

New in version 4.0.1.

elapsed

time elapsed in seconds

num_os_refs

number of OS OSTree refs duplicated

num_flatpak_refs

number of Flatpak OSTree refs duplicated

num_other_refs

number of other OSTree refs duplicated

num_objects

number of OSTree objects duplicated

num_deltas

number of OSTree static deltas duplicated

num_apps

number of Flatpak apps duplicated

size_apps

size of Flatpak apps duplicated in bytes

num_runtimes

number of Flatpak runtimes duplicated

size_runtimes

size of Flatpak runtimes duplicated in bytes

class DailyAppUsage(payload, **kwargs)

Time that an app has an open window.

Aggregation is done by day.

Table name

daily_app_usage_v3

Payload type

s

UUID

49d0451a-f706-4f50-81d2-70cc0ec923a4

UUID name

DAILY_APP_USAGE

New in version 4.0.0.

app_id

application ID

class MonthlyAppUsage(payload, **kwargs)

Time that an app has an open window.

Aggregation is done by month.

Table name

monthly_app_usage_v3

Payload type

s

UUID

22cdaa7c-e125-5f65-a535-c8345e5434f2

UUID name

MONTHLY_APP_USAGE

New in version 4.0.0.

app_id

application ID

class DailyUsers(payload, **kwargs)

Number of different users who opened a session.

Aggregation is done by day.

Table name

daily_users_v3

Payload type

NULL

UUID

a3826320-9192-446a-8886-e2129c0ce302

UUID name

DAILY_USERS

New in version 4.0.0.

class MonthlyUsers(payload, **kwargs)

Number of different users who opened a session.

Aggregation is done by month.

Table name

monthly_users_v3

Payload type

NULL

UUID

584f3525-4805-5d25-a0a3-5807f14b4d91

UUID name

MONTHLY_USERS

New in version 4.0.0.

class DailySessionTime(payload, **kwargs)

Number of seconds spent on the computer with an open session.

Aggregation is done by day.

Table name

daily_session_time_v3

Payload type

NULL

UUID

5dc0b53c-93f9-4df0-ad6f-bd25e9fe638f

UUID name

DAILY_SESSION_TIME

New in version 4.0.0.

class MonthlySessionTime(payload, **kwargs)

Number of seconds spent on the computer with an open session.

Aggregation is done by month.

Table name

monthly_session_time_v3

Payload type

NULL

UUID

2ee16e97-2993-5dcc-b9d2-841152520ad6

UUID name

MONTHLY_SESSION_TIME

New in version 4.0.0.

Deprecated Events

System Shutdown

Note

This event has been deprecated in 3.7.0 and is not sent any more. See the uptime event which replaced it. See T27963.

We records shutdowns as well as the total length of time the computer has been powered on across all boots. Since 2.5.0, we also include the total number of boots the computer has been through.

  • UUID from 2.2.0: SHUTDOWN − ae391c82-1937-4ae5-8539-8d1aceed037d − eos-metrics-instrumentation

  • UUID from 2.5.0: SHUTDOWN_EVENT − 91de63ea-c7b7-412c-93f3-6f3d9b2f864c − eos-metrics-instrumentation

  • UUID from 2.5.2: SHUTDOWN_EVENT − 8f70276e-3f78-45b2-99f8-94db231d42dd − eos-metrics-instrumentation

  • Payload: type (xx)

    • total uptime across all boots

    • number of boots the computer has been through

Note

A serious bug that often prevented this event from being recorded was fixed in the 2.3.0 release. Another serious bug that often prevented this event from being recorded was introduced in 2.4.0 and fixed in 2.5.1. A serious bug that often prevented the boot count from being incremented was fixed in the 2.5.2 release.

Companion App - Device Authenticate

Reported when a user authenticates a device with the Companion App Service

  • UUID from 3.4: 6dad6c44-f52f-4bca-8b4c-dc203f175b97 − eos-companion-app-integration

  • Payload: type a{ss}

    • A dictionary of string keys to string values:

      • deviceUUID: hash of unique device identifier

See T21316.

Companion App - List Applications

Note

The companion app is no more.

Reported when a user gets an application listing from the Companion App Service

  • UUID from 3.4: 337fa66d-5163-46ae-ab20-dc605b5d7307 − eos-companion-app-integration

  • Payload: type a{ss}

    • a dictionary of string keys to string values:

      • deviceUUID: hash of unique device identifier

      • referrer: optional, name of view that the user came from, one of:

        • feed: the content feed

        • search_content: search

        • list_content_for_tags: listing of content for a category

        • list_applications: listing of available applications

        • list_application_sets: listing of categories for an application

        • device_authenticate: when a user first associates a device with a computer

        • refresh: when a user pulls down to refresh

        • retry: when the user manually retries after a timeout

        • back: when the user goes back

        • content: following a link from within content

See T21316.

Companion App - List Sets for Applications

Note

The companion app is no more.

Reported when a user gets a listing of application sets from the Companion App Service.

  • UUID from 3.4: c02a5764-7f81-48c7-aea4-1413fd4e829c

  • Payload: type a{ss}

    • A dictionary of string keys to string values:

      • deviceUUID: hash of unique device identifier

      • applicationId: application ID

      • referrer: see “Companion App - List applications”

See T21316.

Companion App - List Content for Tags of Application

Note

The companion app is no more.

Reported when a user gets a listing of application content for a set of tags from the Companion App Service.

  • UUID from 3.4: bef3d12c-df9b-43cd-a67c-31abc5361f03

  • Payload: type a{ss}

    • A dictionary of string keys to string values:

      • deviceUUID: hash of unique device identifier

      • applicationId: application ID

      • tags: semicolon delimited list of tags

      • referrer: see “Companion App - List applications”

See T21316.

Companion App - View Content

Note

The companion app is no more.

Reported when a user views some requested content EKN-ID on the Companion App.

  • UUID from 3.4: e6541049-9462-4db5-96df-1977f3051578

  • Payload: type a{ss}

    • A dictionary of string keys to string values:

      • deviceUUID: hash of unique device identifier

      • applicationId: application ID

      • contentTitle: content title

      • contentType: content MIME type

      • referrer: see “Companion App - List applications”

See T21316.

Companion App - Get Content Metadata

Note

The companion app is no more.

Reported when a user gets some metadata about some requested content EKN-ID on the Companion App.

  • UUID from 3.4: 3a4eff55-5d01-48c8-a827-fca5732fd767

  • Payload: type a{ss}

    • A dictionary of string keys to string values:

      • deviceUUID: hash of unique device identifier

      • applicationId: application ID

      • contentId: EKN ID

      • referrer: see “Companion App - List applications”

See T21316.

Companion App - Search for Content or Applications

Note

The companion app is no more.

Reported when a user uses the search functionality on the app to search for things.

  • UUID from 3.4: 9f06d0f7-677e-43ca-b732-ccbb40847a31

  • Payload: type a{ss}

    • A dictionary of string keys to string values:

      • deviceUUID: hash of unique device identifier

      • applicationId: application ID

      • tags: semicolon delimited list of tags

      • searchTerm: optional, search term

      • referrer: see “Companion App - List applications”

See T21316.

Companion App - Request Content Feed

Note

The companion app is no more.

Reported when the user opens the Companion App and requests the content feed.

  • UUID from 3.4: af3e89b2-8293-4703-809c-8e0231c128cb

  • Payload: type a{ss}

    • A dictionary of string keys to string values: - deviceUUID: hash of unique device identifier - mode: ‘ascending’ or ‘descending’ - referrer: see “Companion App - List applications”

See T22203.

Companion App - Download Bundled Application

Note

The companion app is no more.

  • UUID from TBC: 7be595662b23-408a-acf6-91490fc1df1c

  • Payload: type a{ss}

    • A dictionary of string keys to string values:

      • deviceUUID: hash of unique device identifier

      • referrer: see “Companion App - List applications”

See T22053.

Network Status Changed

Removed in 3.7.4. See T28301.

We record when the network status changes from one state to another. A common case of this is moving from an “unknown” state to a “connecting” to a “globally connected” state on startup.

See the comprehensive list of status codes.

  • UUID from 2.10: EMTR_EVENT_NETWORK_STATUS_CHANGED − 5fae6179-e108-4962-83be-c909259c0584 − eos-metrics

  • Payload: type (uu)

    • Previous network state

    • New network state

Note

Since https://github.com/endlessm/eos-shell/issues/2684 was fixed in 2.2.0, we no longer misrepresent local and site connectivity as global connectivity.

Social Bar Is Visible

Removed in 3.2.0.

We record when the social bar is made visible to the user and when it is no longer visible. Basically, it corresponds to the user clicking on the social bar icon.

  • UUID from 2.10: EMTR_EVENT_SOCIAL_BAR_IS_VISIBLE − 9c33a734-7ed8-4348-9e39-3c27f4dc2e62 - eos-social

  • Payload of start event: NULL

  • Payload of stop event: NULL

Desktop Searches

Removed in OS 4.0.0.

We record searches made from the desktop search bar.

  • Since: 2.1.2

  • UUID: b02266bc-b010-44b2-ae0f-8f116ffa50eb

  • UUID name: EVENT_DESKTOP_SEARCH in gnome-shell

  • Payload: type (us)

    • search provider

      • local: 0

      • Google: 1

    • query string (what the user searched for)

Note

Since the 2.1.6. release, Google searches have no longer been recorded.

Knowledge App – Article Open/Close

Not stored since Endless OS 4.0.0.

We record when an article is opened or closed in a knowledge app. We record the ID of the content, the entry point (whether the article was accessed via article link, desktop search, app link click, or a nav button), the app ID, the article title, and the content type.

  • Since: SDK 2

  • UUID: fae00ef3-aad7-44ca-aff2-16555e45f0d9

  • UUID name: CONTENT_ACCESS_METRIC_EVENT_ID in eos-knowledge-lib

  • Payload of start event: type (ssss)

    • Entry point

    • Application ID

    • Title

    • Content type

  • Payload of stop event: NULL

See T18516.

Hack Toolbox - Code View Error - Hack Episode 1

Not reported since Hack was moved to Flathub in Endless OS 3.9.1.

  • Since: 3.5.3

  • UUID: e98aa2b8-3f11-4a25-b8e9-b10a635df121

  • UUID name: CODEVIEW_ERROR_EVENT

  • Payload of start event: type sssa(suquq)

    • Application ID that this toolbox belongs to (string)

    • ID of function that was being edited in the toolbox (string) (currently always blank)

    • Contents of code view (string)

    • List of the error messages that are displayed, each containing:

    • Text of error message (string)

    • Start line number where error is shown, 1-based (32-bit unsigned)

    • Start column number where error is shown, 0-based (16-bit unsigned)

    • End line number where error is shown, 1-based (32-bit unsigned)

    • End column number where error is shown, 0-based (16-bit unsigned)

  • Payload of progress and stop events: type s

    • Diff of the contents of the code view to the state from the previous event, in the form of an ed script, chosen because it’s the shortest form that the diff utility can output

See T24429.

Hack Clubhouse - Quest

Not reported since Hack was moved to Flathub in Endless OS 3.9.1.

  • Since: 3.7.4

  • UUID: 50aebb1b-7a93-4caf-8698-3a601a0fc0f6

  • UUID name: QUEST_EVENT

  • Key: The quest name

  • Payload of start event: type (bsas)

    • True if the quest is completed

    • The quest ID

    • The list of pathway names of this quest

  • Payload of stop event: type (bsas)

    • True if the quest is completed

    • The quest ID

    • The list of pathway names of this quest

See T28004.

Uptime

Removed in 4.0.0.

Total length of time the computer has been powered on and total number of boots.

The difference with the system shutdown event is that this is sent periodically while the computer is up, not just at shutdown. This allows catching “dirty” shutdowns and makes it easier to estimate connectivity.

See https://github.com/endlessm/eos-metrics-instrumentation/commit/8dfd1e5b9.

UUID name

UPTIME_EVENT in eos-metrics-instrumentation

UUID was 005096c4-9444-48c6-844b-6cb693c15235 before 2.5.2.

Note

A serious bug that often prevented the boot count from being incremented was fixed in the 2.5.2 release.

New in version 2.5.0.

Shell App Is Open

Removed in 4.0.0.

An application opens and closes.

By subtracting the time of closing from time of opening, we can tell how long an application has been open. This basically includes all applications of interest to non-developers.

UUID name

SHELL_APP_IS_OPEN_EVENT in gnome-shell

New in version 2.1.0.

User Is Logged In

Removed in 4.0.0.

A user logs in and logs out to Endless OS.

As of 2.1.2 we also records which user logged in. This is still anonymous data, so we only record an arbitrary number (the user ID), but we can discover (among other things) how many different users use the computer that way.

UUID name

USER_IS_LOGGED_IN in eos-metrics-instrumentation (since 2.1.2)

UUID was called EMTR_EVENT_USER_IS_LOGGED_IN (ab839fd2-a927-456c-8c18-f1136722666b) before 2.1.2.

Note

A serious bug that often prevented this event from being recorded was introduced in 2.4.0 and fixed in 2.5.1.

New in version 2.1.0.

Network ID

Removed in 4.0.0.

A change in the default route happens after the network connectivity has changed.

The intention behind the payload is to provide a value which is opaque and stable which is the same for every system located on the same physical network (also visible from the eos-network-id command).

See T16934.

UUID name

NETWORK_ID_EVENT in eos-metrics-instrumentation

New in version 3.1.2.

Cache Is Corrupt

Removed in 4.0.0.

Cache has been found to be corrupt and was reset.

We’ve observed that in some situations the metrics recorder daemon’s cached data contains only a few valid items and then corrupt data, and that some other times the whole thing becomes corrupt and completely unusable, bringing down the whole metrics recorder daemon and effectively killing metrics reporting forever for that machine.

As it’s still unclear why that happens, we now detect those situations and correct them when they happen, so that the metrics system can still be used afterwards.

UUID name

CACHE_IS_CORRUPT_EVENT_ID in eos-event-recorder-daemon

New in version 3.0.9.

Cache Metadata Is Corrupt

Removed in 4.0.0.

Cache metadata is corrupt and was reset, any cached metrics were discarded.

We’ve observed that in some situations the metrics recorder daemon’s cached data contains only a few valid items and then corrupt data, and that some other times the whole thing becomes corrupt and completely unusable, bringing down the whole metrics recorder daemon and effectively killing metrics reporting forever for that machine.

As it’s still unclear why that happens, we now detect those situations and correct them when they happen, so that the metrics system can still be used afterwards.

See T19953.

UUID name

CACHE_METADATA_IS_CORRUPT_EVENT_ID in eos-event-recorder-daemon

New in version 3.3.5.

Cache Has Invalid Elements

Removed in 4.0.0.

Some invalid cache elements were found.

We’ve observed that in some situations the metrics recorder daemon’s cached data contains only a few valid items and then corrupt data, and that some other times the whole thing becomes corrupt and completely unusable, bringing down the whole metrics recorder daemon and effectively killing metrics reporting forever for that machine.

As it’s still unclear why that happens, we now detect those situations and correct them when they happen, so that the metrics system can still be used afterwards.

UUID name

CACHE_HAS_INVALID_ELEMENTS_EVENT_ID in eos-event-recorder-daemon

New in version 3.0.9.

Discovery Feed Clicked

Not reported since 3.9.0 when the Discovery Feed was removed from Endless OS.

Something is clicked on the Discovery Feed, including content which is not “clickable”.

The payload tells us about what users are clicking on generally and whether they are clicking on things that we don’t expect to be clicked. The intention is to allow the operator to determine what content should be made clickable and what kinds of content are being opened.

The content_type field that can be included in the payload is currently one of:

  • knowledge_content: text-based article in a knowledge-app

  • knowledge_video: video based article in a knowledge-app

  • knowledge_artwork: “artwork” card (larger images in a “Gallery” style format)

  • undefined: user clicked on a non-clickable area

UUID name

EVENT_DISCOVERY_FEED_CLICK in eos-discovery-feed

New in version 3.2.0.

Discovery Feed Closed

Not reported since 3.9.0 when the Discovery Feed was removed from Endless OS.

Something is clicked on the Discovery Feed, including content which is not “clickable”.

The payload tells us about what users are clicking on generally and whether they are clicking on things that we don’t expect to be clicked. The intention is to allow the operator to determine what content should be made clickable and what kinds of content are being opened.

The content_type field that can be included in the payload is currently one of:

  • knowledge_content: text-based article in a knowledge-app

  • knowledge_video: video based article in a knowledge-app

  • knowledge_artwork: “artwork” card (larger images in a “Gallery” style format)

  • undefined: user clicked on a non-clickable area

UUID name

EVENT_DISCOVERY_FEED_CLICK in eos-discovery-feed

New in version 3.2.0.

Discovery Feed Opened

Not reported since 3.9.0 when the Discovery Feed was removed from Endless OS.

The Discovery Feed is open.

The payload tells us about the language the user is using when interacting with the discovery feed and how they opened it. The intention is to allow the operator to determine which languages the discovery feed is most popular in and how users generally open the feed.

UUID name

EVENT_DISCOVERY_FEED_OPEN in eos-discovery-feed

New in version 3.2.0.

Disk Space Extra

Removed in 4.0.0, along with split-disk support in general.

Total, used and free disk space for /var/endless-extra.

Sent on startup, and every 24 hours.

See T18445.

UUID name

EXTRA_DISK_SPACE_EVENT in eos-metrics-instrumentation

New in version 3.4.3.

Entered Demo Mode

Not reported since demo mode was removed in Endless OS 3.9.0.

The systems enters demo mode.

Note that the machine ID will be reset just before the system enters demo mode, so any metrics collected for this machine ID after this event has been fired are metrics for a system that is in demo mode.

Demo mode was removed in EOS 3.9, so this metric will not be seen for systems on 3.9.0 or later.

See T18983.

UUID name

DEMO_MODE_ENTERED_METRIC in gnome-initial-setup

New in version 3.2.5.

Endless Application Unmaximized

Not stored since Endless OS 4.0.0.

An in-house application is unmaximized for the first time in each run.

We record a metric with the application ID.

UUID name

UNMAXIMIZE_EVENT in eos-sdk

New in version 3.0.0.

Hack Clubhouse Achievement

Not reported since Hack was moved to Flathub in Endless OS 3.9.1.

Achievement reached in the Hack Clubhouse.

See T28004.

UUID name

ACHIEVEMENT_EVENT

New in version 3.7.4.

Hack Clubhouse Achievement Points

Not reported since Hack was moved to Flathub in Endless OS 3.9.1.

Points earned in the Hack Clubhouse.

See T28004.

UUID name

ACHIEVEMENT_POINTS_EVENT

New in version 3.7.4.

Hack Clubhouse Change Page

Not reported since Hack was moved to Flathub in Endless OS 3.9.1.

Page changed in the Hack Clubhouse.

See T28004.

UUID name

CLUBHOUSE_SET_PAGE_EVENT

New in version 3.7.4.

Hack Clubhouse Enter Pathway

Not reported since Hack was moved to Flathub in Endless OS 3.9.1.

Pathway entered in the Hack Clubhouse.

See T28004.

UUID name

CLUBHOUSE_PATHWAY_ENTER_EVENT

New in version 3.7.4.

Hack Clubhouse Mode

Not reported since Hack was moved to Flathub in Endless OS 3.9.1.

Hack mode changed in the Hack Clubhouse.

See T28501.

UUID name

HACK_MODE_EVENT

New in version 3.7.4.

Hack Clubhouse Progress

Not reported since Hack was moved to Flathub in Endless OS 3.9.1.

Progress updated in the Hack Clubhouse.

See T28004.

UUID name

PROGRESS_UPDATE_EVENT

New in version 3.7.4.

Location

Removed in 4.0.0.

The user’s location at city-level granularity.

An event is sent once per boot and subsequently every time it changes by a distance of 15 km or more during the same boot. We include latitude, longitude, altitude if known, and accuracy of the location estimate.

Note

Since around 3.7.0, for privacy reasons this is only recorded on a small set of products − at the time of writing this is fnde, impact, payg, solutions, and spark. See T27743 and T27655 for more details.

UUID name

USER_LOCATION in eos-metrics-instrumentation

New in version 2.1.5.

Missing Codec

Removed in 4.0.0.

A GStreamer-based application tries to install a missing codec.

UUID name

EOS_CODECS_MANAGER_MISSING_CODEC in eos-codecs-manager

New in version 2.6.0.

Monitor Connected

Removed in 4.0.0.

A display is connected (e.g. computer monitor, television) to the machine.

We include any information about the display (e.g. with, height).

UUID name

MONITOR_CONNECTED in mutter

UUID was 566adb36-7701-4067-a971-a398312c2874 before 2.1.7.

New in version 2.1.4.

Monitor Disconnected

Removed in 4.0.0.

A display is disconnected (e.g. computer monitor, television) from the machine.

We include any information about the display (e.g. with, height).

UUID name

MONITOR_DISCONNECTED in mutter

UUID was ce179909-dacb-4b7e-83a5-690480bf21eb before 2.1.7.

New in version 2.1.4.

Shell App Added To Desktop

Removed in OS 3.9.0.

Shell application is added to desktop.

UUID name

SHELL_APP_ADDED_EVENT in gnome-shell

Note

Since 3.9.0 this metric is no longer recorded. See T30661.

New in version 2.1.0.

Shell App Removed From Desktop

Removed in 3.9.0.

Shell application is removed from desktop.

UUID name

SHELL_APP_REMOVED_EVENT in gnome-shell

Note

Since 3.9.0 this metric is no longer recorded. See T30661.

New in version 2.1.0.

Underscan Enabled

Removed in 3.9.0.

Underscan is enabled on a monitor.

UUID name

UNDERSCAN_ENABLED in mutter

New in version 3.8.7.

Windows License Tables

Removed in 4.0.0.

ACPI tables are present on the system, at startup.

The tables we check for are MSDM and SLIC, which hold OEM Windows license information on newer and older systems respectively.

We have not seen systems which have both tables, but they might exist in the wild and would appear with a value of 3. With this information, assuming Metrics Events is not sent, then we can distinguish:

  • SLIC/MSDM > 0 and no dual boot: Endless OS is the sole OS, PC came with Windows

  • SLIC/MSDM > 0 and dual boot: Endless OS installed alongside OEM Windows

  • SLIC/MSDM = 0 and no dual boot: Endless OS is the sole OS, PC came without Windows

  • SLIC/MSDM = 0 and dual boot: Dual-booting with a retail Windows

See T18296.

UUID name

WINDOWS_LICENSE_TABLES_EVENT in eos-metrics-instrumentation

New in version 3.2.0.

Updater Branch Selected

Removed in 4.0.0.

An eos-updater branch has been selected.

UUID name

EOS_UPDATER_METRIC_BRANCH_SELECTED in eos-updater

New in version 2.6.0.

Control Center Panel Opened

Removed in 4.0.0.

A panel is open in the control center.

UUID name

PANEL_OPENED_EVENT_ID in gnome-control-center

New in version 3.2.5.

Control Center Automatic Updates

Removed in 4.0.0.

Automatic updates settings have changed.

UUID name

CC_METRIC_AUTOMATIC_UPDATES in gnome-control-center

New in version 3.9.1.

Tour Response

When a user first logs into GNOME 40+ (which in the case of Endless OS means Endless OS 5), they are offered the chance to take a tour. This event had a boolean payload, true if the user chose to take the tour and false otherwise.

Combined with the image ID from the channel, this data can be separated into new installations, and users upgrading from Endless OS 4 and older.

Over a period of just under a year, we determined:

  • 15% of users upgrading from Endless OS 4 took the tour

  • 32% of users starting from Endless OS 5 took the tour

  • 19% of users overall took the tour

Having validated our belief that relatively few users would opt to choose the tour, and learned that upgrading users are less likely to take it (which is a shame since they are the ones whose desktop changed significantly when they upgrade to Endless OS 5), we removed this event in Endless OS 5.2.0 and discarded the data on the server.

UUID

140643be-fe47-4b4b-985b-d16f8f3973a9

UUID name

TOUR_RESPONSE_EVENT in gnome-shell

New in version 5.0.0.

Test Events

The following are only ever used in tests, and are thus discarded by the server upon reception.

We document them here to make sure they don’t get reused inadvertently by real events.

  • Smoke tests

    • 72fea371-15d1-401d-8a40-c47f379f64fd

    • 9a0cf836-12cd-4887-95d8-e48ccdf6e552

    • b1f87a3f-a464-48d4-8e35-35dd45659010

    • b2b17dfd-c30e-4789-abcc-4a38323127f6

    • b89f9a4a-3035-4fc3-9bef-584367fe2c96

    • fb59199e-5384-472e-af1e-00b7a419d5c2

  • Event recorder tests

    • 350ac4ff-3026-4c25-9e7e-e8103b4fd5d8

    • d936cd5c-08de-4d4e-8a87-8df1f4a33cba

  • Daemon tests

    • 350ac4ff-3026-4c25-9e7e-e8103b4fd5d8

Implementing New Events

Preliminary Considerations

Before adding new events, you have to discuss and justify the metric: what question are you trying to answer, is this the most appropriate metric for answering that question, and does it preserve users’ privacy? Are we legally allowed to collect and store it?

The second step is to define the submission (GVariant) and storage (SQL) formats for the new metric, generate a UUID with the uuidgen command, and document it on this page.

Finally, you can add code in the OS to submit the metrics at relevant points, as done in this example. At its core, this will be a call to emtr_event_recorder_get_default() and then a call to emtr_event_recorder_record_event(). You will probably need to add a dependency on eos-metrics-0-dev to the Debian packaging. The code to submit new metrics can be shipped before the server is ready to process them, as unknown metrics are stored and re-parsed when Azafea is updated.

Technical Introduction

The metrics event processor implements a few events. When it receives a request containing events it doesn’t know about, they will get stored in the following tables:

  • unknown_singular_event for singular metrics;

  • unknown_aggregate_event for aggregate metrics.

The records in those table should contain everything needed to replay them once the corresponding event model has been implemented.

Adding a New Model

New metrics models should be added in azafea/event_processors/endless/metrics/v3/events.py. Do look at the existing ones for examples.

All it takes is defining a new class inheriting from the appropriate base event:

  • SingularEvent;

  • AggregateEvent.

The file is organized in 2 sections, one for each event type. Please keep it tidy by adding your new model in the correct section. It’s also nice to order models alphabetically within each section.

The class needs at least a few special attributes:

  • __tablename__ is the name of the table in PostgreSQL; this will usually be the same as the class name, but using snake_case;

  • __event_uuid__ is the unique identifier for the event; it is how Azafea will know which model to use for any incoming event;

  • __payload_type__ is the format string of the GVariant payload; if the event has no payload, then use None.

If an event has a payload, then you will want that payload to be deserialized and stored into columns of the new table.

This is achieved by adding the right sqlalchemy.schema.Column-s to the model, and implementing the _get_fields_from_payload method. The latter is responsible for deserializing the GVariant payload. By the time the method is called, the payload has already been validated against the __payload_type__.

You can also look at this example of new models addition.

Creating the Tables

After adding a new model, stop Azafea and create the database migration:

[azafea-dev]$ pipenv run azafea -c config.toml make-migration queue-name

The queue-name is the configured name of a queue set up with the handler which contains the new model.

Carefully review the generated migration file, and commit it along with your new event.

You can test that the migration is functional by running it:

[azafea-dev]$ pipenv run azafea -c config.toml migratedb

You can also look at this example of database migration in Azafea.

Deploying

Deployment steps depend on the infrastructure you use for Azafea.

For Endless OS, Azafea changes are deployed to the dev server and tested by looking at https://metabase.dev.endlessm-sf.com/ while submitting the new metrics events from an Endless OS system. The EOS system should be configured with environment=dev in /etc/metrics/eos-metrics-permissions.conf.

If all looks good, deploy the changes to the Azafea production server.

Replaying Previously Unknown Events

If some instances of your new event had been received by Azafea before you implemented it and Azafea had stored them in the corresponding “unknown” table, you can run the migration to create the table, then replay them to “make them known”:

$ pipenv run azafea -c config.toml migratedb
$ pipenv run azafea -c config.toml metrics-3 replay-unknown

The above commands need to be run in an environment with access to Azafea itself. If you deployed Azafea in production using Docker, then you will want to run them in an instance of that production container.