---
# System prepended metadata

title: OSCAL Engagement Metrics Brainstorming

---

# OSCAL Engagement Metrics Brainstorming

## Key themes and goals

Most important for any metric we consider is why a metric is important to us, and what question does it answer for us.

- For OSCAL today and the NIST team, engagement means XYZ.
- For the NIST OSCAL Team, the intended audience for our engagement is XYZ.
  - Developers?
  - Maintaining and growing bought-in developers?
  - Bringing in new developers

## Wendell's observations and recommendations

Three categories of data, are they measurable?

- OSCAL data flows (esp in comparison to pre-OSCAL status quo)
  - FedRAMP data flows
  - Other agency or organization data flows?
    Testimonial info?

- OSCAL Schema promulgation
  - How well can we tell this from Github?
  - Page analytics?
  - OSCAL forks

- Interest in OSCAL
  - Web stats
  - People showing up
  - People asking questions
  - Promotion of products and initiatives by third parties
  
## Chris thoughts

- Attendance at different public meetings
- Community opened issues
- Community opened pull requests
- Google/search engine results about non-NIST information about OSCAL (our project, not the calcium supplement)
- Count of non-public calls
- Counts of content downloads (schemas, official catalogs)
- Metrics around the tools, for example `oscal-cli` from the OSS Maven repository
- Survey audience (in forms for conf registration) their experience/knowledge of OSCAL

## AJ thoughts

- Google Analytics (via DAP) for all OSCAL Team products on pages.nist.gov
- Success stories, are they even a metric?
- Social media posts (LinkedIn/Twitter/etc) by Michaela and the NIST Team? In the official LinkedIn OSCAL group

## Miscellaneous pre-reqs conditions

- If our means and mechanisms cannot generate metrics, we have a problem (GitHub downloads are not trackable)
- Whatever we do, we should commit to consistency for at least a year, and it is simple to do that for one year
  - Revisit metrics the following year for the next iteration of the strategic plan the following year

# Decided are these things to start collection, preparation and reporting:

## Measures

- Google Analytics top five pages by rank as an aggregate monthly count, including month, count, page/URL.
- GitHub Issues opened as an aggregate monthly count, including project, count, month, excluding NIST staff.
- GitHub Pull Requests opened as an aggregate monthly count, including project, count, month, excluding NIST staff.
- Attendance at public meetings as an aggregate MAX count, date and title.

## Methods

- A person will need to collect a count at each public call.  Absent a location, it can be emailed on oscal-team@nist.gov.
- Collected for the previous month, during the first week of the following month.
- To be determined?

## Long-term risks

- More specific, advanced GitHub data points require custom tool development with APIs, do we have bandwidth and commitment?
- Using GitHub for key artifact inhibits analytics (Markdown files, code, schema files) or analysis entirely, how do we mitigate this risk long-term?