# osquery office hours 2023-09-12

YouTube Link: https://youtu.be/jHWiegCYZj8

## Announcements and Highlights since the last meeting

- seph changed stuff in packaging serving. See agenda items

## Any Questions / Issues / PRs people want to discuss?

The intent of this section is to provide a clear time for community members to bring up _anything_.

Broad questions? Bugs? Deployment questions? Blocked PRs?

## Package Serving

Last time seph had real stats, `pkg.osquery.io` was pushing about 2 terabytes a day of data. This is quite a lot, and it's hard to do cheaply. As most of this traffic was going to AWS hosts, seph was leverating that s3 outbound traffic is free inside a region. This led to a complex setup: S3 replicated data to buckets in several regions, and a CloudFront function to send redirects based on ip to the bucket for the region. This worked pretty well.

But it's always nice to reduce costs.

Cloudflare's R2 is an S3 compatible platform with "free" data egrees. It's not wholly free, they charge a small amount for the number of `GET` requests.

There is a big wrinkly in using Cloudflare -- They really only want to work with domains they are they registrar for. We cannot move `osquery.io` there (it's held by the Linux Foundation), so I registered a new domain to be hosted with them.

To test out serving like this, and to see what the costs are like, seph adjusted the AWS CloudFront function to redirect everything to R2. It seems to work. seph suspects that the Cloudfront R2 costs are going to run about $100/month given the request rate, so there will probably be more iteration. Current theory is that the AWS hosts will get redirect to AWS buckets, and the remainder can be served by R2.

## Codesigning Updates

To support serving things through R2, seph started revisiting and refactoring the codesign workflows.

Today, there is a single workflow to sign and push code. This combined workflow means that when code goes from beta to stable, it is rebuilt.

seph refactored that to split it into a `build` and `promote` workflow. And additionally updated the `promote` to support pushing to both S3 and R2.

This work is in https://github.com/osquery/osquery-codesign/pull/43 (not public)

## macOS `managed_policies`

Question that originated on [slack](https://osquery.slack.com/archives/C08VA8R6F/p1690970615433469), and has now made it to office hours

> at the managed _policies, it seems that when the values are not integers, strings or booleans, they are reported as empty strings, probably because of this. It would be interesting to report all the values, but I understand it is important to maintain backward compatibilities. What about introducing a "key" column that is a flattened path to the value (dot separated key or array index)? Has something similar already been discussed ?

Looking at this, it looks like an incomplete EAV implementation. Today this table has columns for:
- domain
- name
- value

But that does not correctly handle nested data. So the proposal is to add appropriate columns, like `key` which would contain a flattened form of the full key.

seph is a big EAV fan, but notes a couple of things:
- `key` feels misnamed. And suggests something like `fullkey` or something
- It's worth having a `parent`
- This does have a small API breakage -- it creates the potential for keys with the same `name`. But it seems like a reasonable tradeoff.

Other conversation is what to use for the flattening. Proposal was `.`, but `/` might be more natural.

## 5.10.0 Release

Some discussion about PRs to review here

## Look At Open CVS / Security Tickets

[List of Issues tagged with Security](https://github.com/osquery/osquery/issues?q=is%3Aopen+is%3Aissue+label%3Asecurity)

Stefano is looking at these, and almost through the two.

## Look at old PRs

_(If there's time, we've been trying to re-visit old PRs)_

[Reverse Sorted List of PRs](https://github.com/osquery/osquery/pulls?q=is%3Apr+is%3Aopen+sort%3Acreated-asc)