Finally caught up with most of the PyCon 2018 videos. Here are my favorite so far.
Alex Gaynor – Learning from Failure: Post Mortems
Continue reading “Favorite PyCon 2018 Talks”
When I’m on my home Wi-Fi, I rarely connect to a VPN. When I’m out, I always make the habit of doing so, except when I forget. I was hoping VPN clients would have a feature to automatically connect based on a list of networks, but after some quick research, I guess not. Maybe I’m wrong, but ended up with this nifty solution. I’m currently using
Viscosity as my VPN client, but should work similarly for other clients that allow some scripting.
Continue reading “Connecting to a VPN automatically when not at home”
These are the slides for a recent talk on
osquery I gave at a DevOpsCT meetup.
osquery is an instrumentation framework that exposes an operating system as a high-performance relational database. This allows you to write SQL queries to explore operating system data.
This query returns any process whose original binary has been deleted or modified.
This query returns hosts whose primary disk is currently unencrypted.
This query allows us to see where specific logins have occurred within your infrastructure.
With osquery, SQL tables represent abstract concepts such as running processes, loaded kernel modules, open network connections, browser plugins, hardware events or file hashes.
File integrity monitoring is available for Linux and Darwin using inotify and FSEvents. The daemon reads a list of files/directories from the osquery configuration. The actions to those selected files populate the file_events table.
As file changes happen, events will appear in the file_events table. During a file change event, the md5, sha1, and sha256 for the file will be calculated if possible. A sample event looks like this…
The osquery daemon uses a default filesystem logger plugin. Like the config, output from the filesystem plugin is written as JSON. There are two types of logs: Status logs and Query schedule results logs, including logs from snapshot queries. osquery includes logger plugins that support configurable logging to a variety of interfaces. The built in logger plugins are filesystem (default), tls, syslog (for POSIX), windows_event_log (for Windows), kinesis, firehose, and kafka.
The results of your scheduled queries are logged to the “results log”. These are differential changes between the last (most recent) query execution and the current execution. Each log line is a JSON string that indicates what data has been added/removed by which query. The first time the query is executed (there is no “last” run), the last run is treated as having null results, so the differential consists entirely of log lines with the added indication.
Snapshot logs are an alternate form of query result logging. A snapshot is an ‘exact point in time’ set of results, no differentials. If you always want a list of mounts, not the added and removed mounts, use a snapshot. In the mounts case, where differential results are seldom emitted (assuming hosts do not often mount and unmount), a complete snapshot will log after every query execution. This will be a lot of data amortized across your fleet.
Query packs help you group queries by function or problem domain into files that are easy to download, distribute, and update. There are some query packs distributed by default…
An equivalent of the OSSEC rootkit database.
An incident response query pack to help you detect and respond to breaches
These are the slides for a recent 5 minute talk on Python Type Hints I gave at newhaven.io meetup. Slides are also available on
Let’s talk about type hints
PEP 3107 introduced syntax for function annotations, but the semantics were deliberately left undefined. There has now been enough 3rd party usage for static type analysis that the community would benefit from a standard vocabulary and baseline tools within the standard library.
PEP 484 introduces a provisional module to provide these standard definitions and tools, along with some conventions for situations where annotations are not available.
While these annotations are available at runtime through the annotations attribute…
…no type checking happens at runtime. Instead, the proposal assumes the existence of a separate off-line type checker which users can run over their source code voluntarily.
This PEP aims to provide a standard syntax for type annotations, opening up Python code to easier static analysis and refactoring, potential runtime type checking, and code generation utilizing type information.
Of these goals, static analysis is the most important. This includes support for off-line type checkers, as well as providing a standard notation that can be used by IDEs for code completion and refactoring.
integer of arbitrary size
floating point number
an arbitrary object
list of str objects
tuple of two int objects
tuple of an arbitrary number of int objects
dictionary from str keys to int values
iterable object containing ints
sequence of booleans
dynamically typed value with an arbitrary type
You can define new generic classes. Here is a very simple generic class that represents a stack. The Stack class can be used to represent a stack of any type
Generic type variables can also be used to define generic functions. As with generic classes, the type variable can be replaced with any type. That means first can be used with any sequence type, and the return type is derived from the sequence item type.
mypy is a static type checker for Python. If you sprinkle your code with type annotations, it can type check your code and find common bugs. As mypy is a static analyzer, your code’s type annotations are just hints and don’t interfere when running your program.
You can install it with pip
When you run your program with a standard Python interpreter, the annotations are treated primarily as comments.
When we run mypy against our source code…
…you’ll get useful errors before runtime.
If you use Visual Studio Code, the Python extension has great support for mypy.
Even though we now have typings, Python will remain a dynamically typed language, and the authors have no desire to ever make type hints mandatory, even by convention.
Most things in CoreOS Container Linux can be run in containers, except when it doesn’t make sense. Here’s how I got
osquery up and running.
osquery is an operating system instrumentation framework for Windows, OS X (macOS), Linux, and FreeBSD. The tools make low-level operating system analytics and monitoring both performant and intuitive.
osquery exposes an operating system as a high-performance relational database. This allows you to write SQL queries to explore operating system data. With osquery, SQL tables represent abstract concepts such as running processes, loaded kernel modules, open network connections, browser plugins, hardware events or file hashes.
Continue reading “Running osquery on CoreOS”