Export realtime game statistics to disk in JSON and other formats.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
chksm e4c5ec5c95 Updates README since recent testing, adds changelog 2 years ago
locale/en Version 1.0.1 imported from releases 2 years ago
script get_flow_count should now support *FlowStatistics types 2 years ago
README.md Updates README since recent testing, adds changelog 2 years ago
changelog.txt Updates README since recent testing, adds changelog 2 years ago
control.lua Version 1.0.1 imported from releases 2 years ago
info.json Bumps version, updates metadata 2 years ago
settings.lua Version 1.0.1 imported from releases 2 years ago
thumbnail.png Version 1.0.1 imported from releases 2 years ago

README.md

Statorio

A Factorio mod to export realtime JSON statistics

Choose your metrics and desired frequency
Receive sweet JSON on disk
Numbers go brrr


The goal is to expose as much of the game API as possible (now and in the future, vanilla or modded, single and multiplayer) and export it in a machine-readable way.

This is experimental, you may encounter (and please report) crashes, performance hits, security issues.

How to use

  1. Install the mod
  2. Start playing
  3. Check {GAME_DIR}/script-output/stats.json for JSON data (see game directory help)
  4. Choose metrics with the /stat console command

Installation

Just like any other mod. For GUI versions:

  • Launch the game
  • Choose Mods
  • Choose Install tab
  • Wait for list to populate
  • Search for Statorio
  • Click the search result
  • Click Install on the right pane
  • Click Confirm

Or for headless versions:

  • Copy this mod's ZIP (or directory) to Factorio's /mod directory
  • Restart the server

Mod settings

Global runtime settings can be changed from the "Mod Settings" menu in game. Look for the Statorio section. On multiplayer servers this can only be changed by admins.

Frequency

How often (in game ticks) to collect metrics and write to disk. 60 game ticks is roughly equal to 1 second. A good value depends on the number of metrics you watch (more is slower), the cost of each metric (some eat CPU), etc.

Filename

Use a simple filename like stats.json or a path like statorio/out.json. Files are written to Factorio's script-output directory.

Changing the filename setting will delete the old file (if it exists).

For statsd formatted metrics use a .statsd file extension. The mod will write out lines like this suitable for statsd consumption:

game.players.1.online_time:36709853|g
game.players.2.online_time:47257460|g

Start playing

Play any single player game on your local computer and Statorio will start running. If you save the game then Statorio will become a dependency of your save, and request installation (if its missing) each time the game is loaded. Its safe to remove if you don't want to use it any more.

For multiplayer games Statorio must be installed on the server as well as for all players in the game. You can't just run it locally for public servers, for example.

Limitation: If installed on a multiplayer game then all connected clients will also calculate and export statistics locally. This is a bug and planned feature. Classy.

Check the output data

Look in Factorio's data directory for a folder called script-output. This is the only location on disk that mods are permitted to write files. By default Statorio creates a file called stats.json (can be changed in configuration).

Windows users might look in C:\Users\{USERNAME}\AppData\Roaming\Factorio\script-output\

Linux users could look in ~/.factorio/script-output/

You might see glorious JSON key pairs like this:

{
  "game.forces.player.item_production_statistics.get_flow_count.iron-plate.input.one_minute": 532.77591973244,
  "game.forces.player.item_production_statistics.get_flow_count.iron-plate.output.one_minute": 359.19732441472
}

Actually its probably an empty file, because we haven't defined any metrics yet:

{}

Also, metrics won't be listed in the JSON output if a value is blank, or missing, or an error occurred. However values of 0 should make it through.

Available JSON data types are used, so expect floating point numbers, integers, boolean values and strings.

For large files: There is an issue where Factorio is not writing the file atomically. If you use something like fswatch to watch for changes to the JSON then add a short delay to allow Factorio to finish writing to the file.

Console command

/stat test <metric> will test if a metric works, if so display the current value

/stat add <metric> will add a metric (see examples below)

/stat list will show all metrics being exported right now

/stat del <number> will delete a metric with that number (see the list output)

Some examples

Metric Description
game.forces.player.get_entity_count.gun-turret Turrets deployed by the vanilla player force
game.forces.*.get_entity_count.biter-spawner Biter spawners deployed by the vanilla AI enemy force
game.forces.*.get_entity_count.transport-belt Transport belts deployed for all forces
game.forces.player.logistic_networks.nauvis.*.available_logistic_robots Number of available logistics bots on all player networks
game.players.*.name and game.players.*.online_time Get all player names and total play time on this map
game.forces.player.item_production_statistics.get_flow_count.iron-plate.count.one_minute Iron plates produced in the last minute
game.forces.*.entity_build_count_statistics.get_flow_count.stone-furnace.input.one_hour Stone furnaces built by each force in the last hour
game.forces.*.get_entity_count.stone-furnace Total number of stone furnaces for each force
game.forces.player.current_research.research_unit_count Current research progress
game.forces.enemy.evolution_factor Enemy evolution factor
game.forces.*.evolution_factor All forces evolution factor
game.forces.*.ai_controllable Human vs AI forces
game.forces.player.entity_build_count_statistics.input_counts.stone-furnace Stone furnaces produced
game.forces.player.entity_build_count_statistics.output_counts.stone-furnace Stone furnaces consumed
game.forces.*.get_entity_count.stone-furnace

Metric names

At its core the metric naming convention follows the structure of Factorio's game object, an instance of LuaGameScript. All metrics begin with game. to show this (and allow for future expansion).

Reading basic properties is simply a matter of following the LuaGameScript API documentation above. For example game.difficulty or game.ticks_played. Simple data types like uint, boolean, string, float, double etc is straightforward enough.

Accessing child classes is easy by reference using the dot notation. For example game.map_settings.path_finder.short_cache_size

Traversable data types like CustomDictionary can be referenced by an entity's name or index in the table. For example game.forces.player.rockets_launched reads the force called player, and game.forces.1.rockets_launched reads the first force defined in the table.

Wildcards can be used for traversable data types. For example game.forces.*.rockets_launched returns a metric for each force.

Some functions can be used but implementation is an ongoing effort. Currently supported are:

get_entity_count(prototype)

Used on anything that supports get_entity_count.

Requires a Factorio prototype as an argument.

For example game.forces.player.get_entity_count.stone-furnace

This has a performance hit. No wildcard support.

get_flow_count(prototype, [input|output], precisionIndex)

Used to get production and consumption total counts, or flow count values for a given time frame, from anywhere in the API that implements LuaFlowStatistics.

prototype is a Factorio prototype to get flow count for. Wildcard * is accepted.

[input|output] in the context of flow statistics describe on which side of the associated GUI the values are shown. Input values are shown on the left side, output values on the right side. For items and liquids input=production and output=consumption, power input=consumption and output=production., for kill statistics input=kills and output=deaths.

precisionIndex can be one of Factorio's defines.flow_precision_index like one_second, one_minute, ten_minutes, one_hour.

For example game.forces.player.kill_count_statistics.get_flow_count.*.input.one_hour for all kills or game.forces.player.kill_count_statistics.get_flow_count.character.output.one_hour deaths. game.pollution_statistics.get_flow_count.*.input.one_hour for all pollutants on the map.

Realtime graphs

Ore mining graph

Production graph

There's a lot of ways to make graphs from the Statorio JSON output (NodeRED, Grafana, custom), but I haven't explored them yet to write a guide.

I use Netdata (minimal, powerful, open source, and cloud account not required) which includes a metrics collector plugin called statsd. Netdata is installed on the same server as the Factorio headless server for simplicity but there's no reason it couldn't be elsewhere. Follow the quick setup guide and get it working for the usual server metrics first. Just remember when browsing the docs that you're using the Netdata Agent only.. other features need a (free) cloud account.

Configure Statorio to output a filename ending .statsd - the file now contains one metric per line in a format that statsd can digest (instead of JSON). A simple bash command or similar script can be used to watch the file for changes and stream it to statsd over TCP. Here is a sample command, run it from Factorio's script-output directory:

while true; do fswatch -1 stats.statsd | xargs -0 -n1 -I{} sleep 1; ncat --send-only localhost 8125 < stats.statsd; done;

Refresh the Netdata dashboard. You should see simple metric graphs appear under the "statsd" heading. Ugly but we know its working.

Finally the Netdata statsd plugin can be configured with your own synthetic charts to show realtime graphs.

Here's an example:

# /etc/netdata/statsd.d/statorio.conf

[app]
    name = Factorio
    metrics = game.*
    private charts = yes
    gaps when not collected = no
    memory mode = ram

[player.production.raw_material]
    title = Plates
    family = Production
    context = game.production.raw_material
    units = items/m
    type = area
    dimension = game.forces.player.item_production_statistics.get_flow_count.iron-plate.input.one_minute 'Iron'
    dimension = game.forces.player.item_production_statistics.get_flow_count.copper-plate.input.one_minute 'Copper'
    dimension = game.forces.player.item_production_statistics.get_flow_count.steel-plate.input.one_minute 'Steel'

Restart Netdata with the new configuration sudo systemctl restart netdata and now you should have realtime graphs.

Graphs are updated as often as statsd receives metrics, which in theory should be the number of game ticks you set in Statorio mod settings. I have it at 60-180 ticks (approx 1-3 seconds) on a moderate size base with no problems.

Now you can create more synthetic graphs for the metrics you want to watch, group them in a logical way, and even create Netdata alarms for your factory.

It's even possible to make custom dashboards using just basic HTML and the included dashboard.js library. Create gauges, charts, sparklines, pie charts and more. Super 📈

Exploring the vanilla game API

If you can't find what you need in Factorio's game API then here are some clues to what's available.

Game

Where almost everything happens. Accessed with game.. See API docs for LuaGameScript.

Some examples...

Metric Output
game.difficulty
game.tick 14829
game.ticks_played 14829
game.ticks_paused 442

Players

Anything related to human players who have connected to the game. Access an array of all players who have ever joined with game.players or players connected now with game.connected_players. See API docs for LuaPlayer.

Get a list of connected players with /stat test game.players.*.name to begin exploring.

Some examples...

Metric Output
game.players.1.name platypus
game.players.1.connected true
game.players.1.admin true
game.players.1.online_time 42914
game.players.1.afk_time 4914
game.players.1.last_online
game.players.1.in_combat false
game.players.1.crafting_queue_size
game.players.1.crafting_queue_progress

Forces

Forces are like teams, and most statistics are found here. Access an array of forces with game.forces. Vanilla games contain player, enemy and neutral forces. See API docs for LuaForce.

Get a list of forces with /stat test game.forces.*.name to begin exploring.

Example team modifiers and bonuses...

Metric Output
game.forces.player.worker_robots_speed_modifier
game.forces.player.character_inventory_slots_bonus
game.forces.player.maximum_following_robot_count
game.forces.player.research_progress
game.forces.player.rockets_launched 0
game.forces.enemy.evolution_factor 0.0013831931
game.forces.player.friendly_fire true

Surfaces

Access information about the land/environment through an array of available surfaces. Vanilla games have one surface called nauvis. Access from game.surfaces. See API Docs for LuaSurface.

Metric Output
game.surfaces.nauvis.daytime
game.surfaces.nauvis.wind_speed
game.surfaces.nauvis.wind_orientation
game.surfaces.nauvis.peaceful_mode
game.surfaces.nauvis.ticks_per_day

Pollution

game.pollution_statistics... for map pollutants (input) and offsetting pollution sinks (output). For example game.pollution_statistics.get_flow_count.boiler.input.one_minute to see how much pollution boilers are causing.

Much more

I ran out of time. Check out the API docs and please tell me if you think of examples that should be listed here.

Why?

  • Graphs and dashboards
    • Bring a spare screen into your game with interesting graphs and metrics
    • Watch and publish the state of a multiplayer game
    • Use a graph or gauge as a source in OBS streams etc
  • Chat bots
  • Feedback and control (maybe)
  • It's just JSON 🤷

Roadmap

Pull requests for any of these are very welcome:

  • Console commands for admins only
  • Cached settings for performance
  • Check for multiplayer desyncs
  • More game API functions exposed
  • Robust API calling methods
  • Multiplayer settings
  • Test and set commands parse icons as entity names
  • Monitor own mod performance and server impact
  • Expose metrics to other mods
  • Snarf metrics from other mods?
    • LTN
    • ...
  • Extremely basic UI output
  • More output file formats beyond JSON
  • Support for simulations
  • Translations of the mod and supporting documentation
  • Sanitise game speed vs tick output

If you'd like to add anything else then its worth reaching out first.

Join in

Please use the Factorio forum for feedback. I'd love to hear about bugs, improvement suggestions from players, server admins and mod makers, and screenshots of how you use the mod in your group :)