chksm
e4c5ec5c95
|
2 years ago | |
---|---|---|
locale/en | 2 years ago | |
script | 2 years ago | |
README.md | 2 years ago | |
changelog.txt | 2 years ago | |
control.lua | 2 years ago | |
info.json | 2 years ago | |
settings.lua | 2 years ago | |
thumbnail.png | 2 years ago |
README.md
Statorio
A Factorio mod to export realtime JSON statistics
Choose your metrics and desired frequency
Receive sweet JSON on disk
Numbers go brrr
The goal is to expose as much of the game API as possible (now and in the future, vanilla or modded, single and multiplayer) and export it in a machine-readable way.
⚠️ This is experimental, you may encounter (and please report) crashes, performance hits, security issues.
How to use
- Install the mod
- Start playing
- Check
{GAME_DIR}/script-output/stats.json
for JSON data (see game directory help) - Choose metrics with the
/stat
console command
Installation
Just like any other mod. For GUI versions:
- Launch the game
- Choose Mods
- Choose Install tab
- Wait for list to populate
- Search for
Statorio
- Click the search result
- Click Install on the right pane
- Click Confirm
Or for headless versions:
- Copy this mod's ZIP (or directory) to Factorio's
/mod
directory - Restart the server
Mod settings
Global runtime settings can be changed from the "Mod Settings" menu in game. Look for the Statorio section. On multiplayer servers this can only be changed by admins.
Frequency
How often (in game ticks) to collect metrics and write to disk. 60 game ticks is roughly equal to 1 second. A good value depends on the number of metrics you watch (more is slower), the cost of each metric (some eat CPU), etc.
Filename
Use a simple filename like stats.json
or a path like statorio/out.json
. Files are written to Factorio's script-output
directory.
Changing the filename setting will delete the old file (if it exists).
For statsd formatted metrics use a .statsd
file extension. The mod will write out lines like this suitable for statsd consumption:
game.players.1.online_time:36709853|g
game.players.2.online_time:47257460|g
Start playing
Play any single player game on your local computer and Statorio will start running. If you save the game then Statorio will become a dependency of your save, and request installation (if its missing) each time the game is loaded. Its safe to remove if you don't want to use it any more.
For multiplayer games Statorio must be installed on the server as well as for all players in the game. You can't just run it locally for public servers, for example.
Limitation: If installed on a multiplayer game then all connected clients will also calculate and export statistics locally. This is a bug and planned feature. Classy.
Check the output data
Look in Factorio's data directory for a folder called script-output
. This is the only location on disk that mods are permitted to write files. By default Statorio creates a file called stats.json
(can be changed in configuration).
Windows users might look in C:\Users\{USERNAME}\AppData\Roaming\Factorio\script-output\
Linux users could look in ~/.factorio/script-output/
You might see glorious JSON key pairs like this:
{
"game.forces.player.item_production_statistics.get_flow_count.iron-plate.input.one_minute": 532.77591973244,
"game.forces.player.item_production_statistics.get_flow_count.iron-plate.output.one_minute": 359.19732441472
}
Actually its probably an empty file, because we haven't defined any metrics yet:
{}
Also, metrics won't be listed in the JSON output if a value is blank, or missing, or an error occurred. However values of 0 should make it through.
Available JSON data types are used, so expect floating point numbers, integers, boolean values and strings.
For large files: There is an issue where Factorio is not writing the file atomically. If you use something like fswatch
to watch for changes to the JSON then add a short delay to allow Factorio to finish writing to the file.
Console command
/stat test <metric>
will test if a metric works, if so display the current value
/stat add <metric>
will add a metric (see examples below)
/stat list
will show all metrics being exported right now
/stat del <number>
will delete a metric with that number (see the list output)
Some examples
Metric | Description |
---|---|
game.forces.player.get_entity_count.gun-turret |
Turrets deployed by the vanilla player force |
game.forces.*.get_entity_count.biter-spawner |
Biter spawners deployed by the vanilla AI enemy force |
game.forces.*.get_entity_count.transport-belt |
Transport belts deployed for all forces |
game.forces.player.logistic_networks.nauvis.*.available_logistic_robots |
Number of available logistics bots on all player networks |
game.players.*.name and game.players.*.online_time |
Get all player names and total play time on this map |
game.forces.player.item_production_statistics.get_flow_count.iron-plate.count.one_minute |
Iron plates produced in the last minute |
game.forces.*.entity_build_count_statistics.get_flow_count.stone-furnace.input.one_hour |
Stone furnaces built by each force in the last hour |
game.forces.*.get_entity_count.stone-furnace |
Total number of stone furnaces for each force |
game.forces.player.current_research.research_unit_count |
Current research progress |
game.forces.enemy.evolution_factor |
Enemy evolution factor |
game.forces.*.evolution_factor |
All forces evolution factor |
game.forces.*.ai_controllable |
Human vs AI forces |
game.forces.player.entity_build_count_statistics.input_counts.stone-furnace |
Stone furnaces produced |
game.forces.player.entity_build_count_statistics.output_counts.stone-furnace |
Stone furnaces consumed |
game.forces.*.get_entity_count.stone-furnace |
Metric names
At its core the metric naming convention follows the structure of Factorio's game
object, an instance of LuaGameScript. All metrics begin with game.
to show this (and allow for future expansion).
Reading basic properties is simply a matter of following the LuaGameScript API documentation above. For example game.difficulty
or game.ticks_played
. Simple data types like uint
, boolean
, string
, float
, double
etc is straightforward enough.
Accessing child classes is easy by reference using the dot notation. For example game.map_settings.path_finder.short_cache_size
Traversable data types like CustomDictionary
can be referenced by an entity's name or index in the table. For example game.forces.player.rockets_launched
reads the force called player
, and game.forces.1.rockets_launched
reads the first force defined in the table.
Wildcards can be used for traversable data types. For example game.forces.*.rockets_launched
returns a metric for each force.
Some functions can be used but implementation is an ongoing effort. Currently supported are:
get_entity_count(prototype)
Used on anything that supports get_entity_count
.
Requires a Factorio prototype as an argument.
For example game.forces.player.get_entity_count.stone-furnace
This has a performance hit. No wildcard support.
get_flow_count(prototype, [input|output], precisionIndex)
Used to get production and consumption total counts, or flow count values for a given time frame, from anywhere in the API that implements LuaFlowStatistics
.
prototype is a Factorio prototype to get flow count for. Wildcard *
is accepted.
[input|output] in the context of flow statistics describe on which side of the associated GUI the values are shown. Input values are shown on the left side, output values on the right side. For items and liquids input=production and output=consumption, power input=consumption and output=production., for kill statistics input=kills and output=deaths.
precisionIndex can be one of Factorio's defines.flow_precision_index like one_second
, one_minute
, ten_minutes
, one_hour
.
For example game.forces.player.kill_count_statistics.get_flow_count.*.input.one_hour
for all kills or game.forces.player.kill_count_statistics.get_flow_count.character.output.one_hour
deaths. game.pollution_statistics.get_flow_count.*.input.one_hour
for all pollutants on the map.
Realtime graphs
There's a lot of ways to make graphs from the Statorio JSON output (NodeRED, Grafana, custom), but I haven't explored them yet to write a guide.
I use Netdata (minimal, powerful, open source, and cloud account not required) which includes a metrics collector plugin called statsd
. Netdata is installed on the same server as the Factorio headless server for simplicity but there's no reason it couldn't be elsewhere. Follow the quick setup guide and get it working for the usual server metrics first. Just remember when browsing the docs that you're using the Netdata Agent only.. other features need a (free) cloud account.
Configure Statorio to output a filename ending .statsd
- the file now contains one metric per line in a format that statsd
can digest (instead of JSON). A simple bash command or similar script can be used to watch the file for changes and stream it to statsd over TCP. Here is a sample command, run it from Factorio's script-output
directory:
while true; do fswatch -1 stats.statsd | xargs -0 -n1 -I{} sleep 1; ncat --send-only localhost 8125 < stats.statsd; done;
Refresh the Netdata dashboard. You should see simple metric graphs appear under the "statsd" heading. Ugly but we know its working.
Finally the Netdata statsd plugin can be configured with your own synthetic charts to show realtime graphs.
Here's an example:
# /etc/netdata/statsd.d/statorio.conf
[app]
name = Factorio
metrics = game.*
private charts = yes
gaps when not collected = no
memory mode = ram
[player.production.raw_material]
title = Plates
family = Production
context = game.production.raw_material
units = items/m
type = area
dimension = game.forces.player.item_production_statistics.get_flow_count.iron-plate.input.one_minute 'Iron'
dimension = game.forces.player.item_production_statistics.get_flow_count.copper-plate.input.one_minute 'Copper'
dimension = game.forces.player.item_production_statistics.get_flow_count.steel-plate.input.one_minute 'Steel'
Restart Netdata with the new configuration sudo systemctl restart netdata
and now you should have realtime graphs.
Graphs are updated as often as statsd receives metrics, which in theory should be the number of game ticks you set in Statorio mod settings. I have it at 60-180 ticks (approx 1-3 seconds) on a moderate size base with no problems.
Now you can create more synthetic graphs for the metrics you want to watch, group them in a logical way, and even create Netdata alarms for your factory.
It's even possible to make custom dashboards using just basic HTML and the included dashboard.js
library. Create gauges, charts, sparklines, pie charts and more. Super 📈
Exploring the vanilla game API
If you can't find what you need in Factorio's game API then here are some clues to what's available.
Game
Where almost everything happens. Accessed with game.
. See API docs for LuaGameScript.
Some examples...
Metric | Output |
---|---|
game.difficulty |
|
game.tick |
14829 |
game.ticks_played |
14829 |
game.ticks_paused |
442 |
Players
Anything related to human players who have connected to the game. Access an array of all players who have ever joined with game.players
or players connected now with game.connected_players
. See API docs for LuaPlayer.
Get a list of connected players with /stat test game.players.*.name
to begin exploring.
Some examples...
Metric | Output |
---|---|
game.players.1.name |
platypus |
game.players.1.connected |
true |
game.players.1.admin |
true |
game.players.1.online_time |
42914 |
game.players.1.afk_time |
4914 |
game.players.1.last_online |
|
game.players.1.in_combat |
false |
game.players.1.crafting_queue_size |
|
game.players.1.crafting_queue_progress |
Forces
Forces are like teams, and most statistics are found here. Access an array of forces with game.forces
. Vanilla games contain player
, enemy
and neutral
forces. See API docs for LuaForce.
Get a list of forces with /stat test game.forces.*.name
to begin exploring.
Example team modifiers and bonuses...
Metric | Output |
---|---|
game.forces.player.worker_robots_speed_modifier |
|
game.forces.player.character_inventory_slots_bonus |
|
game.forces.player.maximum_following_robot_count |
|
game.forces.player.research_progress |
|
game.forces.player.rockets_launched |
0 |
game.forces.enemy.evolution_factor |
0.0013831931 |
game.forces.player.friendly_fire |
true |
Surfaces
Access information about the land/environment through an array of available surfaces. Vanilla games have one surface called nauvis
. Access from game.surfaces
. See API Docs for LuaSurface.
Metric | Output |
---|---|
game.surfaces.nauvis.daytime |
|
game.surfaces.nauvis.wind_speed |
|
game.surfaces.nauvis.wind_orientation |
|
game.surfaces.nauvis.peaceful_mode |
|
game.surfaces.nauvis.ticks_per_day |
Pollution
game.pollution_statistics...
for map pollutants (input) and offsetting pollution sinks (output). For example game.pollution_statistics.get_flow_count.boiler.input.one_minute
to see how much pollution boilers are causing.
Much more
I ran out of time. Check out the API docs and please tell me if you think of examples that should be listed here.
Why?
- Graphs and dashboards
- Bring a spare screen into your game with interesting graphs and metrics
- Watch and publish the state of a multiplayer game
- Use a graph or gauge as a source in OBS streams etc
- Chat bots
- Feedback and control (maybe)
- It's just JSON 🤷
Roadmap
Pull requests for any of these are very welcome:
- Console commands for admins only
- Cached settings for performance
- Check for multiplayer desyncs
- More game API functions exposed
- Robust API calling methods
- Multiplayer settings
- Test and set commands parse icons as entity names
- Monitor own mod performance and server impact
- Expose metrics to other mods
- Snarf metrics from other mods?
- LTN
- ...
- Extremely basic UI output
- More output file formats beyond JSON
- Support for simulations
- Translations of the mod and supporting documentation
- Sanitise game speed vs tick output
If you'd like to add anything else then its worth reaching out first.
Join in
Please use the Factorio forum for feedback. I'd love to hear about bugs, improvement suggestions from players, server admins and mod makers, and screenshots of how you use the mod in your group :)