I have something I've been working on that should be useful to a lot of people (including me), give me another few days to firm up the foundations, then I'll share what I have so far.
The general idea is:
Over time, any app / project produces a certain number of (errors, results, unique values) per (day / hour / minute). If that number changes by a high %, that's probably a problem, and as the person in charge of that app, you should know about it.
Let's say you run a google scraper with 1000 proxies, and normally on any given day 100 of those are banned by google, which is fine, your app reports that value of 100 to my system. Let's say today, for whatever reason, it's now 500 proxies banned. The value of 500 is reported to my system, my system knows something's up, and shows you.
Types of data that can be recorded:
Incrementing an integer for a given time period.
Recording the smallest integer seen in a given time period.
Recording the largest integer seen in a given time period.
Recording the first integer seen in a given time period.
Recording the number of unique items seen in a given time period (uses hyperloglog).
My intent is to start recording dozens to hundreds of metrics in my own projects, it's the only way I can have confidence that things are properly working in production.
This is the API for python projects, but it'll be easy to implement in any language, probably including JS:
Code:client = Client(r, periods=('10m', '1h', '24h')) client.incr(processed_jobs=5, failed_jobs=4) client.max(big_number=20010, hi=5, x=77) client.min(smallest_penis=1) client.set(a=1, b=2, c=3) client.count(x='potato', y='lemon')
What you want has already been coded in PHP.
And before you ask, no I am not selling it ( I use it for metrics outside of programming stuff, but it can be used in this fashion as well )