Page 244 of 250

Re: Coding: Fleeting Thoughts

Posted: Wed Sep 20, 2017 7:31 pm UTC
by hotaru
Flumble wrote:Is there a linuxy way to monitor services and send an email whenever a services dies or comes back up (and preferably every hour while something's down)?
I'm currently using systemd timers to run a small script to log the status of things like the xkcd minecraft server and I could run a sendmail command when the status is negative, but it seems there ought to be a better way.
If there's no linuxy way, I'll look into graphite+grafana or <insert better suggestion>.

you could use OnFailure= in the the unit file with https://github.com/kylemanna/systemd-utils to send an email notification.

Re: Coding: Fleeting Thoughts

Posted: Wed Sep 20, 2017 8:00 pm UTC
by Xeio
commodorejohn wrote:
Hey, those Homestar Runner cartoons aren't gonna play themselves.
Actually something more banal, extracting media from them. Granted there is swfexport to do that, though it makes Windows Defender very very angry, and also some of the files I'm exporting from seem to just... sometimes not follow the spec so... fun.

It was kinda interesting though. Why do I do this for fun.

Related, it's really annoying that BitMap.SetPixel is soooo much slower than using an unsafe block with UnlockBits. Like I get being slower due to some bounds checks, but why is it so much slower?

Re: Coding: Fleeting Thoughts

Posted: Wed Sep 20, 2017 8:09 pm UTC
by commodorejohn
Yeah, swfexport was already spotty back in the days when Flash MX was new, I can't imagine it copes well with newer versions.

This takes me back to the days of extracting random J-pop tracks from animutations because they never credited the original source and I had no idea what to Google...

Re: Coding: Fleeting Thoughts

Posted: Thu Sep 21, 2017 10:51 pm UTC
by Flumble
hotaru wrote:you could use OnFailure= in the the unit file with https://github.com/kylemanna/systemd-utils to send an email notification.

I'm not running the services on that that device, just the monitors (which are mostly of the form "try to establish a TCP connection/HTTP session" to find out whether the (black box) program is running properly). I can let the monitor purposefully stop with an error code to trigger OnFailure, but that's as ugly as having a separate script to send emails.


My eye is caught by the influxdb TICK stack now, mostly because it comes with docker images and a simple logging plugin that runs a script periodically and takes care of the output by itself ...and you can configure it to send emails when something goes wrong.

Re: Coding: Fleeting Thoughts

Posted: Sat Sep 23, 2017 11:09 am UTC
by Tub
Flumble wrote:I can let the monitor purposefully stop with an error code to trigger OnFailure, but that's as ugly as having a separate script to send emails.

What's ugly about that? Your watchdog reports failure via exit code, and systemd does the notification as configured. That seems like it's working as intended.

It would be cleaner to have the OnFailure on the server where the minecraft server is running, but apparently that's not an option?

Also, what exactly is causing the minecraft server to fail? Does the server process crash? Does the server process enter a deadlock?. Does the whole server crash and fail to reboot? Network connectivity issues? Something else entirely? Depending on the problem, I'd suggest different fixes.

Flumble wrote:My eye is caught by the influxdb TICK stack now, mostly because it comes with docker images and a simple logging plugin that runs a script periodically and takes care of the output by itself ...and you can configure it to send emails when something goes wrong.

That seems like overkill. If you seek reliability, then complexity is your enemy. Unless you're willing to ignore issues like updates, security fixes and general maintenance, then dumping a docker image somewhere is usually more trouble than it's worth. It's certainly more brittle than a sendmail script. I may be a pessimist, but I wouldn't be surprised if that monitoring solution has more downtime than the minecraft server you meant to monitor.


On my home network, I'm using collectd. It's still overkill for monitoring a single service, but it has plugins for many things I want to monitor (network bandwidth on my router, temperature of all CPUs, free disk space and SMART errors etc), supports custom plugins (e.g. I have a few raspberry pis with DHT22 temperature/humidity sensors) and it has threshold-based alerts, including things like "only notify if the server couldn't be reached 3 times in a row".
Distributed logging is supported, e.g. every computer runs a collectd instance, which reports the data to a central server where it gets written to some kind of database. Collectd is flexible about its storage backends. IIRC influxdb works, but I'm using rrdtool for its simplicity and built-in graphing features.

Configuration of the whole thing takes a while, but installation is trivial (most distributions have a package), updating is trivial (the configuration format is very stable; only once did I have to manually adjust something) and the overhead is low enough that I can just start an instance on each computer I need to monitor, including the low-powered raspberry pis.

Re: Coding: Fleeting Thoughts

Posted: Mon Oct 16, 2017 8:07 pm UTC
by Xeio
Calculating the number of years between two dates, particularly when the client wants it to be a decimal... based on # of months out of 12... is apparently a non-trivial problem.

Re: Coding: Fleeting Thoughts

Posted: Mon Oct 16, 2017 9:53 pm UTC
by Tub
The bad news: dates are hard.
The good news: if you cheat, the client (probably) won't notice. Because the client doesn't understand dates either, they have no idea what they're requesting, and thus won't notice even if your results are a few millenia off.

Then again, even though your problem is underspecified, I don't think it'd be more than 2-5 lines with a standard date API?

Re: Coding: Fleeting Thoughts

Posted: Tue Oct 17, 2017 1:17 pm UTC
by Xeio
Well, I probably over-engineered given you're right they probably have no clue what they're asking for, but something like this seemed to work:

Code: Select all

if(larger.Year > smaller.Year)
{
    years = larger.Year - smaller.Year
    if(smaller.AddYears(years) > larger) years--;
    smaller = smaller.AddYears(years);
}

if(smaller.Month < larger.Month)
{
    months = larger.Month - smaller.Month;
    if(smaller.AddMonths(months) > larger) months--;
}
else if(smaller.Month > larger.Month)
{
    months = 12 - smaller.Month + larger.Month;
    if(smaller.AddMonths(months) > larger) months--;
}

total = years + months / 12;


.Net doesn't provide something ilke this out of box, primarily because a TimeSpan doesn't make sense to have a .Months or .Years property. A timespan is measured in ticks, and months/years have different amounts of ticks depending on when the span would start...

EDIT: I think I fixed it more, though it's even more verbose now. :P

Re: Coding: Fleeting Thoughts

Posted: Tue Oct 17, 2017 2:19 pm UTC
by Thesh
Basically, you wish .NET had a DateDiff function like Visual Basic and SQL Server.

Re: Coding: Fleeting Thoughts

Posted: Tue Oct 17, 2017 6:43 pm UTC
by Xeio
Hmmm, well there is interop so I probably could include the VB DLL... On the other hand that's pretty terrible so I'd rather not.

Re: Coding: Fleeting Thoughts

Posted: Tue Oct 17, 2017 8:15 pm UTC
by Tub
Just connect to an SQL server and SELECT DATEDIFF($date1, $date2) :roll:

With a date library based on unix time, it's rather simple.

Code: Select all

function date_diff(date1, date2) {
  var diff_in_ms = Math.abs(date1.getTime() - date2.getTime());
  return diff_in_ms / 1000 / 60 / 60 / 24 / 365.25; // close enough.
}


But microsoft does not like unix time, and if you want the difference between the beginning of the months, try this:

Code: Select all

function date_diff_months(date1, date2) {
  var date1_in_months = date1.getYear() * 12 + date1.getMonth();
  var date2_in_months = date2.getYear() * 12 + date2.getMonth();
  return Math.abs(date1 - date2) / 12;
}

This works, no matter if your language returns getYear() as 2 or 4 digits or 3 digits since 1900, and no matter if getMonth() returns 0- or 1-based values.

Re: Coding: Fleeting Thoughts

Posted: Fri Oct 27, 2017 8:52 am UTC
by You, sir, name?
Working with time is hard. Working with dates is even harder. There's leap days, leap seconds, sometimes the timezone changes based on the date, indexing is comically inconsistent on a lot of big APIs.

My favorite is stuff is financial rules like "find the third wednesday of the month that date X is in, unless it's a red day, then the following business day". That's... kinda hard. On a level where I sort of want to just create a static table of wednesdays per month from 1970 through 2040 or whatever and then use that instead of looking it up. You know I'm going to use it for my unit test anyway.

Re: Coding: Fleeting Thoughts

Posted: Fri Oct 27, 2017 2:04 pm UTC
by Tub
You, sir, name? wrote:My favorite is stuff is financial rules like "find the third wednesday of the month that date X is in, unless it's a red day, then the following business day". That's... kinda hard.

Is it? You're only working with dates, thus no timezones, leap seconds or anything actually troublesome. And weeks are surprisingly regular - there has not been an instance where, say, a wednesday was skipped or where two wednesdays followed each other. Calculating whether a specific date is a wednesday is something that any crappy date class should be able to do for you.

The "for (day = 1; day < 30; day++)"-solution isn't beautiful, but works (and may thus be a good reference implementation for your test cases, so your actual implementation can be [18,17,16,15,21,20,19][new Date(d.getFullYear(), d.getMonth(), 1).getDay()] or something).

The hardest part is the function that returns all holidays, because that needs constant updates, and maybe you got to deal with the cases where the calculated date changes because a holiday appeared.

Re: Coding: Fleeting Thoughts

Posted: Sat Oct 28, 2017 3:45 pm UTC
by Xanthir
Tub wrote:
And weeks are surprisingly regular - there has not been an instance where, say, a wednesday was skipped or where two wednesdays followed each other.

Russia in 1918 would like to have a word with you. (If I'm reading the calendar right, they had two thursdays in a row.)

(But I'm being a pedant.)

Re: Coding: Fleeting Thoughts

Posted: Sat Oct 28, 2017 4:39 pm UTC
by Tub
Sorry, I don't believe in dates before 1970.

Seriously, do you have a source for that? Google is surprisingly resistant against the search terms I'm using..

Re: Coding: Fleeting Thoughts

Posted: Sat Oct 28, 2017 6:42 pm UTC
by Xenomortis
https://en.wikipedia.org/wiki/Adoption_ ... ern_Europe
The USSR switched from the Julian to the Gregorian calendar in 1918 - the 30th Jan 1918 (Wednesday) was followed by 14th Feb (Thursday).
Not quite "two sequential Thursdays", but a date-time complication nonetheless.

Re: Coding: Fleeting Thoughts

Posted: Mon Oct 30, 2017 2:03 am UTC
by phlip
Possible confounding factor here being that 30 Jan 1918 on the Gregorian calendar was a Thursday... but, 30 Jan 1918 on the Julian calendar was a Wednesday, so Russia had a Wednesday followed by a Thursday, nothing unusual there, just the actual date did weird things.

There are also cases where a place jumped the dateline... switching timezone from UTC-12 to UTC+12 (or vice-versa, I can't remember). They skipped a day, but on both the "days of the week" and "days of the month" sense, so the idea that a particular calendar date is a particular day of the week, and these are an entirely regular pattern, still holds up. As long as you're still working with the Gregorian calendar, your Wednesdays are going to have dates that are 7 days apart.

Re: Coding: Fleeting Thoughts

Posted: Mon Oct 30, 2017 1:12 pm UTC
by ucim
In any case, the issue is the same as travelers who cross a time zone. They have (say) 3:13 followed by 2:14. But it's 3:13 central time, followed by 2:14 mountain time, which is not incongruous at all. If you don't include the timezone (or the calendar system), you are not specifying time completely, and the ambiguity is simply stored in the unstated parts.

Tomorrow, "the thirty-first" is going to be followed by "the first", and few people will be confused by the omission of the month name.

Of course, for those following the One True Calendar, it will be 25 Voyage, 4AT, followed by 26 Voyage, 4AT. It's just a question of standards.

Jose

Re: Coding: Fleeting Thoughts

Posted: Tue Oct 31, 2017 4:56 am UTC
by Xanthir
phlip wrote:
Possible confounding factor here being that 30 Jan 1918 on the Gregorian calendar was a Thursday... but, 30 Jan 1918 on the Julian calendar was a Wednesday, so Russia had a Wednesday followed by a Thursday, nothing unusual there, just the actual date did weird things.

Ah, indeed. I just did a cal lookup and counted backwards, and didn't realize the Julian day was different.

Re: Coding: Fleeting Thoughts

Posted: Sun Nov 19, 2017 10:51 pm UTC
by Tub
Today's random thought: "I wish I had a javascript-to-php transpiler."

As horrible as that sounds, it would solve a real problem I have.

Re: Coding: Fleeting Thoughts

Posted: Mon Nov 20, 2017 12:42 pm UTC
by Xenomortis
No, I can see that one would need a translator if they wanted to convert one lunatic's mad ramblings into the dialect of a depressed drunk.
What I can't envisage is wanting to communicate with either of these people whilst sane and/or sober.

Re: Coding: Fleeting Thoughts

Posted: Mon Nov 20, 2017 11:43 pm UTC
by Xanthir
Tub wrote:
Today's random thought: "I wish I had a javascript-to-php transpiler."

As horrible as that sounds, it would solve a real problem I have.

AND I HAVE A SOLUTION FOR YOU

My friend's project, Cassis, is a subset of JS and PHP that works the same across both languages (plus a supporting library to get a bunch of functions working in both). You can write code in Cassis and deploy it both in browsers and servers. ^_^

It purposely takes on a more PHP-centric, functions-not-methods approach, as that maximizes the surface area of identical syntax.

Re: Coding: Fleeting Thoughts

Posted: Tue Nov 21, 2017 4:56 pm UTC
by Tub
Xanthir wrote:My friend's project, Cassis, is a subset of JS and PHP that works the same across both languages (plus a supporting library to get a bunch of functions working in both). You can write code in Cassis and deploy it both in browsers and servers. ^_^

That's great, now all I need is a JavaScript to Cassis transpiler.

Honestly, that's not even a transpiler. Actual transpilers exist, and they can transpile a lot more code - just not the code I have. Solutions like Cassis are a dime a dozen, though most of them never bother going for a public release - you always end up implementing only the subset of the APIs that your project needs, which means that reusability is low. I've had similar code somewhere, but it's useless for my current problem - existing code that was actually written as JavaScript.

The basic premise is interesting though. Remembering that $ is a valid character in javascript variable names, you can observe that php and javascript appear lexically compatible:

Code: Select all

function isPrime($n) {
    for ($i=2;$i<$n;$i++) {
        if ($n % $i == 0)
            return false;
    }
    return true;
}

This is both valid JavaScript and PHP! We can use variables (though with restricted naming), loops, if/else, functions and simple arithmetic operations.

Recent PHP versions (>5.4) also offer the [] array literal syntax, which allows compatible syntax for arrays!

Code: Select all

function sumPrimes($array) {
    $sum = 0;
    for ($i=0;$i<count($array);$i++) {
        $e = $array[$i];
        if (isPrime($e))
            $sum += $e;
    }
    return $sum;
}

$sum = sumPrimes([2,4,6,9,5,8,13]);

Well, it's not entirely compatible. count($array) is not a javascript function. That's where Cassis enters the scene, re-implementing count() in javascript. Also, let's implement print().

Code: Select all

function print(n) {
    console.log(n);
}

function count(arr) {
    return arr.length;
}

Minor differences apply (carriage returns after print, count() behavior on non-arrays), but those could be fixed if one felt like it.
All together, this is our first useful example program!

Code: Select all

function isPrime($n) {
    for ($i=2;$i<$n;$i++) {
        if ($n % $i == 0)
            return false;
    }
    return true;
}

function sumPrimes($array) {
    $sum = 0;
    for ($i=0;$i<count($array);$i++) {
        $e = $array[$i];
        if (isPrime($e))
            $sum += $e;
    }
    return $sum;
}

print(
sumPrimes([2,4,6,9,5,8,13]));

Code: Select all

> cat lib.js code.phpjs | node ; echo '-------' ; cat lib.php code.phpjs | php
20
-------
20


This wasn't a lot of code, right? Very manageable, very readable, even tested and correct. Did you spot the problem?

Re: Coding: Fleeting Thoughts

Posted: Thu Nov 23, 2017 11:40 pm UTC
by gd1
So basically in C,
Array = duct tape
Malloc = wd40
?

Re: Coding: Fleeting Thoughts

Posted: Sun Nov 26, 2017 5:34 pm UTC
by Tub
Tub wrote:This wasn't a lot of code, right? Very manageable, very readable, even tested and correct. Did you spot the problem?

Aww, nobody wanted to bite.

Here's the thing: php does not have variable declarations. You just initialize them, and then you have your variable.

Code: Select all

$i = 0;

Functions generate a new scope, but blocks don't. To access a variable from a parent scope, you must use the global keyword.

Code: Select all

$a = 0;$b = 1;
function a() {
   global $a;
   echo $a; // 0
   echo $b; // PHP Notice:  Undefined variable: b in..
}


JavaScript on the other hand does have variable declarations, which creates those variables in a local scope (function level with var, block level with let or const).
When using a variable that was not declared in the current scope, JavaScript bubbles up the scope hierarchy until a variable with the name was found. If no variable was found, reads return undefined, but writes will create the variable in the top-level scope.

Code: Select all

var $i = 0;
function a() {
   console.log($i); // 0
   var $a = 1;
   $b = 2; // $b is created in the top-level scope
   console.log($a, $b); // 1, 2
}
console.log($a, $b); // undefined, 2
}


tl;dr: in my code above, when interpreted as JavaScript, $i is a global variable, and both loops keep influencing each other. The only reason it appears to return the correct value is by careful choice of the array. Xanthir, you should get better friends.

Re: Coding: Fleeting Thoughts

Posted: Mon Nov 27, 2017 1:43 am UTC
by Xanthir
Tub wrote:
Xanthir, you should get better friends.

And you should be less randomly insulting?

Re: Coding: Fleeting Thoughts

Posted: Mon Nov 27, 2017 1:39 pm UTC
by Tub
I meant to end the post on a flippant twist, but.. yeah, sarcarsm and the internet. Apologies.

I do encourage your friend to extend the documentation though. He's clearly trying to get others to use the code, with a name and no less than two domains and everything, but then he refuses to even mention the giant, non-obvious and hard to debug footgun hidden within. That seems kinda impolite.

Re: Coding: Fleeting Thoughts

Posted: Mon Nov 27, 2017 9:08 pm UTC
by Xanthir
Hm, indeed, and I'm not sure off the top of my head how to fix it; PHP technically has a "var" keyword, but it's used in class declarations, not normal code.

ETA: Ah, he has some comment hacks to put var declarations for all the function's variables at the function's head, only visible to JS: https://github.com/tantek/cassis/blob/master/cassis.js#L530 I've asked him to document it.

Re: Coding: Fleeting Thoughts

Posted: Tue Nov 28, 2017 2:52 pm UTC
by Tub
Yeah, I already checked, the only way to solve it in a syntax compatible way is to require all variables to be declared const :roll:

Re: Coding: Fleeting Thoughts

Posted: Tue Nov 28, 2017 3:15 pm UTC
by chridd
Tub wrote:Yeah, I already checked, the only way to solve it in a syntax compatible way is to require all variables to be declared const :roll:
...or you could use default arguments:

Code: Select all

function f($n, $i = 0, $sum = 0) {
   for($i = 0; $i <= $n; $i++)
      $sum += $i;
   return $sum;
}
f(10)
(gives 55 in both JS and PHP)

Re: Coding: Fleeting Thoughts

Posted: Tue Nov 28, 2017 9:46 pm UTC
by Xanthir
Tub wrote:
Yeah, I already checked, the only way to solve it in a syntax compatible way is to require all variables to be declared const :roll:

Why would they need to be declared const? That would prevent the variables from being bound to new values, but PHP imposes no such constraints.

chridd wrote:
...or you could use default arguments:

Ah, that's extremely clever! Means you're accidentally exposing a weird API, but still, very clever. ^_^

Re: Coding: Fleeting Thoughts

Posted: Wed Nov 29, 2017 3:17 am UTC
by Tub
Xanthir wrote:Why would they need to be declared const? That would prevent the variables from being bound to new values, but PHP imposes no such constraints.

Code: Select all

$a = 3; // valid js (but it's a global), valid php
var $a = 3; // valid js, invalid php
let $a = 3; // valid js, invalid php
const a = 3; // valid js, valid php

It's useless for implementing loops or anything that depends on user input, but at least the syntax is compatible. It's also good training if you ever plan to program directly in a SSA IR.

chridd wrote:Ah, that's extremely clever! Means you're accidentally exposing a weird API, but still, very clever. ^_^

Clever indeed, unfortunately not supported by all browsers. Default arguments are a more recent addition to JS, and IE won't support it.

Re: Coding: Fleeting Thoughts

Posted: Wed Nov 29, 2017 6:48 am UTC
by gd1
So, when programming in C apparently we deal with streams... is it okay to cross the streams?

<stdio.h>, <stdlib.h>, etc. Why so many std's in C?

linux command line stuff: cat, tail, scratch, :3 :>

Re: Coding: Fleeting Thoughts

Posted: Wed Nov 29, 2017 2:59 pm UTC
by commodorejohn
The best thing about std's, there are so many to choose from ;)

Re: Coding: Fleeting Thoughts

Posted: Wed Nov 29, 2017 3:26 pm UTC
by ucim
gd1 wrote:Why so many std's in C?
There was sport at its making?

Jose

Re: Coding: Fleeting Thoughts

Posted: Wed Nov 29, 2017 10:13 pm UTC
by hotaru
gd1 wrote:Why so many std's in C?

at least C doesn't have "using namespace std"...

Re: Coding: Fleeting Thoughts

Posted: Thu Nov 30, 2017 12:17 am UTC
by Xanthir
Tub wrote:
Xanthir wrote:Why would they need to be declared const? That would prevent the variables from being bound to new values, but PHP imposes no such constraints.

Code: Select all

$a = 3; // valid js (but it's a global), valid php
var $a = 3; // valid js, invalid php
let $a = 3; // valid js, invalid php
const a = 3; // valid js, valid php

It's useless for implementing loops or anything that depends on user input, but at least the syntax is compatible. It's also good training if you ever plan to program directly in a SSA IR.

Ah, I see. It still wouldn't be a good idea; PHP's const has some significant constraints on its usage, due to the fact that it defines a compile-time constant, and thus can only appear in the outermost scope. (Thus, it's not a block-scoped variable like JS's const is; it's not even a function-scoped variable. It also can only contain a very constrained set of literal expressions, that can be evaluated at compile-time.) I guess you can use a bare `const` in both for defining global constants to simple literal values, but that's about it.

chridd wrote:Ah, that's extremely clever! Means you're accidentally exposing a weird API, but still, very clever. ^_^

Clever indeed, unfortunately not supported by all browsers. Default arguments are a more recent addition to JS, and IE won't support it.

Meh.

Re: Coding: Fleeting Thoughts

Posted: Thu Nov 30, 2017 3:23 pm UTC
by Yakk
I have been messing with pipes.

My primitive plumbing types are:

Sink of T is: void(T)
Source of T is: void(Sink of T)
Pipe from T to U is: void(Source of T, Sink of U)

This creates something interesting, in that when you invoke a Source you never know how much you'll be Sinking. At the same time, the Source can produce them in any number and doesn't have to pass an object that represents that enumeration to you, it can do that internally.

Maybe this is just useful because we don't have good generators/coroutines in C++ yet; a source of T being a generator of T might be simpler?

It still gets you some interesting operations. (PipeA|PipeB)(src, sink) is PipeB( src|PipeA, sink), which is just so tidy.

I sort of want a Sink of T to be a void(Source of T), but that leads to an infinite definition recursion.

Converting U(T) to a Pipe(T->U) is easy. Converting Pipe(T->range of U) to Pipe(T->U) is easy (Pipe(range U->U) is trivial to write). Sadly, Pipe(U->range U) basically requires terminator signals (to know when to flush the buffer). Which makes me think I have a design problem.

Turning a T into a Source of T is a bit awkward, in that we either copy T needlessly or need to know if the Source will be invoked more than once.

I've experimented a bit with multiple argument sinks and sources. But it gets awkward, because notation on how to "line them up" is tricky. And moving them "together" is also awkward. A source of (A,B,C) isn't the same as a source of (A) a source of (B) and a source of (C), because the (A,B,C) source guarantees they come in 3s. On the other hand, a sink of (A,B,C) can be generated by stiching together a sink(A), sink(B) and sink(C). You lose some information, but it works.

Is there something analogous to covariant/contravariant going on here? If feels similar in some sense.

This is continuation passing style, in that source's don't return T's they instead take a continuation that consumes Ts, and similar for pipes. So I'd guess that functional programming has a rich heritage of design knowledge to plumb here that I'm ignorant of.

Is there a relationship with React Streaming?

Re: Coding: Fleeting Thoughts

Posted: Fri Dec 01, 2017 1:02 am UTC
by Xanthir
You might be interested in looking into Streams, like RxJS and the like. The concepts are very similar, and they've thought about many of these problems already.

Re: Coding: Fleeting Thoughts

Posted: Mon Dec 04, 2017 10:33 pm UTC
by Xeio
Todays Google doodle is nice.