1190: "Time"

[mscha]

DONG...

-- posted by newpixbot

[Kieryn]

OK, so I had some free time today...

Basically I just re-parsed the entire thread (up to page 370 ish) using an XML parser (this site uses xhtml, so that helps) and also a natural language processing library (OpenNLP).
I also excluded any blockquotes from the post content, so it's all just original text, nothing duplicated.

As an example use of the data, I was able to distill the top nouns unique to this thread.

• newpix 734
• sandcastle 253
• castles 159
• coma 150
• flags 110
• sandcastles 103
• poles 94
• hash 71
• newpages 69
• buffygirl 60

Spoiler:

Full list: http://kieryn.com/xkcd-time/data/top5000timeNouns.txt

I got this list by filtering on nouns that are not in this top list of common words ftp://ftp.itri.bton.ac.uk/bnc/lemma.num and not in the top 5000 result of the same process done on the General>INTRO THREAD THE THIRD thread. This removed any common xkcd forums words and so really distilled it to the Time words. If anyone's interested the General forum word list is here: http://kieryn.com/xkcd-time/data/genCommonNouns.txt

The raw data from this parsing is available for anyone to use here: http://kieryn.com/xkcd-time/data/forumData.zip.

This zip is a single file in JSON format that was serialized using a C# DataContractJsonSerializer -- so sorry if there are crappy characters in there.
For anyone that uses C#, then the data should deserialize perfectly into the following classes:

Code: Select all

public class ForumData
    {
        public List<Post> Posts { get; set; }
        public List<TaggedToken> TaggedTokens { get; set; }
    }
    public class Post
    {
        public int PostId { get; set; }
        public string User { get; set; }
        public DateTime PostDate { get; set; }
        public string ContentText { get; set; }
    }

    //Represents individual words
    public class TaggedToken
    {
        public int PostId { get; set; }  //Original post
        public string Token { get; set; }  //Word or symbol
        public string Tag { get; set; }  //Part of speech as identified by OpenNLP parsing
        public string TagDescription { get; set; }
    }


To deserialize:

Code: Select all

            var ser = new DataContractJsonSerializer(typeof(ForumData));
            using (var file = File.OpenRead(@"C:\forumData.json"))
            {
                var dataContext = ser.ReadObject(file) as ForumData;
                //do stuff...
            }


I'm surprised buffygirl beat Blitzgirl.

Yeah, you are right. Apparently I only got 6 instances where she was tagged as a noun. The NLP must not be that great. I need to recheck this -- thanks for pointing it out. Back later

Aha. Bug in my code meant I was only looking at lower case instances of nouns. No mixed case. I will recalculate and report shortly.

[tman2nd]

Time won't auto-update for me anymore. Does anyone have an idea why?

[Angelastic]

I'm surprised buffygirl beat Blitzgirl.

Yeah, you are right. Apparently I only got 6 instances where she was tagged as a noun. The NLP must not be that great. I need to recheck this -- thanks for pointing it out. Back later

Does it treat proper nouns differently? If so, it probably assumes BlitzGirl is a proper noun since it begins with a capital letter. That would also explain the lack of Megan and Cueball in both lists.

[tman2nd]

I'm surprised buffygirl beat Blitzgirl.

Yeah, you are right. Apparently I only got 6 instances where she was tagged as a noun. The NLP must not be that great. I need to recheck this -- thanks for pointing it out. Back later

Does it treat proper nouns differently? If so, it probably assumes BlitzGirl is a proper noun since it begins with a capital letter. That would also explain the lack of Megan and Cueball in both lists.

Megan and Cueball are on this list, so they don't count.

[cmyk]

OK, so I had some free time today...

Basically I just re-parsed the entire thread (up to page 370 ish) using an XML parser (this site uses xhtml, so that helps) and also a natural language processing library (OpenNLP).
I also excluded any blockquotes from the post content, so it's all just original text, nothing duplicated.

As an example use of the data, I was able to distill the top nouns unique to this thread.

• newpix 734
• sandcastle 253
• castles 159
• coma 150
• flags 110
• sandcastles 103
• poles 94
• hash 71
• newpages 69
• buffygirl 60

Spoiler:

Full list: http://kieryn.com/xkcd-time/data/top5000timeNouns.txt

I got this list by filtering on nouns that are not in this top list of common words ftp://ftp.itri.bton.ac.uk/bnc/lemma.num and not in the top 5000 result of the same process done on the General>INTRO THREAD THE THIRD thread. This removed any common xkcd forums words and so really distilled it to the Time words. If anyone's interested the General forum word list is here: http://kieryn.com/xkcd-time/data/genCommonNouns.txt

The raw data from this parsing is available for anyone to use here: http://kieryn.com/xkcd-time/data/forumData.zip.

This zip is a single file in JSON format that was serialized using a C# DataContractJsonSerializer -- so sorry if there are crappy characters in there.
For anyone that uses C#, then the data should deserialize perfectly into the following classes:

Code: Select all

public class ForumData
    {
        public List<Post> Posts { get; set; }
        public List<TaggedToken> TaggedTokens { get; set; }
    }
    public class Post
    {
        public int PostId { get; set; }
        public string User { get; set; }
        public DateTime PostDate { get; set; }
        public string ContentText { get; set; }
    }

    //Represents individual words
    public class TaggedToken
    {
        public int PostId { get; set; }  //Original post
        public string Token { get; set; }  //Word or symbol
        public string Tag { get; set; }  //Part of speech as identified by OpenNLP parsing
        public string TagDescription { get; set; }
    }


To deserialize:

Code: Select all

            var ser = new DataContractJsonSerializer(typeof(ForumData));
            using (var file = File.OpenRead(@"C:\forumData.json"))
            {
                var dataContext = ser.ReadObject(file) as ForumData;
                //do stuff...
            }


Shocked to not see "trebuchet" in the top five.

I made a new feature- multiple people on one graph.

When looking at it this way, I can see why some people would think cmyk would be Helper! Suspicious

I made a new feature- multiple people on one graph.

When looking at it this way, I can see why some people would think cmyk would be Helper! Suspicious

I agree, Very suspicious. Perhaps cmyk should be advanced as the new Pope Permanent Past Prior Present Punctual for Perpetuity (until Blitzgirl reaches the end/present/future(?) of time of course) and see if he goes MIA as well.

Well, according to this graph, cmyk and Helper's posting habits don't line up, so I don't think they're the same.

According to that graph, cmyk never comas. cmyk is the robot that Helper's consciousness was transferred into.

Of course, Helper isn't as nearly as handsome as me, either.

Pope of Perpetuity sounds about right, as I was on a work bender for the last couple weeks*, pulling 24 to 36 hrs between (painfully short) comas. Of course this thread helped with productivity.

*any CG/VFX/Game Devs and Artists... Can I get a holla?!


ETA: also, as seriously fun and cool as that graph is, it kinda makes me sad. A sleepy cmyk is a cranky cmyk!

[Kieryn]

I'm surprised buffygirl beat Blitzgirl.

Yeah, you are right. Apparently I only got 6 instances where she was tagged as a noun. The NLP must not be that great. I need to recheck this -- thanks for pointing it out. Back later

Does it treat proper nouns differently? If so, it probably assumes BlitzGirl is a proper noun since it begins with a capital letter. That would also explain the lack of Megan and Cueball in both lists.

Megan and Cueball are on this list, so they don't count.

Proper nouns are tagged separately yes... though I included all nouns.
Anyway, bug fixed -- here is the corrected list:

newpix 990
cueball 683
frames 478
blitzgirl 324
sandcastle 294
temporal 185
castles 182
coma 167
timewaiters 136
sandcastles 116

http://kieryn.com/xkcd-time/data/top5000timeNouns.txt

[Angelastic]

Megan and Cueball are on this list, so they don't count.

Where? I did search for them there as well, but either they're not there or my text search is somehow broken.

Ah well, looks like Kieryn found the bug.

[Angelastic]

Anyway, bug fixed -- here is the corrected list:
http://kieryn.com/xkcd-time/data/top5000timeNouns.txt

…
longpix 53
scaffolding 51
newpic 52
reckoning 51
outsiders 50
semen 51
…

Did you use a half-hearted merge sort?

[buffygirl]

Spoiler:

OK, so I had some free time today...

Basically I just re-parsed the entire thread (up to page 370 ish) using an XML parser (this site uses xhtml, so that helps) and also a natural language processing library (OpenNLP).
I also excluded any blockquotes from the post content, so it's all just original text, nothing duplicated.
As an example use of the data, I was able to distill the top nouns unique to this thread.

• newpix 734
• sandcastle 253
• castles 159
• coma 150
• flags 110
• sandcastles 103
• poles 94
• hash 71
• newpages 69
• buffygirl 60

Spoiler:

Full list: http://kieryn.com/xkcd-time/data/top5000timeNouns.txt

I got this list by filtering on nouns that are not in this top list of common words ftp://ftp.itri.bton.ac.uk/bnc/lemma.num and not in the top 5000 result of the same process done on the General>INTRO THREAD THE THIRD thread. This removed any common xkcd forums words and so really distilled it to the Time words. If anyone's interested the General forum word list is here: http://kieryn.com/xkcd-time/data/genCommonNouns.txt

The raw data from this parsing is available for anyone to use here: http://kieryn.com/xkcd-time/data/forumData.zip.

This zip is a single file in JSON format that was serialized using a C# DataContractJsonSerializer -- so sorry if there are crappy characters in there.
For anyone that uses C#, then the data should deserialize perfectly into the following classes:

Code: Select all

public class ForumData
    {
        public List<Post> Posts { get; set; }
        public List<TaggedToken> TaggedTokens { get; set; }
    }
    public class Post
    {
        public int PostId { get; set; }
        public string User { get; set; }
        public DateTime PostDate { get; set; }
        public string ContentText { get; set; }
    }

    //Represents individual words
    public class TaggedToken
    {
        public int PostId { get; set; }  //Original post
        public string Token { get; set; }  //Word or symbol
        public string Tag { get; set; }  //Part of speech as identified by OpenNLP parsing
        public string TagDescription { get; set; }
    }


To deserialize:

Code: Select all

            var ser = new DataContractJsonSerializer(typeof(ForumData));
            using (var file = File.OpenRead(@"C:\forumData.json"))
            {
                var dataContext = ser.ReadObject(file) as ForumData;
                //do stuff...
            }


Ooooh! I'm unique to this thread!!! manvandmaan, you are added to my hat queue. Which I fulfill in no particular order.

EDIT1: Hmmmm, shortlived uniqueness...
EDIT2: spoilered list

[htom]

It's an Isabella Blow hat.

Spoiler:

IsabellaBlow PhilipTreacy hat-crop.jpg

She designs some very unique...
whatever those things are. They're not really all hats, some of them are closer to masks. I'd be a little scared to meet that person.

I'm with HAL on this ... that's not a hat; I have no idea what it is, but certainly won't keep the rain off, which is, I would have thought, one of the primary attributes required of a hat.

I think it's more a fascinater than a hat. (At first I thought it was a pair of partially eaten waffles.)

[buffygirl]

It's an Isabella Blow hat.

Spoiler:

IsabellaBlow PhilipTreacy hat-crop.jpg

She designs some very unique...
whatever those things are. They're not really all hats, some of them are closer to masks. I'd be a little scared to meet that person.

I'm with HAL on this ... that's not a hat; I have no idea what it is, but certainly won't keep the rain off, which is, I would have thought, one of the primary attributes required of a hat.

I think it's more a fascinater than a hat. (At first I thought it was a pair of partially eaten waffles.)

Right. It's been replaced. I had feared it might be too detailed. I wanted to use it cuz it sorta looked like a fancy sandcastle.

[Kieryn]

Anyway, bug fixed -- here is the corrected list:
http://kieryn.com/xkcd-time/data/top5000timeNouns.txt

…
longpix 53
scaffolding 51
newpic 52
reckoning 51
outsiders 50
semen 51
…

Did you use a half-hearted merge sort?

Ha

Kind of -- I actually added a bump up for longer words:

Code: Select all

.OrderByDescending(c=>c.count*4 + Math.Min(c.noun.Length, 20))

[mojacardave]

Firstly, I would like to request one of those nifty graphs...

Your wish is my command and all that (do you not have Excel though? The tool is dead easy to use!)

[Caswallon]

Wow, there has been a lot of actuall analysis in this thread since my coma, I rather enjoy it

[mscha]

SUPERSTRONG...

-- posted by newpixbot

[ggh]

Pope of Perpetutuity sounds about right, as I was on a work bender for the last couple weeks*, pulling 24 to 36 hrs between (painfully short) comas. Of course this thread helped with productivity.

*any CG/VFX/Game Devs and Artists... Can I get a holla?!


ETA: also, as seriously fun and cool as that graph is, it kinda makes me sad. A sleepy cmyk is a cranky cmyk!

Holla, Holla! I was wondering what your gig was - you sure have the tools and you know how to use them.
I winced when the request was made for more blur, and wowed by your speedy turnaround. How'd you do that?

[cmyk]

SUPERSTRONG...

-- posted by newpixbot

I'm beginning to wonder if it's a scale model, she's drafting with sand, as a template for what they will eventually build at 1:1 for the structure they've semi-framed out. Perhaps using materials stronger than sand — or IKEA particle boards, even.

[Kieryn]

Okay playing with highcharts... http://jsfiddle.net/fgxda/20/embedded/result/

Gonna go do some required 'outside' stuff for a bit. Hopefully soon I can create a dynamic /searchable version of this, kind of like this baby names thing:
http://www.babynamewizard.com/voyager

[cmyk]

Pope of Perpetutuity sounds about right, as I was on a work bender for the last couple weeks*, pulling 24 to 36 hrs between (painfully short) comas. Of course this thread helped with productivity.

*any CG/VFX/Game Devs and Artists... Can I get a holla?!


ETA: also, as seriously fun and cool as that graph is, it kinda makes me sad. A sleepy cmyk is a cranky cmyk!

Holla, Holla! I was wondering what your gig was - you sure have the tools and you know how to use them.
I winced when the request was made for more blur, and wowed by your speedy turnaround. How'd you do that?

(holla!)

Actually, at the moment, I couldn't find an easy way to download all the frames from any particular source and import it into After Effects as an image sequence. The one source I did find, the file names were all semi-hash tags, and the dates were all the same, so there was no way to easily sort them (that I could see) as an ordered sequence of frames. So, I took the lazy route and screen recorded the sequence from one of the scripted player websites.

After I captured that, I had to mess with the fps/playback rates a bit, then just applied motion blur using a post-process plugin called Reel Smart Motion Blur. Handy tool for faking such an effect.

[NoMouse]

SUPERSTRONG...

-- posted by newpixbot

This is really starting to look like the castle from Disney logo. The bridge is even at the right position. I know, it's not perfect. But it's the closest one so far.

[Kieryn]

Pope of Perpetutuity sounds about right, as I was on a work bender for the last couple weeks*, pulling 24 to 36 hrs between (painfully short) comas. Of course this thread helped with productivity.

*any CG/VFX/Game Devs and Artists... Can I get a holla?!


ETA: also, as seriously fun and cool as that graph is, it kinda makes me sad. A sleepy cmyk is a cranky cmyk!

Holla, Holla! I was wondering what your gig was - you sure have the tools and you know how to use them.
I winced when the request was made for more blur, and wowed by your speedy turnaround. How'd you do that?

(holla!)

Actually, at the moment, I couldn't find an easy way to download all the frames from any particular source and import it into After Effects as an image sequence. The one source I did find, the file names were all semi-hash tags, and the dates were all the same, so there was no way to easily sort them (that I could see) as an ordered sequence of frames. So, I took the lazy route and screen recorded the sequence from one of the scripted player websites.

After I captured that, I had to mess with the fps/playback rates a bit, then just applied motion blur using a post-process plugin called Reel Smart Motion Blur. Handy tool for faking such an effect.

Here's the way I get the frames:
1) Using chrome, go to http://geekwagon.net/projects/xkcd1190/.
2) In the JS console type:

Code: Select all

$(images).each(function(){console.log(this.toString());});

This outputs all the URLs to the console
3) Copy and paste/save the URLs then I wrote a program that loops through them and saves each image numbered 1..n

If you use C# the code is as simple as:

Code: Select all

           
            var client = new System.Net.WebClient();
            var i = 0;
            foreach (var url in File.ReadAllLines(@"C:\imgUrls.txt")) {
                client.DownloadFile(url, @"C:\frames\" + i + ".png");
                i++;
            }


[ggh]

Pope of Perpetutuity sounds about right, as I was on a work bender for the last couple weeks*, pulling 24 to 36 hrs between (painfully short) comas. Of course this thread helped with productivity.

*any CG/VFX/Game Devs and Artists... Can I get a holla?!


ETA: also, as seriously fun and cool as that graph is, it kinda makes me sad. A sleepy cmyk is a cranky cmyk!

Holla, Holla! I was wondering what your gig was - you sure have the tools and you know how to use them.
I winced when the request was made for more blur, and wowed by your speedy turnaround. How'd you do that?

(holla!)

Actually, at the moment, I couldn't find an easy way to download all the frames from any particular source and import it into After Effects as an image sequence. The one source I did find, the file names were all semi-hash tags, and the dates were all the same, so there was no way to easily sort them (that I could see) as an ordered sequence of frames. So, I took the lazy route and screen recorded the sequence from one of the scripted player websites.

After I captured that, I had to mess with the fps/playback rates a bit, then just applied motion blur using a post-process plugin called Reel Smart Motion Blur. Handy tool for faking such an effect.

Wow, I remember hearing of that tool, but never used it. It does a real nice job, or rather, you did a real nice job using it.

[cmyk]

I did some further image analysis to begin to rule out out possible correlations with other meaningful mathematic or scientific representations.

Here's the famous Mandelbrot fractal:








and here's Oprah Winfrey:






...my inductive research continues...

[cmyk]

Pope of Perpetutuity sounds about right, as I was on a work bender for the last couple weeks*, pulling 24 to 36 hrs between (painfully short) comas. Of course this thread helped with productivity.

*any CG/VFX/Game Devs and Artists... Can I get a holla?!


ETA: also, as seriously fun and cool as that graph is, it kinda makes me sad. A sleepy cmyk is a cranky cmyk!

Holla, Holla! I was wondering what your gig was - you sure have the tools and you know how to use them.
I winced when the request was made for more blur, and wowed by your speedy turnaround. How'd you do that?

(holla!)

Actually, at the moment, I couldn't find an easy way to download all the frames from any particular source and import it into After Effects as an image sequence. The one source I did find, the file names were all semi-hash tags, and the dates were all the same, so there was no way to easily sort them (that I could see) as an ordered sequence of frames. So, I took the lazy route and screen recorded the sequence from one of the scripted player websites.

After I captured that, I had to mess with the fps/playback rates a bit, then just applied motion blur using a post-process plugin called Reel Smart Motion Blur. Handy tool for faking such an effect.

Wow, I remember hearing of that tool, but never used it. It does a real nice job, or rather, you did a real nice job using it.

Yep, it's a great tool. The default setting works pretty great 90% of the time (though for post-MB in CG, it can use motion vector data from your scene, which helps it determine proper vectors for the blurring). I just doubled the the strength for this, and turned it off or down on the dialog(ue) frames. Easy peasy.

[HAL9000]

and here's Oprah Winfrey:

Made my day.

[histrion]

SUPERSTRONG...

-- posted by newpixbot

This is really starting to look like the castle from Disney logo. The bridge is even at the right position. I know, it's not perfect. But it's the closest one so far.

Yeah, or maybe Neuschwanstein? Or Hogwarts?

[buffygirl]

Okay playing with highcharts... http://jsfiddle.net/fgxda/20/embedded/result/

Gonna go do some required 'outside' stuff for a bit. Hopefully soon I can create a dynamic /searchable version of this, kind of like this baby names thing:
http://www.babynamewizard.com/voyager

I am loving all this masturbat- enlightening chartmaking!

[mscha]

TRIPHTHONG...

-- posted by newpixbot

[cmyk]

and here's Oprah Winfrey:

Made my day.

She is a light unto us all...

[cmyk]

Spoiler:

SUPERSTRONG...

-- posted by newpixbot

This is really starting to look like the castle from Disney logo. The bridge is even at the right position. I know, it's not perfect. But it's the closest one so far.

Yeah, or maybe Neuschwanstein? Or Hogwarts?

I dunno... sorta looking like Megatron.

[Valarya]

Anyway, bug fixed -- here is the corrected list:

newpix 990
cueball 683
frames 478
blitzgirl 324
sandcastle 294
temporal 185
castles 182
coma 167
timewaiters 136
sandcastles 116

Thaaaaat's more like it!

and here's Oprah Winfrey:

Made my day.

I had to stop scrolling I was laughing so hard.

[cmyk]

Anyone else think rot13 sort of looks like Cthulhu's native R'lyehian language? Kinda freakin' me out a bit.

[KarMann]

Anyone else think rot13 sort of looks like Cthulhu's native R'lyehian language? Kinda freakin' me out a bit.

It may have crossed my mind.

[cmyk]

Anyone else think rot13 sort of looks like Cthulhu's native R'lyehian language? Kinda freakin' me out a bit.

It may have crossed my mind.

See? That's what I get for skimming.

*shame*

[HAL9000]

Spoiler:

SUPERSTRONG...

-- posted by newpixbot

This is really starting to look like the castle from Disney logo. The bridge is even at the right position. I know, it's not perfect. But it's the closest one so far.

Yeah, or maybe Neuschwanstein? Or Hogwarts?

I dunno... sorta looking like Megatron.

Which Megatron? There's actually been rather a few iterations, all of them rather different.
This is what the original Megatron toy transformed into:
http://upload.wikimedia.org/wikipedia/en/5/55/Megatron-g1toy.jpg

[mscha]

LEGONG...

-- posted by newpixbot

[SBN]

Today, on the outside I tamed the savage beasts¹, beat back the encroaching jungle², did battle with a powerful foe³, and went on a mighty quest⁴. Then I hurried back to wait for it.

¹ Took my Newfoundlands to obedience class.
² Mowed the lawn.
³ Called my satellite provider to get a new receiver, to replace the one that has failed.
⁴ Took a nice walk around the park.

[cmyk]

The only Megatron is the original Megatron that somehow defied known material physics and transformed into a gun a fraction of his Decepticon-mode size and mass.

[jovialbard]

The only Megatron is the original Megatron that somehow defied known material physics and transformed into a gun a fraction of his Decepticon-mode size and mass.

Maybe he was made out of some kind of foam that compressed? A foam that looked and behaved almost exactly like metal... I'll just call it astro-foam and be done with it.