I have a project for a REAL programmer...

A place to discuss the implementation and style of computer programs.

Moderators: phlip, Moderators General, Prelates

User avatar
josiahstevenson
Posts: 36
Joined: Sat Dec 27, 2008 9:24 pm UTC
Location: Austin, TX
Contact:

I have a project for a REAL programmer...

Postby josiahstevenson » Mon Apr 20, 2009 11:24 pm UTC

I'd like to see someone write [read: I would buy] a program to help students register for classes. Such a program should be able to:

*optimize the student's class schedule given a prioritized list of preferences in any order the student chooses including
-what (e.g. CHEM 201, MATH 308)
-when (e.g no classes on friday or before 10 am)
-where (e.g as close as possible to my dorm or as close as possible to the location of an immediately preceding class, near the dining hall or Taco Bell)
-who (professor preference)
*log onto the university website and register for the courses the student wants the minute registration opens. Like an ebay sniper.
*Ideally, it should be able to retrieve a course/section list automatically and be able to recognize from the section description where the class is being held
*It also needs to allow for on-the-fly revision if a section it selects is full (of honors kids or grad students) by the time it registers.

I'd also appreciate any direction regarding where to start doing this myself (if noone else wants to do it, that is), or any referrals to existing programs that do this.

User avatar
quintopia
Posts: 2906
Joined: Fri Nov 17, 2006 2:53 am UTC
Location: atlanta, ga

Re: I have a project for a REAL programmer...

Postby quintopia » Tue Apr 21, 2009 12:16 am UTC

There are not and cannot be any existing programs that do this. In particular, the part about logging on and registering for you is highly interface-dependent, and every school uses a different web face for that function.

User avatar
Durandal
Posts: 659
Joined: Thu Nov 08, 2007 12:12 am UTC

Re: I have a project for a REAL programmer...

Postby Durandal » Tue Apr 21, 2009 12:18 am UTC


User avatar
mrbaggins
Posts: 1611
Joined: Tue Jan 15, 2008 3:23 am UTC
Location: Wagga, Australia

Re: I have a project for a REAL programmer...

Postby mrbaggins » Tue Apr 21, 2009 12:51 am UTC

If it's a web interface, you could possibly get a Greasemonkey script to at least HELP with it.
Why is it that 4chan is either infinitely awesome, infinitely bad, or "lolwut", but never any intermediary level?

elminster
Posts: 1560
Joined: Mon Feb 26, 2007 1:56 pm UTC
Location: London, UK, Dimensions 1 to 42.
Contact:

Re: I have a project for a REAL programmer...

Postby elminster » Tue Apr 21, 2009 3:14 am UTC

This isn't really a hard thing to do (Dependant on how complex the preferences are) but, as quintopia mentioned, there isn't a piece of software that could do this for your particular case.

I've made several autoplayers for html based games (Very similar kind of coding problem) and it takes relatively little skill to make something that does the job; granted they were done using .NET components, which is about half the work already done.

Also, they obviously won't provide an API, so parsing the info from the html is the only option; that means you'd have to know the format and relevant contents of all the web pages at each stage or else the program can only be made to assume it's doing it right. For example, while looking at a table of courses and related information, it may seem obvious what's related, but the html tags could easily have no indication of what data is what or how it's related to each other.

In short, if you don't have access to what the pages will be like (The html source), it would be either impossible or the solution would be very unreliable.
Image

User avatar
thedufer
Posts: 263
Joined: Mon Aug 06, 2007 2:11 am UTC
Location: Northern VA (not to be confused with VA)
Contact:

Re: I have a project for a REAL programmer...

Postby thedufer » Tue Apr 21, 2009 4:38 am UTC

schedulizer.com does this, but only works for a few schools so far, and they seem unwilling to add more. Send them an email about your school, though. Who knows, maybe you'll get it?

User avatar
ConcatenatedWords
Posts: 17
Joined: Thu May 29, 2008 9:41 pm UTC

Re: I have a project for a REAL programmer...

Postby ConcatenatedWords » Tue Apr 21, 2009 5:49 am UTC

Aren't optimizing the class schedule and scraping the course data from the website separate problems? Perhaps the interface-dependent stuff could be abstracted in a class interface. There could be an abstract base class that defines the interface, base classes that implement the common html parsing code for various registration software (such as WebAdvisor), and instances representing individual schools' sites.
"I hereby endorse this product and/or service. I'm not a paid actor! I'm a real consumer just like you!"

User avatar
cerbie
Posts: 934
Joined: Sat Jul 05, 2008 5:14 am UTC
Location: USA

I'm with Elminster on this

Postby cerbie » Tue Apr 21, 2009 7:53 am UTC

josiahstevenson wrote:*log onto the university website and register for the courses the student wants the minute registration opens. Like an ebay sniper.
*Ideally, it should be able to retrieve a course/section list automatically and be able to recognize from the section description where the class is being held

I'd also appreciate any direction regarding where to start doing this myself (if noone else wants to do it, that is), or any referrals to existing programs that do this.
Everything else shouldn't be too hard--mostly initial design, and taking time to test everything. A nice little RDB to keep up with all data (in terms of managing the data, this is begging for a proper relational DB--generally, SQL of some kind beaten into submission), and some program to grab needed pieces of that data and work on them, would do all the rest of the work. You can do the optimizations different ways, get efficient means of working it, slow means of working it, etc., but that wouldn't be a huge deal--it should be pretty parallel, allowing you to throw hardware at it (I would think the hard limits would be how fast the school web servers respond to registration requests).

However, gathering info from third-party web sources--accurately--properly linking it to other data (map coordinates, FI), and doing the actual logging in and registering? Ew. Those are some tricky requirements.

ConcatenatedWords wrote:Aren't optimizing the class schedule and scraping the course data from the website separate problems?
It looks that way to me, too.
Perhaps the interface-dependent stuff could be abstracted in a class interface. There could be an abstract base class that defines the interface, base classes that implement the common html parsing code for various registration software (such as WebAdvisor), and instances representing individual schools' sites.
Thinking about classes and such right off seems like jumping the gun a bit. As long as you can tell it, "Log in. Good? OK, try registering for this/these course/courses. Good? OK, log out," and also fail over nicely at every step, then it's abstracted well enough. OOP may be good for various reasons, or it may not be. Properly defining how you can use the systems as a non-human stream processor or tree walker is what's really important.

Regardless of implementation, the human time taken doing the above, and verifying that data retrieved is treated right, and that you can use that data properly for customers, is where it starts looking iffy, IMO (how do you have confidence in it if you can't actually test it live? How convoluted are the UIs in these systems? How many man-hours are going to be needed to verify courses in the system are all correct? Will the schools have anything to say about it if they find out? How do you test it without them finding out, anyway?).

P.S. to grammar nazis: I'm feeling lazy.
DSenette: (...) on the whole, even a trained killer cow is kind of stupid.

User avatar
thedufer
Posts: 263
Joined: Mon Aug 06, 2007 2:11 am UTC
Location: Northern VA (not to be confused with VA)
Contact:

Re: I have a project for a REAL programmer...

Postby thedufer » Tue Apr 21, 2009 1:54 pm UTC

You want to be careful loading large amounts of data through the school's web interface. Experience says that they'll respond to this as if it is what it looks like - a DOS attack.

0xBADFEED
Posts: 687
Joined: Mon May 05, 2008 2:14 am UTC

Re: I have a project for a REAL programmer...

Postby 0xBADFEED » Tue Apr 21, 2009 2:19 pm UTC

Entering in all your preferences for professors, locations, times, etc. would probably take a comparable amount of time to registering for classes.

Seriously, how long does it take you? I never spent more than an hour or two registering for semester classes. So I mean that's what, 16 hours of work over 4 years. The kind of effort you're talking about for saving 4 hours of work a year just isn't feasible. You could easily spend months (maybe years depending on features) writing a system like this.

And you'll still be left with a brittle system because you're outside the school's system operating without an API so any UI changes on the web-side will likely break your scraper.

A system like this would be useful (and probably profitable). But I can't really see it succeeding without cooperation from the school. And it's much too much work to do as a one-off project.

You could easily write a solver that would at least work out what combinations of classes are viable given time constraints and professor preference. But you would still have to do the actual registration yourself.

User avatar
cerbie
Posts: 934
Joined: Sat Jul 05, 2008 5:14 am UTC
Location: USA

Re: I have a project for a REAL programmer...

Postby cerbie » Tue Apr 21, 2009 11:47 pm UTC

0xBADFEED wrote:(...)
And you'll still be left with a brittle system because you're outside the school's system operating without an API so any UI changes on the web-side will likely break your scraper.
Not to mention: something odd happens, then a tech support guy/admin at the school looks at the logs, then he sees something else odd with your IP(s)...that will end well. If I were a management guy at said school, I wouldn't be treating the students who used it all that nicely.

A system like this would be useful (and probably profitable). But I can't really see it succeeding without cooperation from the school. And it's much too much work to do as a one-off project.

You could easily write a solver that would at least work out what combinations of classes are viable given time constraints and professor preference. But you would still have to do the actual registration yourself.
QFT
DSenette: (...) on the whole, even a trained killer cow is kind of stupid.

elminster
Posts: 1560
Joined: Mon Feb 26, 2007 1:56 pm UTC
Location: London, UK, Dimensions 1 to 42.
Contact:

Re: I have a project for a REAL programmer...

Postby elminster » Wed Apr 22, 2009 1:08 am UTC

Something like this is hardly going to put a strain on their web servers. Even if you were to download a page every 2 seconds, which would be fine for this kind of thing, it's barely going to do anything to the webserver (Assuming it's dedicated and at least reasonable specs). Some prefetchers would put more load on the web servers (Some websites make sure to disallow them).

Judging by the emphasis on speed, I assumed the goal of it was to register classes before anyone else could. So saving on time overall wouldn't really matter if that was the goal.

If you did write something to do this, make sure you put in checks to make sure the relevant content is still intact before and while attempting it. As mentioned, getting info straight form the site can very easily be broken.
I know while making an autoplayer for a game, I noticed they set honeypots to catch any scripts (They occasionally changed a form input names by 1 letter) and constant updates would break the system anyway, so checking was a must.

A part compromise is to automate the simple parts and make the user check things at every stage to make sure an unexpected error hasn't occurred.
Image

User avatar
eternauta3k
Posts: 519
Joined: Thu May 10, 2007 12:19 am UTC
Location: Buenos Aires, Argentina

Re: I have a project for a REAL programmer...

Postby eternauta3k » Wed Apr 22, 2009 8:13 pm UTC

When I read "project for a REAL programmer" I expected something that requires intimate knowledge of algorithms and data structures, maybe large-scale project management. This looks like lots of rote work looking at HTML and hoping they don't change it. However, the optimization part might be interesting. This is something that should be done on the university's servers so the needs of students are met as fairly as possible.
VectorZero wrote:It takes a real man to impact his own radius

That's right, slash your emo-wrists and spill all your emo-globin

User avatar
josiahstevenson
Posts: 36
Joined: Sat Dec 27, 2008 9:24 pm UTC
Location: Austin, TX
Contact:

Re: I have a project for a REAL programmer...

Postby josiahstevenson » Sat Apr 25, 2009 8:35 am UTC

The most important part would be the optimization. Actually having it execute the registration automatically would be a nice touch - one that seems, given the situation, not to be worth the extra trouble.

User avatar
Emu*
Posts: 689
Joined: Mon Apr 28, 2008 9:47 am UTC
Location: Cardiff, UK
Contact:

Re: I have a project for a REAL programmer...

Postby Emu* » Sat Apr 25, 2009 8:51 am UTC

josiahstevenson wrote:*optimize the student's class schedule given a prioritized list of preferences in any order the student chooses including
-what (e.g. CHEM 201, MATH 308)
-when (e.g no classes on friday or before 10 am)
-where (e.g as close as possible to my dorm or as close as possible to the location of an immediately preceding class, near the dining hall or Taco Bell)
-who (professor preference)


This is a very complex multi-restriction problem; Almost every year, one of the Final Year Projects available at my university is a system to allocate supervisors to classes, rooms to classes or classes onto the timetable. This is for people who have already been at university for 2 years.

It's not as easy as you think.
Cosmologicon wrote:Emu* implemented a naive east-first strategy and ran it for an hour, producing results that rivaled many sophisticated strategies, visiting 614 cells. For this, Emu* is awarded Best Deterministic Algorithm!

kmatzen
Posts: 214
Joined: Thu Nov 15, 2007 2:55 pm UTC
Location: Ithaca, NY

Re: I have a project for a REAL programmer...

Postby kmatzen » Sat Apr 25, 2009 6:35 pm UTC

I use mschedule.com

Edit: It's open source so you could extend functionality. This was a senior design project at U of M a few years. It's popular enough that our online registration was reworked to export its class data to it rather than mschedule going in and scraping the database.

User avatar
josiahstevenson
Posts: 36
Joined: Sat Dec 27, 2008 9:24 pm UTC
Location: Austin, TX
Contact:

Re: I have a project for a REAL programmer...

Postby josiahstevenson » Sat Apr 25, 2009 8:49 pm UTC

Emu* wrote:
josiahstevenson wrote:*optimize the student's class schedule given a prioritized list of preferences in any order the student chooses including
-what (e.g. CHEM 201, MATH 308)
-when (e.g no classes on friday or before 10 am)
-where (e.g as close as possible to my dorm or as close as possible to the location of an immediately preceding class, near the dining hall or Taco Bell)
-who (professor preference)


This is a very complex multi-restriction problem; Almost every year, one of the Final Year Projects available at my university is a system to allocate supervisors to classes, rooms to classes or classes onto the timetable. This is for people who have already been at university for 2 years.

It's not as easy as you think.

I never said it was or would be easy; I said I had a project for a REAL programmer.

User avatar
Emu*
Posts: 689
Joined: Mon Apr 28, 2008 9:47 am UTC
Location: Cardiff, UK
Contact:

Re: I have a project for a REAL programmer...

Postby Emu* » Sun Apr 26, 2009 9:37 am UTC

I pity all the unreal kernel coders.
Cosmologicon wrote:Emu* implemented a naive east-first strategy and ran it for an hour, producing results that rivaled many sophisticated strategies, visiting 614 cells. For this, Emu* is awarded Best Deterministic Algorithm!

dosboot
Posts: 143
Joined: Sun Jul 01, 2007 5:26 am UTC

Re: I have a project for a REAL programmer...

Postby dosboot » Sun Apr 26, 2009 8:07 pm UTC

my 2 cents: for something like this maybe you should specify how much you will pay. Your request isn't something that can be quickly thrown together nor is it - I imagine - something that people will enjoy working on in their spare time.


Return to “Coding”

Who is online

Users browsing this forum: No registered users and 11 guests