Facnotify

Nachtwind · Posted: Sat Mar 04, 2006 7:02 am Post subject:

As there is no way here to see wether favorite Artist have submitted new Pictures i wrote a small programm to check if they did. For that purpose it simply parses the Profile Page of that user.

I just wanted to ask if i am allowed to program this further or if there are any objections against it (perhaps because of traffic or other things..)
_________________
Quite obviously, our problems do not come from what we invent, but from how we use our sophisticated toys. The difficulties stem not from our hardware or software, but from ourselves.
Barbarossa,
Anatomy of a Rebellion
[img]http://misc.shalmirane.de/violance.JPG[/img]

Stratadrake · Posted: Sat Mar 04, 2006 7:10 am Post subject:

How does the tool work, exactly? I mean what language did you write it in.

Automated e-mail alerts for new pictures is already being planned. (Though it would be nicer to separate it from the faves system)
_________________
Strata here: [url=http://www.nanowrimo.org/eng/user/242293]Nanowrimo[/url] - [url=www.fanart-central.net/user-Stratadrake.php]FAC[/url] - [url=http://stratadrake.deviantart.com]dA[/url] - [url=www.furaffinity.net/user/Stratadrake/]FA[/url]
[size=9]Disclaimer: Posts may contain URLs. Click [url=http://tvtropes.org/pmwiki/pmwiki.php/Main/TVTropesWillRuinYourLife]at your own risk.[/url][/size]

Nachtwind · Posted: Sat Mar 04, 2006 7:36 am Post subject:

Well.. to be more precisly:

Its quite easy. As you have some nice URL shortcuts to the users profile i use this to get a page of the user.
There is this Number of Pictures table cell. Only read this part of the File, delete every text with the help of a simple RegEx function and you have the number of pictures as well as numbers of pending ones. The next step is just checking these numbers against numbers already known to the system and saved in a local file and if theres a change simply tell it to the user.

To be more technical:
The program is written in C# (Net2.0) and uses HTTP Get to catch the page of the user and thats why i am asking. These pages are quite big so theres some amount of traffic when fetching.

Anyhow, this program would be completly free of any FavList here (but could be implemented though) and tells the user if there are new pics of a certain artist or not.

_________________
Quite obviously, our problems do not come from what we invent, but from how we use our sophisticated toys. The difficulties stem not from our hardware or software, but from ourselves.
Barbarossa,
Anatomy of a Rebellion
[img]http://misc.shalmirane.de/violance.JPG[/img]

Nachtwind · Posted: Sat Mar 04, 2006 8:29 am Post subject:

Got a screenshot and the program is done so far.

[img]http://shalmirane.de/FACN.jpg[/img]
_________________
Quite obviously, our problems do not come from what we invent, but from how we use our sophisticated toys. The difficulties stem not from our hardware or software, but from ourselves.
Barbarossa,
Anatomy of a Rebellion
[img]http://misc.shalmirane.de/violance.JPG[/img]

Touzoku-joou · Oldbie Joined: 30 Oct 2004 Posts: 2399 Location: interweb

Wow, that's pretty nifty looking. :0
_________________
[url=http://www.fanart-central.net/user-Touzoku-Joou.php]FAC[/url]

PhunkYMunkY · Posted: Sat Mar 04, 2006 1:04 pm Post subject:

I don't think these things change quite often enough for it to have it's own basic program... A html update page on the site including an email, I believe, is enough. And I believe the html update page is in the process of being formed.

See what Denis says though.

Nachtwind · Posted: Sat Mar 11, 2006 8:14 am Post subject:

Ok. You can find the tool here: http://alphaclan.org/facn.rar

Just unrar it and run. Please be sure you have Microsoft Net2.0 (runtime or SDK) installed on your computer.
_________________
Quite obviously, our problems do not come from what we invent, but from how we use our sophisticated toys. The difficulties stem not from our hardware or software, but from ourselves.
Barbarossa,
Anatomy of a Rebellion
[img]http://misc.shalmirane.de/violance.JPG[/img]

Stratadrake · Posted: Sat Mar 11, 2006 11:53 am Post subject:

FAC cannot officially support this program, however if users want to download and use it themselves, that's their choice, their risk; no need to stop them either.
_________________
Strata here: [url=http://www.nanowrimo.org/eng/user/242293]Nanowrimo[/url] - [url=www.fanart-central.net/user-Stratadrake.php]FAC[/url] - [url=http://stratadrake.deviantart.com]dA[/url] - [url=www.furaffinity.net/user/Stratadrake/]FA[/url]
[size=9]Disclaimer: Posts may contain URLs. Click [url=http://tvtropes.org/pmwiki/pmwiki.php/Main/TVTropesWillRuinYourLife]at your own risk.[/url][/size]

Nachtwind · Posted: Sat Mar 11, 2006 1:42 pm Post subject:

ok thank you. My only question was if you diallow the program. If you dont i will keep it online.
_________________
Quite obviously, our problems do not come from what we invent, but from how we use our sophisticated toys. The difficulties stem not from our hardware or software, but from ourselves.
Barbarossa,
Anatomy of a Rebellion
[img]http://misc.shalmirane.de/violance.JPG[/img]

Stratadrake · Posted: Sat Mar 11, 2006 5:41 pm Post subject:

From what I can tell at a glance, the program is basically just an automated version of checking said artist(s)'s profile page yourself. Along those lines, there's no logical reason we [i]can't[/i] allow it, I'm only saying we can't officially give support (i.e. help) for it.

Here's a challenge: Can you add an ability to watch specific categories?

As a fellow programmer, you could outline it like this:

1- User copy/pastes the URL for an FAC category into the utility.
2- Utility identifies/caches the category being viewed (extract numeric ID from URL: /cat-##.php" or "/category.php?parent=## ...", and extract name from the page data)
3- Utility identifies/caches most recent pic (or few pics) in category (use URL: "category.php?parent=##&sort=date%20desc")
4- When user starts up utility again, it fetches most recent pic(s) from category (same as #3) and compares to cached list to determine if there are new pics in the category.
_________________
Strata here: [url=http://www.nanowrimo.org/eng/user/242293]Nanowrimo[/url] - [url=www.fanart-central.net/user-Stratadrake.php]FAC[/url] - [url=http://stratadrake.deviantart.com]dA[/url] - [url=www.furaffinity.net/user/Stratadrake/]FA[/url]
[size=9]Disclaimer: Posts may contain URLs. Click [url=http://tvtropes.org/pmwiki/pmwiki.php/Main/TVTropesWillRuinYourLife]at your own risk.[/url][/size]

Nachtwind · Posted: Sun Mar 12, 2006 10:11 am Post subject:

Its not a matter IF i can do it but how much time i get for it :P

Of course its possible.. should be imho not even a problem at all.
The only "problem" is to find out which number corresponds to a certain category. If i had this list i could do it even better... this way i can only do a number only query. So its nothing great. Still no problem at all...
I will take me some time to try to get some kind of "automatic" category lister... for this would be great (Hey, what about a small help? in form or:

<?
$arr = DATABASEFETCH(Categorys);
foreach ($arr as $index => $value) {
echo $value."\n";
}
?>

;)
_________________
Quite obviously, our problems do not come from what we invent, but from how we use our sophisticated toys. The difficulties stem not from our hardware or software, but from ourselves.
Barbarossa,
Anatomy of a Rebellion
[img]http://misc.shalmirane.de/violance.JPG[/img]

Nachtwind · Posted: Sun Mar 12, 2006 10:36 am Post subject:

oh i see... i can strip the name from the TITLE tag of the page... this makes it easier ^^
_________________
Quite obviously, our problems do not come from what we invent, but from how we use our sophisticated toys. The difficulties stem not from our hardware or software, but from ourselves.
Barbarossa,
Anatomy of a Rebellion
[img]http://misc.shalmirane.de/violance.JPG[/img]

Nachtwind · Posted: Sun Mar 12, 2006 11:27 am Post subject:

Ok. just getting myself a decent database of all categories ^^
_________________
Quite obviously, our problems do not come from what we invent, but from how we use our sophisticated toys. The difficulties stem not from our hardware or software, but from ourselves.
Barbarossa,
Anatomy of a Rebellion
[img]http://misc.shalmirane.de/violance.JPG[/img]

cstdenis · Posted: Sun Mar 12, 2006 1:18 pm Post subject:

I wont disallow it, but I do discourage use of it for those who have large numbers of favorites. The profile pages take a fair amount of processing power to display and some people have hundreds of favorite artists.

I could setup an RSS or webservice for this when I have time, but by then the new changes to fac will be up which may include this kind of functionality.
_________________
You will obey or molten silver will be poured into your ears.

Nachtwind · Posted: Sun Mar 12, 2006 1:38 pm Post subject:

some XML backand would be cool ;0)

But till then i write this. And see, the list here is not the same you have on FAC so it might be a smaller list for a user than on the page.
_________________
Quite obviously, our problems do not come from what we invent, but from how we use our sophisticated toys. The difficulties stem not from our hardware or software, but from ourselves.
Barbarossa,
Anatomy of a Rebellion
[img]http://misc.shalmirane.de/violance.JPG[/img]

cstdenis · Posted: Sun Mar 12, 2006 1:56 pm Post subject:

Ah, I see.

Thought you were scraping the user's profile to get their favorites.

you should have it save the number of pictures so you can calculate number of new pictures since last run.

Do you know php programming?
_________________
You will obey or molten silver will be poured into your ears.

Ogrim_Doomhammer · Oldbie Joined: 06 Dec 2004 Posts: 2116

Denis, I think you have got yourself the new FAC programmer Smile

_________________
[url=http://www.fanart-central.net/user-Ogrim_Doomhammer.php][img]http://i19.photobucket.com/albums/b167/Ogrim_Doomhammer/lennonbannercopy.jpg[/img][/url]

www.pandora.com Find New Music You Like.... Free

Nachtwind · Posted: Sun Mar 12, 2006 4:32 pm Post subject:

I know PHP quite good (got almost my mother language in the last years ;0))
So i can think of what you mean by the page is using many ressources.
Anyhow. Of course i am saving some values locally (SQLite DataBase). Any other way wouldnt be performant and i think even possible...
_________________
Quite obviously, our problems do not come from what we invent, but from how we use our sophisticated toys. The difficulties stem not from our hardware or software, but from ourselves.
Barbarossa,
Anatomy of a Rebellion
[img]http://misc.shalmirane.de/violance.JPG[/img]

Stratadrake · Posted: Sun Mar 12, 2006 6:47 pm Post subject:

Another way to get the most recent pictures of that artist:

Type a URL like:

www.fanart-central.net/user.php?aid=______&pageID=99999

Which will direct you to the most recent page of their user gallery. You could extract the six-digit picture id (pid) values from the picture URL's and compare those to the last visit to determine if the picture is new or not. (higher pid values are always newer submissions)
_________________
Strata here: [url=http://www.nanowrimo.org/eng/user/242293]Nanowrimo[/url] - [url=www.fanart-central.net/user-Stratadrake.php]FAC[/url] - [url=http://stratadrake.deviantart.com]dA[/url] - [url=www.furaffinity.net/user/Stratadrake/]FA[/url]
[size=9]Disclaimer: Posts may contain URLs. Click [url=http://tvtropes.org/pmwiki/pmwiki.php/Main/TVTropesWillRuinYourLife]at your own risk.[/url][/size]

Nachtwind · Posted: Sun Mar 12, 2006 6:54 pm Post subject:

I thought of that too, but in the end it would be the same serverload for FAC as if i was parsing the whole list and getting the data that way. And you would only see if a user has only 1 new picture. If someone uploads 2 or more pics at a time you could not track that.

Btw, as far as Categories are concearned i am almost done... though i could get myself hang for a small problem i am still at. (Why is that variable true after setting it to false?!)

_________________
Quite obviously, our problems do not come from what we invent, but from how we use our sophisticated toys. The difficulties stem not from our hardware or software, but from ourselves.
Barbarossa,
Anatomy of a Rebellion
[img]http://misc.shalmirane.de/violance.JPG[/img]