AniDB Plugin update

[nickpick]

Hi,

I updated your AniDB plugin to properly display names, fixed the director and official website fields and added support for cast. Should work fine.

Cheers,

~ Nick.

PS: You guys really should add the ability to mark multiple objects as seen/members of collection with a few clicks.

Code:

# -*- coding: utf-8 -*-

__revision__ = '$Id: PluginMovieAniDB.py 1148 2009-02-05 20:31:57Z mikej06 $'

# Copyright (c) 2005-2009 Piotr Ożarowski
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU Library General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA

# You may use and distribute this software under the terms of the
# GNU General Public License, version 2 or later

import gutils, movie
import string, re
from gutils import decompress

plugin_name         = 'AnimeDB'
plugin_description  = 'Anime DataBase'
plugin_url          = 'www.anidb.net'
plugin_language     = _('English')
plugin_author       = 'Piotr Ożarowski'
plugin_author_email = '<ozarow+griffith@gmail.com>'
plugin_version      = '2.6'

aid_pattern = re.compile('[?&;]aid=(\d+)')

class Plugin(movie.Movie):
def __init__(self, id):
self.encode = 'utf-8'
if string.find(id, 'http://') != -1:
self.url = str(id)
self.movie_id = 'anidb'
else:
self.movie_id = str(id)
self.url = "http://anidb.net/perl-bin/animedb.pl?show=anime&aid=%s" % self.movie_id

def initialize(self):
self.page = decompress(self.page)
if self.movie_id == 'anidb':
aid =  aid_pattern.search(self.page)
if aid:
self.movie_id = aid.groups()[0]
self.url = "http://anidb.net/perl-bin/animedb.pl?show=anime&aid=%s" % self.movie_id
else:
return False
self.page = gutils.after(self.page, 'id="layout-content"')
pos = string.find(self.page, 'class="g_section anime_episodes">')
if pos >0:
self.page = self.page[:pos]

def get_image(self):
match = re.search('img\d*.anidb.net/pics/anime/\d*.jpg', self.page)
if match is not None:
self.image_url = 'http://' + match.group()
else:
self.image_url = ''

def get_o_title(self):
self.o_title = gutils.trim(self.page, '<span class="i_icon i_audio_ja" title=" language: japanese"><span>ja</span></span>', '</td>')
self.o_title = gutils.trim(self.o_title, '<label>', '</label>')

def get_title(self):
self.title = gutils.trim(self.page, '<h1 class="anime">Anime: ', '</h1>')

def get_director(self):
#self.director = gutils.trim(self.page, '<a title="Direction (&#x76E3;&#x7763;)" href="animedb.pl?show=creator&amp;creatorid=1482">', '</a>')
self.director = gutils.trim(self.page, '<a title="Direction (&#x76E3;&#x7763;)" href=', 'a>')
self.director = gutils.trim(self.director, '>', '</')

def get_plot(self):
self.plot = gutils.trim(self.page, 'class="desc">', '</div>')
self.plot = self.plot.replace('<br/>', '\n')

def get_year(self):
self.year = gutils.trim(self.page, '"field">Year', '</td>')
self.year = gutils.after(self.year, '"value">')[-4:]

def get_runtime(self):
self.runtime = gutils.trim(self.page, '<label>Complete Movie</label>', '</tr>')
self.runtime = gutils.trim(self.runtime, '<td class="duration">', 'm')

def get_genre(self):
self.genre = gutils.trim(self.page, '>Categories<', '</td>')
self.genre = gutils.after(self.genre, 'value">')
self.genre = gutils.strip_tags(self.genre)
if len(self.genre) and self.genre.endswith('- similar'):
self.genre =  self.genre[:-9]
elif self.genre == '-':
self.genre = ''
self.genre = string.replace(self.genre, '\n', '')

#CAST WORK STARTS HERE
def get_cast(self):
self.cast = 'Characters:\n---------------'
castv = gutils.trim(self.page, '<table id="characterlist" class="characterlist">', '</table>')
if castv != '':
castparts = string.split(castv, '<tr ')
for index in range(2, len(castparts), 1):
castpart = castparts[index]
castcharacter = gutils.clean(gutils.trim(castpart, '<td rowspan="1" class="name">', '</td>'))
#castcharacter = gutils.clean(gutils.trim(castcharacter, '<a href="animedb.pl?show=character', '</a>'))
#castcharacter = gutils.clean(gutils.trim(castcharacter, '">', '</a>'))
castentity = gutils.clean(gutils.trim(castpart, '<td rowspan="1" class="entity">', '</td>'))
castactor = gutils.clean(gutils.trim(castpart, '<td class="name"><a href="animedb.pl?show=creator&amp;creatorid=', 'd>'))
castactor = gutils.clean(gutils.trim(castactor, '">', '</t'))
if castv == ' ':
castactor = 'unknown'
castrelation = gutils.clean(gutils.trim(castpart, '<td rowspan="1" class="relation">', '</td>'))
castappearance = gutils.clean(gutils.trim(castpart, '<td rowspan="1" class="eprange">', '</td>'))
#self.cast += '\n' + castactor + ' - ' + castcharacter + ' [' + castentity + '; ' + castrelation + '; appears in episodes: ' + castappearance + ']'
self.cast += '\n\n' + '[' + castcharacter + '] voiced by ' + castactor + '\n' + castentity + '; ' + castrelation + '; appears in episodes: ' + castappearance

#CAST WORK ENDS HERE
def get_classification(self):
self.classification = ''

def get_studio(self):
self.studio = gutils.trim(self.page, '<tr class="producers">', '</tr>')
if self.studio == '':
self.studio = gutils.trim(self.page, '<tr class="g_odd producers">', '</tr>')
self.studio = gutils.trim(self.studio, '<td class="value">', '</td>')
self.studio = gutils.strip_tags(self.studio)
if len(self.studio) and self.studio[:2] == " (":
self.studio = self.studio[2:]
if self.studio[len(self.studio)-1:] == ')':
self.studio = self.studio[:len(self.studio)-1]
self.studio = string.replace(self.studio, '\n', '')

def get_o_site(self):
#self.o_site = gutils.trim(self.page, '"field">URL', '</td>')
#self.o_site = gutils.trim(self.o_site, 'href="', '"')
self.o_site = gutils.trim(self.page, '<th class="field">Resources</th>', '</tr>') #class varies, tag used
self.o_site = gutils.trim(self.o_site, '<a href="', '" rel="anidb::extern">Official page</a>')

def get_site(self):
self.site = self.url

def get_trailer(self):
self.trailer = ''

def get_country(self):
self.country = ''

def get_rating(self):
self.rating = gutils.clean(gutils.after(gutils.trim(self.page, '<span class="rating', '</a>'), '>'))
if self.rating:
try:
self.rating = str(round(float(self.rating)))
except:
self.rating = ''

def get_notes(self):
self.notes = ''
# ...type and episodes
atype = gutils.trim(self.page, '"field">Type', '</td>')
atype = gutils.clean(atype)
if atype != '':
self.notes += "Type: %s\n" % atype
episodes = gutils.trim(self.page, '>Episode list<', '</table>')
if episodes != '':
parts = string.split(episodes, '<tr ')
for index in range(2, len(parts), 1):
part = parts[index]
nr = gutils.clean(gutils.trim(part, 'class="id eid">', '</td>'))
title = gutils.clean(gutils.after(gutils.trim(part, '<label', '</td>'), '>'))
duration = gutils.clean(gutils.trim(part, 'class="duration">', '</td>'))
airdate = gutils.clean(gutils.trim(part, 'class="date airdate">', '</td>'))
self.notes += '\n' + nr + ': ' + title + ' (' + duration + ', ' + airdate + ')'

class SearchPlugin(movie.SearchMovie):
def __init__(self):
self.encode = 'utf-8'
self.original_url_search = 'http://anidb.net/perl-bin/animedb.pl?show=animelist&do.search=search&adb.search='
self.translated_url_search = 'http://anidb.net/perl-bin/animedb.pl?show=animelist&do.search=search&adb.search='

def search(self,parent_window):
self.open_search(parent_window)
self.page = decompress(self.page)

tmp = string.find(self.page, '>Anime List - Search for: ')
if tmp == -1: # already a movie page
self.page = 'movie'
else: # multiple matches
self.page = gutils.trim(self.page, 'class="animelist"', '</table>');
self.page = gutils.after(self.page, '</tr>');

return self.page

def get_searches(self):
if self.page == 'movie':    # already a movie page
self.number_results = 1
self.ids.append(self.url)
self.titles.append(self.title)
else: # multiple matches
elements = string.split(self.page,"</tr>")
self.number_results = elements[-1]

if len(elements[0]):
for element in elements:
aid = aid_pattern.search(element)
if not aid:
continue
title = gutils.clean(gutils.trim(element, '<td class="name">', '</a>'))
type = gutils.clean(gutils.after(gutils.trim(element, '<td class="type', '</td>'), '>'))
self.ids.append(aid.groups()[0])
if type:
self.titles.append(title + ' (' + type + ')')
else:
self.titles.append(title)
else:
self.number_results = 0

[asterisk20xx]

AniDB changed the class for the Plot Summary on their pages and broke this plugin. All data imports fine except for the Plot data.

This line

     self.plot = gutils.trim(self.page, 'class="desc">', '</div>')

needs to be changed to the following to get plot data importing again:

     self.plot = gutils.trim(self.page, 'class="g_bubble desc">', '</div>')

Attached is a corrected plugin for those who don't know what that means. Save it to your \Griffith\lib\plugins\movie\ folder

[mike]

Thanks for the info. I fixed it already. Additional I fixed the import of the original site url.
http://svn.berlios.de/svnroot/repos/griffith/trunk/lib/plugins/movie/PluginMovieAniDB.py

[deridiot]

*sigh* ok where to start... STOP RAPING OUR FREE SERVICE...

scraping webpages isn't nice you know. AniDB is a free service without any ads or spamware. nor do we take donations. we are a smallscale website with limited resources and we seriously do NOT approve of the webpages being scraped. that's silly, wrong and costs unnecessary resources. it's one thing to scrape services like imdb which got the money and power to not give a crap about a handful people scraping their service. it's different when you are constantly up your asses with load.

there is no point in what you are doing whatsoever especially since we DO provide APIs to retrieve data:

http://wiki.anidb.info/w/API

we do not appreciate this behaviour and seriously hope you reconsider your decisions. (read: fix your shit or i will make it my new hobby breaking your code daily several times)

der idiot
senior mod and main coder for AniDB

[POX]

I will rewrite AniDB module to use XML API, didn't know one exists, sorry.

[deridiot]

i appreciate that decision. have a nice day

[deridiot]

fast forward half a year and nothing happened. i would hardly consider myself an impatient person, but i gave you a reasonable amount of time to act...

[asterisk20xx]

Currently the AniDB plugin is broken again. Bumping to see if there's been any progress with getting the official API working with Griffith.

[CinéphOli]

On the same line of discussion, there is also an API for allocine.fr to be considered (both json and xml, please chose xml )…
Some info here (in French only, sorry, but straightforward)

And I must confess that I can only concur with deridiot, when the API exists it is much more efficient (and polite…) to use it. It is the way to go.
I suggest that the plugin AniDB be removed as long as it doesn't use the API. We must encourage people like deridiot to develop/maintain movie databases, not make their life more miserable.

[mike]

I didn't look into the AniDb issue because POX want to do that.
But I switched the Allocine plugin to the official api. Thanks for the hint. Didn't know that before.
If someone knows such an api for other movie sites please drop a post here.
http://svn.berlios.de/svnroot/repos/griffith/trunk/lib/plugins/movie/PluginMovieAllocine.py

[mike]

New version of the anidb plugin is finished and will be included in the next release. 

[CinéphOli]

Mike, thank you for the aniDB plugin. I don't really use it myself, but I got an error when looking for Akira:

Code:

2011-09-19T17:02:07: E: Griffith(movie:304):
Traceback (most recent call last):
  File "/home/olivier/Documents/Informatique/programmes/SVN/griffith/lib/movie.py", line 275, in parse_movie
    self.get_plot()
  File "/home/olivier/Documents/Informatique/programmes/SVN/griffith/lib/plugins/movie/PluginMovieAniDB.py", line 82, in get_plot
    self.plot = self._xml.find('description').text
AttributeError: 'NoneType' object has no attribute 'text'
Traceback (most recent call last):
  File "./griffith", line 1422, in on_results_select_press_event
    return self._resultswin_process(selected_key)
  File "./griffith", line 272, in populate_dialog_with_results
    add.populate_with_results(self)
  File "/home/olivier/Documents/Informatique/programmes/SVN/griffith/lib/add.py", line 197, in populate_with_results
    plot_buffer.set_text(gutils.convert_entities(self.movie.plot))
  File "/home/olivier/Documents/Informatique/programmes/SVN/griffith/lib/gutils.py", line 219, in convert_entities
    in_entity = ENTITY.search(text)
TypeError: expected string or buffer

This happened after selecting the first option of the returned short list, as shown on the capture screen hereunder (joined file). Griffith doesn't crash, but it doesn't collect any information.

PS: any reason in particular for choosing JSON over XML for the allocine.fr API? Is it easier?

[mike]

Your thanks should go to POX. He started the rewriting of the plugin. I contributed only some small fixes.
And my thanks goes to you because you made some tests in the past and reported the errors here.
And sure, the same thanks goes to every other user of griffith.
Hopefully you will do that in the future too.
Btw. the current error with the movie Akira is fixed.

[CinéphOli]

So, my thanks to POX too, of course !
And don't mention my error reporting activity, you guys provide me with a nice application that I enjoy to use, it's only normal to participate to its enhancement as much as possible. I wish I could do more, "programmaly" speaking…
And, btw, thank you also for the Allociné API plugin, I forgot to mention it last time.

Now, back to the AniDB plugin. You will say that I'm a PITA (I let you imagine what it stands for  …), but I found another little problem: the sorting order of the episodes list in the Notes tab is not correct, being based on a character sort instead of a numeric sort: 1, 10, 100 … 19 before 2. A very classic sorting problem, illustrated in the capture screen below.
Since some of those TV series can count hundreds of episodes, I suppose the Griffith user will be more than happy not having to edit this field manually.

I searched for MAR (MAR Heaven in the short list).

[mike]

Believe me, you are not a PITA (at least for me)
I fixed the sorting order. Only episodes with non-numerical numbers are not correctly sorted.
In your examples there are episodes which start with "C".