pahelper / phoenixadult.bundle Goto Github PK

View Code? Open in Web Editor NEW

This project forked from phoenixplexcode/phoenixadult.bundle

352.0 39.0 138.0 9.28 MB

Plex Meta-Data Agent for scene videos from multiple adult sites

Python 99.98% Batchfile 0.02%

phoenixadult.bundle's Introduction

PhoenixAdult metadata agent

This metadata agent helps fill Plex with information for your adult videos by pulling from the original site.

Features

Currently the features of this metadata agent are:

Scrapes any available Metadata, including:
- Scene Title
- Scene Summary
- Studio
- Originating Site / Subsite / Site Collection (saved as the Tagline, and also a Collection for easy searching)
- Release Date
- Genres / Categories / Tags
- Porn Stars (stored as Actors, with photo)
- Scene Director(s)
- Movie Poster(s) / Background Art
Function to strip common "scene" tags to assist with matching
Function to replace abbreviated site names with full site names to assist with matching
Function to clean up / merge genres
Function to clean up / merge actresses with aliases (e.g. Doris Ivy is Gina Gerson)
Function to locate an image for actors where the original site doesn't provide one
Function to manually add actors for sites the agent doesn't support
Function to automatically rename files (WIP)

File Naming

The agent will try to match your file automatically, usually based on the filename. You can assist it by renaming your video appropriately. If the video is not successfully matched, you can try to manually match it using the [Match...] function in Plex. See the manual searching document for more information. Best practice for each site is listed in the sitelist document.

Plex Video Files needs to be set as the library scanner for best results.

Here are some naming structures we recommend:

SiteName - YYYY-MM-DD - Scene Name .[ext]
SiteName - Scene Name .[ext]
SiteName - YYYY-MM-DD - Actor(s) .[ext]
SiteName - Actor(s) .[ext]

Real world examples:

Blacked - 2018-12-11 - The Real Thing.mp4
Blacked - Hot Vacation Adventures.mp4
Blacked - 2018-09-07 - Alecia Fox.mp4
Blacked - Alecia Fox Joss Lescaf.mp4

Some sites do not have a search function available. This is where SceneID and Direct URL come in to play. These usually don't make the most intuitive filenames, so it is often better to use the [Match...] function in Plex. See the manual searching document for more information.

If you would prefer to integrate SceneIDs into your filenames, instead of manually matching in Plex, here are some naming structures we recommend:

SiteName - YYYY-MM-DD - SceneID .[ext]
SiteName - SceneID .[ext]
SiteName - SceneID - Scene Name .[ext]

Real world examples:

EvilAngel - 2016-10-02 - 119883 (taken from the URL https://www.evilangel.com/en/video/Allie--Lilys-Slobbery-Anal-Threesome/119883)
MomsTeachSex - 314082 (taken from the URL https://momsteachsex.com/tube/watch/314082)
Babes - 3075191 - Give In to Desire (taken from the URL https://www.babes.com/scene/3075191/1)

Installation

How to find the plug-in folder location: https://support.plex.tv/hc/en-us/articles/201106098-How-do-I-find-the-Plug-Ins-folder-

Get the PAhelper source zip in GitHub release at https://github.com/PAhelper/PhoenixAdult.bundle > "Clone or download > Download Zip
Open PhoenixAdult.bundle-master.zip and copy the folder inside (PhoenixAdult.bundle-master) to the plug-ins folders
Rename folder to "PhoenixAdult.bundle" (remove "-master")

Reporting a bug

We try to maintain bug-free code, but bugs do happen. If you are having difficulty matching a scene, please refer to Known Issues before submitting an Issue.

Known Limitations

Some sites do not have many high quality images that can be used as poster or background art. I have found the forums at ViperGirls.to to be a great resource for artwork in these situations.

Change Log/Updates

To view the most detailed changes to code, check the commit log. Additional information can be obtained from the list of merged pull requests.

Supported Networks

To view the full list of supported sites, check out the sitelist doc. If your favorite site isn't supported, head over to Issue #1 to add your request to the list, or vote on the current requests.

Preferences

To view the full list of supported preferences, check out the preferences doc.

phoenixadult.bundle's People

Contributors

Stargazers

Watchers

Forkers

pahelper2 rufusriley mutluerol woodbridgesub gguy35 flalfa1 balletootu sinjin69 bigmac00 bigthree8 ordinarygulp anotherpahelper spanishu cuddlebear92 ferengi82 cristiano1976pt david-fillion allegedschool b0nensfw jamesalexw blackbackedbat aicha1566 delposibl hdj983hbas cactuzjak coresus-codes chonka1 kargii joekarlsson pornportal bmgav889 ltrain83 justmultiplex keysersose12 nickeich1117 fakegit kobatwelve m0uthless pibi73 stevef9432203 a18project martibx7 yougreektube pedrolara-boop smvemjsuneptune yes12397 emmortal451 docmcfukin7 datahodor powermodder dirtyracer1337 albyofdoom heartest anondoejohn anonhelper123 trimpo87 icebluebear leloubs enbious shadowwalker86 mbartozzi degeneratesmut rolandalvares cj12312021 secretsanta007 solemnmonk rasheed-edvo oikmeg sakibulalam haleybojackson limeizhen1 pitaf-youp riksd007 g6grim scene-switch angrybatt ha2278 lk97gre chempke wolhlow tinyturtle69 mike6460 macr0dev extant-squash mrx-31415 codeanator thisismyname34 tylerdurden55 orinocoz therxlord cptn-code 11samype nicebluemice adtdev20 soilaccountant hwatbobby skinemax sunlei023 diamondpete xnwabudike

phoenixadult.bundle's Issues

No search results for Lubed.com releases

This might be me not correctly formatting my titles for searches, but I can't make any of the lubed titles show up in plex. I took a look at the way the title search breaks things down, but I can't make this video file work, for example: "Lubed: Wet & Wild Besties".

On the Lubed.com website, the title following the lubed URL is "wet-wild-besties" and I have tried searching with the Name set to "Wet Wild Besties", "Lubed - Wet Wild Besties", "Lubed - Wet & Wild Besties", and just "Lubed" and gotten no results every time.

Brazzers metadata broken

Had been working after the fix pushed 12/27/18, but seems to be broken again after the recent couple commits. Images are pulling but not genres/actors/etc.

Latest Update broke it

There is not a single match..

2019-01-22 21:10:46,742 (70fff460) : DEBUG (networking:166) - Requesting 'http://127.0.0.1:32400/:/plugins/com.plexapp.system/messaging/function/X0J1bmRsZVNlcnZpY2U6QWxsU2VydmljZ$ 2019-01-22 21:10:46,756 (76fb9000) : DEBUG (runtime:1117) - Created a thread named 'get_server_info' 2019-01-22 21:10:46,758 (76fb9000) : DEBUG (core:150) - Finished starting framework core 2019-01-22 21:10:46,760 (76fb9000) : DEBUG (core:560) - Loading plug-in code 2019-01-22 21:10:46,763 (6feff460) : DEBUG (networking:166) - Requesting 'http://127.0.0.1:32400' 2019-01-22 21:10:49,653 (76fb9000) : DEBUG (core:566) - Finished loading plug-in code 2019-01-22 21:10:52,733 (6feff460) : DEBUG (core:538) - Machine identifier is a8ccd6cc11af57bbc26337b366ffa5d20f3152d5 2019-01-22 21:10:52,774 (6feff460) : DEBUG (core:539) - Server version is 1.14.1.5488-cc260c476 2019-01-22 21:10:53,670 (70fff460) : DEBUG (services:362) - Loaded services 2019-01-22 21:10:53,760 (706ff460) : DEBUG (services:438) - No shared code to load

Agent won't load in Plex

I can not see the agent anymore, it wont load. Restarted my server a couple of times.

In the server settings/agents all the rest of my agents show up except the PhoenixAdult.

FYI: I've installed a second install on my windows machine and I do not see it there either.

PorndoePremium do not match

There seems to be issues with the PorndoePremium site. I do not get any match anymore

XEmpire not pulling actor name / picture

nice to see that we have a scanner now for XEmpirel. Would you be able to add as well that the actor gets pulled as well as the actor image?

Many thanks!

Naughty America - Add subsite as a Collection

Matched a NaughtyOffice video and discovered the subsite wasn't added as a Tagline or as a Collection.

Correct Hardx, Darkx, Lesbianx, Eroticax search results

I believe these suffer from the same extraneous results as the Babes search results...

Sweet Sinner/Sweetheart Video/Girlfriends Films support

Hey, first of all many, many, many thanks for adding these 2 sites!

But can you make it so the search can also find the full movie's? The agent searches in/for scene's (those I get back as results)

The scene's are tagged with: Studio XEmpire, can this be changed to Sweet Sinner/Sweetheart Video?

I'm testing with a few scene's/ movies but for example: Sweet Sinner - Father Figure 10 - Wet Watercolors isn't found ( I've also tried Sweet Sinner - Wet Watercolors) on there site it is found but on page 5 of the search results. I first thought that the search results were shown in the order they were on the site but that is incorrect...

21sextreme support

I wanted to add my 21sextreme scenes but run into a problem with 2 of them namely:
21sextury - You rang M'lord – part 1 https://www.21sextreme.com/en/video/You-rang-Mlord---part-1/92603
and
21sextury - You rang M'lord – part 2 https://www.21sextreme.com/en/video/You-rang-Mlord---part-2-Scene-01/92602

I've tried all sorts of combinations (shortening the search, searching with the scene number) but didn't get the scenes to show up, can you take a look/crack at it?

FootsieBabes incorrectly matches Babes

Because of the logic used to match a search string to an existing site, FootsieBabes scenes (siteID 369) try to search against Babes.com (siteID 271).

The code causing the problem is in PAsearchSites.getSearchSiteIDByFilter() around line 514. I've been thinking about rewriting that anyway so the site can be anywhere in the filename, not just the beginning...

Also, I believe Blacked and BlackedRaw suffer from the same accidental matching problem, but when I tried to dissect Phoenix's workaround, I just broke the sites (all Blacked searches only returned BlackedRaw results). The workaround has something to do with having double entries in the searchSites list (0,1 and 3,4), maybe the way they're ordered [1,0,4,3] with a bit of extra code in init.py that redirects [3] to [0] and [4] to [1] around line 61.

bugs to fix

1-evil angel is not working after adding it to gamma entertainment
2- all Gamma Entertainment not working
2- fame digital has more sub-sites network to list them

GHSecrets get full name from image links

GHSecrets lists the girls with shortened forms of their porn industry name, such as "Brandi B" instead of "Brandi Bae", or "Stevie" instead of "Stevie Grey". It appears their full name is used as part of the image path for their scenes and could be used to automatically name the actresses properly in some (most? all?) cases, e.g. "/images/brandibae/01.jpg"

Girlsway Poster support

I am having trouble to get posters for some Girlsway scenes to be precise:
Girlsway - Crush On Mommy
Girlsway - The Pick-Up Artist
Girlsway - Last-Minute Christmas
Girlsway - Finally Home
Girlsway - Showcases: Alina Lopez - 2 Scenes in 1

Evil Angel

1 -All titles after 2016 are working fine because they name the files with different names.
all productions before 2016 works as follow:
xxxmovie, scene#1
xxxmovie, scene#2
xxxmovie, scene#3

so while searching results will show as:
xxxmovie [Evil Angel]
xxxmovie [Evil Angel]
xxxmovie [Evil Angel]

so u have to check manually to see if the movie will pair with the scene #

2- Some times results of BTS and Outlooks doesnt show.
3- special characters such as ( / : ) not recognized in the search while its there in the title name

Girlfriends Films support

I see that you've added/crossed Girlfriends Films, so I tried a few scenes/DVD releases but no dice! The agent only comes back with: No matches found.

For example: Conjoined (for a whole DVD test)
https://www.girlfriendsfilms.com/en/dvd/Conjoined/29303
I've tried it with:
Girlfriendsfilms - Conjoined
Girlfriends films - Conjoined
Girlfriendsfilms - 29303

And Women Seeking Women 153 scene 1 (for a scene test)
https://www.girlfriendsfilms.com/en/video/Women-Seeking-Women-153/134666
Girlfriendsfilms - Women Seeking Women 153
Girlfriendsfilms - Jade Nile Olive Glass
Girlfriendsfilms - 134666
(Could you show how to format the titles of individual scenes because on their website all scenes are named (of in this case) Women Seeking Women 153 with the change of the actresses names on the scene's page itself.

What am I doing wrong?

PAactors.py

Another helpful thing would be a name/alias mapper similar to PheonixUsers genre mapper in the agents Plex settings. It would be nice to be able to add a name and then map any aliases in there to bring everything together under one name. Maybe the template is already there in the PAgenres.py file? Just an idea if easy enough.

PassionHD and FantasyHD not working

Hi,
if you add a clip from PassionHD or FantasyHD and you name this way:
PassionHD - <name of the clip> or Passion-HD - <name of the clip> the agent searches the clip for all of the networks without results. You need to remove the dash in the PAsearchSites.py to make it work with its plugin.

Secondly, when you do that, an SSL handshake error occurs, which is the same faced with EmpireX websites, as reported here: #13

Naughty America seems to be broken as well

I cannot get any meta data
here the log output

`2019-01-14 16:12:37,624 (70eff460) : DEBUG (networking:166) - Requesting 'http://127.0.0.1:32400/library/metadata/14506/tree'
2019-01-14 16:12:37,687 (70eff460) : INFO (init:406) - UPDATE CALLED*
2019-01-14 16:12:37,690 (70eff460) : INFO (init:409) - 5
2019-01-14 16:12:37,700 (70eff460) : DEBUG (networking:166) - Requesting 'https://tour.naughtyamerica.comscene/ella-hughes-fucking-in-the-bed-with-her-bubble-butt-24135'
2019-01-14 16:12:37,996 (70eff460) : CRITICAL (agentkit:1078) - Exception in the update function of agent named 'PhoenixAdult', called with guid 'com.plexapp.agents.phoenixadult://scene_ella-hughes-fucking-in-the-bed-with-her-bubble-butt-24135|5?lang=en' (most recent$
File "/usr/lib/plexmediaserver/Resources/Plug-ins-cc260c476/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/api/agentkit.py", line 1076, in _update
agent.update(obj, media, lang, **kwargs)
File "/home/pi/Library/Application Support/Plex Media Server/Plug-ins/PhoenixAdult.bundle/Contents/Code/init.py", line 448, in update
metadata = PAsearchSites.siteNaughtyAmerica.update(metadata,siteID,movieGenres,movieActors)
File "/home/pi/Library/Application Support/Plex Media Server/Plug-ins/PhoenixAdult.bundle/Contents/Code/siteNaughtyAmerica.py", line 32, in update
detailsPageElements = HTML.ElementFromURL(url)
File "/usr/lib/plexmediaserver/Resources/Plug-ins-cc260c476/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/api/parsekit.py", line 482, in ElementFromURL
method=method,
File "/usr/lib/plexmediaserver/Resources/Plug-ins-cc260c476/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/api/networkkit.py", line 67, in _http_request
req = self._core.networking.http_request(url, *args, **kwargs)
File "/usr/lib/plexmediaserver/Resources/Plug-ins-cc260c476/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/components/networking.py", line 370, in http_request
return HTTPRequest(self._core, url, data, h, url_cache, encoding, errors, timeout, immediate, sleep, opener, follow_redirects, method)
File "/usr/lib/plexmediaserver/Resources/Plug-ins-cc260c476/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/components/networking.py", line 141, in init
self.load()
File "/usr/lib/plexmediaserver/Resources/Plug-ins-cc260c476/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/components/networking.py", line 181, in load
f = self._opener.open(req, timeout=self._timeout)
File "/usr/lib/plexmediaserver/Resources/Plug-ins-cc260c476/Framework.bundle/Contents/Resources/Platforms/Shared/Libraries/urllib2_new.py", line 438, in open
response = self._open(req, data)
File "/usr/lib/plexmediaserver/Resources/Plug-ins-cc260c476/Framework.bundle/Contents/Resources/Platforms/Shared/Libraries/urllib2_new.py", line 456, in _open
'_open', req)
File "/usr/lib/plexmediaserver/Resources/Plug-ins-cc260c476/Framework.bundle/Contents/Resources/Platforms/Shared/Libraries/urllib2_new.py", line 416, in _call_chain
result = func(*args)
File "/usr/lib/plexmediaserver/Resources/Plug-ins-cc260c476/Framework.bundle/Contents/Resources/Platforms/Shared/Libraries/urllib2_new.py", line 1225, in https_open
return self.do_open(httplib.HTTPSConnection, req)
File "/usr/lib/plexmediaserver/Resources/Plug-ins-cc260c476/Framework.bundle/Contents/Resources/Platforms/Shared/Libraries/urllib2_new.py", line 1192, in do_open
raise URLError(err)
URLError: <urlopen error [Errno -2] Name or service not known>

2019-01-14 16:12:38,009 (70eff460) : DEBUG (model:229) - Serializing to /home/pi/Library/Application Support/Plex Media Server/Metadata/Movies/4/59ee1e886ae76e9d9c3bf74f97944864f45db3a.bundle/Contents/com.plexapp.agents.phoenixadult/Info.xml
2019-01-14 16:12:38,123 (70eff460) : DEBUG (runtime:88) - Sending packed state data (21284 bytes)
2019-01-14 16:12:38,126 (70eff460) : DEBUG (runtime:924) - Response: [200] str, 16 bytes
`

Neighbor Affair (NaughtyAmerica) not matching after latest changes

I have a file named like so:

Neighbor Affair - 2018-12-23 - Mandy Muse.mp4

I've tried every permutation I can think of, but no luck. Did the the recent changes break this somehow?

[site request] kink.com sites, especially The Upper Floor and Sex & Submission

Would love to have kink.com supported. Doing some investigation now, and it seems like actress searches work pretty well, as long as you quote the actress names:

TheUpperFloor.18.05.04.Aiden.Starr.Karlee.Grey.And.Vanessa.Sky.XXX.720p.MP4-KTR =>
https://www.kink.com/search?q=%22Aiden+Starr%22+%22Karlee+Grey%22+%22Vanessa+Sky%22

TheUpperFloor.17.02.21.Kacie.Castle.And.Sadie.Santana.XXX.MP4-SDXXX =>
https://www.kink.com/search?q=%22Kacie+Castle%22+%22Sadie+Santana%22

Trying to see if I can make a date + actress search with "Abella Danger". She has a bunch of videos, but so far I can't find the correct url scheme to find it nicely.

[bug] My Wifes Hot Friend search not working

Using these searches, nothing turns up:

MyWifesHotFriend - 2018-12-04 - lauren phillips
My Wifes Hot Friend - 2018-12-04 - lauren phillips

Naughty America web search can find it, if I search for "milks". The title of the video being "Lauren Phillips milks a big cock". I haven't been able to get this into a search I can use the in Plex Match dialog box however.

[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE]

sitePornPros has error with ssl

LesbianX search results give SSL error

I don't have the error in front of me, but it basically said liburl2 couldn't connect because no matching SSL cypher was found. I think that basically means the SSL cert for lesbianx.com is newer than the current python code can handle.

New Site Support

(Last Updated 04/20/2020)

Requested New Sites:

PervCity
- Anal Overdose
- Chocolate BJs
- Oral Overdose
- Banging Beauties
- Up Her Asshole
abbywinters.com
forbiddenfruitsfilms.com (store.forbiddenfruitsfilms.com/streaming-video-on-demand)
getyourkneesdirty.com
Intersec Interactive
- KinksterCash.com
- InsexArchives.com
- Cruel Romance
- Agan Medon
thetalesfromtheedge.com (offshoot of NewSensations)
assylum.com
slavemouth.com
dickdrainers.com
Wasteland
Other BangBros Sites (not part of main site)
- https://filthyfamily.com
- http://blacksonmoms.com
FamilyLust (https://www.familylust.com)
KinkyFamily (Dirty Flix network) (http://kinkyfamily.com)
ExclusiveTeenPorn (http://www.exclusiveteenporn.com)
DickOnTrip (https://www.dickontrip.com)
Nutabu (http://www.nutabu.com)
SinX Network (https://www.sinx.com/)
- Tainster (https://www.tainster.com)
Scoreland (scoreland.com)
Sister Trick (https://www.sistertrick.com)
Girls Under Arrest (https://www.girlsunderarrest.com)
My Life In Miami (https://mylifeinmiami.com)
Exploited College Girls (https://exploitedcollegegirls.com)
Backroom Casting Couch (https://backroomcastingcouch.com)
Amateur Creampies (http://www.amateurcreampies.com)
ATK Network (DVDs and some scenes: https://www.atkdvd.com)
ATK MEMBERSHIP SITES
- ATKingdom.com
- ATKHairy.com
- ATKGalleria.com
- ATKExotics.com
- ATKPremium.com
- ATKPetites.com
- ATKArchives.com
Sex Like Real (https://sexlikereal.com)
Digital Sin (Perhaps via http://promo.newsensations.com/index.php)
Czech Casting (https://czechcasting.com)
PornCZ Network (https://www.porncz.com)
Third Movies (http://www.thirdmovies.com/search?search=)
PurgatoryX (https://tour.purgatoryx.com)
Jacquie et Michel (www.jacquieetmicheltv.net)
https://pornmegaload.com
https://50plusmilfs.com
https://60plusmilfs.com
allfinegirls.com
collegerules.com
freshouttahighschool.com
FTV Network
- https://www.ftvgirls.com/
- https://ftvmilfs.com/
hookuphotshot.com
lanewgirl.com
www.horrorporn.com
https://pornhub.com
https://affect3dstore.com/animation.html
https://www.dlsite.com/ecchi-eng/works/animation
StudioFow
In the Crack (https://inthecrack.com/)
I Feel Myself (http://www.ifeelmyself.com/)
Beautiful Agony (https://www.beautifulagony.com)
ZeroSpaces (https://zerospaces.com)
Top Web Models (Universal Search i believe)
- https://biggulpgirls.com
- https://deepthroatsirens.com
- https://facialsforever.com
- https://poundedpetite.com
- http://shesbrandnew.com
- https://cougarseason.com/
- https://2girls1camera.com/
Hussie Pass
FILF
Woodman Films Network
- woodmancastingx.com
- Wake Up N Fuck
- Woodman Films
Colette
Fetish Network (http://www.fetishnetwork.com)
666 Bukkake (https://www.666bukkake.com)
PureXXXFilms
KillerGram
SinDrive
BrokenTeens
PinkClub
RodneyMoore
AnatomikMedia

Requested Additional Support for Existing Sites:

www.mypervmom.com (has scenes that aren't on www.pervmom.com)
MileHighNetwork (DVD support requested)
Elegant Angel (Individual scene support requested)

Sites We Can't Currently Support:

Site doesn't provide enough info, all scenes redirect to a "Join Now" page

Best way to support your work

Hi,
I'm not a Plex plugin developer but I know how to work with selectors and Python. Since I'm here, I want to help you to work on some sites that I personally use. Could you please give me some tips on how can I (or the community) help you with?

This is an amazing work started by PhoenixPlexCode and since he disappeared, I appreciated your work and I seriously want to contribute if you want.

Clean up Darkx, Lesbianx, Eroticax files

They are pretty much clones of the Hardx code I wrote, but there are subtle differences that break these clone sites.

Remove Director
Remove or replace Tagline
Add photos from the free photos link below each video

Problems with SexyHub / Fitness rooms

Fitnessrooms.com seems to be within the sexyhub network, but the results can only be found when you got to fitnessrooms.com and search there directly, you cannot find it if you search at sexyhub.com, so fitness rooms must be searched at fitnessrooms.com to work.

Digital Playground support

I've read that you added Digital Playground to the agent but when I search for for example: Digital Playground - Cock Therapy it returns: No matches found.

Can you take a look at it? Or tell me what I am doing wrong?

Thanks for all your hard work!

Correct Babes search results

I noticed my implementation of search results from Babes.com brings in a lot of extra things. I should be able to limit that to just videos results...

Hardcoding missing (and unchanging) data

A few of the new sites I've added don't have genre or category lists to accompany their videos, so the genres have to be manually generated. A couple of mine are pretty sparse and can be improved. For example, count the number of "Actors" and add "1 on 1", "Threesome", "Foursome", etc. genres where appropriate...

EvilAngel problem

File "/var/db/plexdata/Plex Media Server/Plug-ins/PhoenixAdult.bundle/Contents/Code/siteEvilAngel.py", line 54, in update
metadata.director = directedBy.strip()

Tonights Girlfriend Support

Hi, great work here. Thx a lot so far.

Could you possibly also add support for https://www.tonightsgirlfriend.com site? Within the Naughty America Site is only a selection called "Tonights Girlfriend Classic", which only covers older releases.

Thx.

Private genres don't pull correctly

In the example I tested, I got a genre of "Accept" and nothing more. Might also check on the images while in the code; it only pulled one poster and no bg...

Missing Pure Taboo art

I'm missing the Art of Pure Taboo video's, can thuis be forced?

Blacked seems to have an error

`2019-01-14 16:06:59,609 (70eff460) : INFO (init:64) - MEDIA TITLE Blacked Bunny
2019-01-14 16:06:59,616 (70eff460) : INFO (init:64) - Getting Search Settings for: Blacked Bunny
2019-01-14 16:06:59,619 (70eff460) : INFO (PAsearchSites:429) - ^^^^^^^4
2019-01-14 16:06:59,622 (70eff460) : INFO (PAsearchSites:431) - ^^^^^^^ Shortening Title
2019-01-14 16:06:59,624 (70eff460) : INFO (PAsearchSites:432) - blacked vs blacked
2019-01-14 16:06:59,627 (70eff460) : INFO (PAsearchSites:435) - 1
2019-01-14 16:06:59,629 (70eff460) : INFO (PAsearchSites:481) - 6
2019-01-14 16:06:59,632 (70eff460) : INFO (init:66) - Site ID: 1
2019-01-14 16:06:59,634 (70eff460) : INFO (init:67) - Search Title: Bunny
2019-01-14 16:06:59,637 (70eff460) : INFO (init:76) - Bunny
2019-01-14 16:06:59,647 (70eff460) : DEBUG (networking:166) - Requesting 'https://www.blacked.com/search?q=Bunny'
2019-01-14 16:07:00,007 (70eff460) : INFO (siteBlacked:7) - EXCLUSIVE Can't Be Contained 8.0Bunny Colby & Louie SmallsJanuary 10, 2019
2019-01-14 16:07:00,011 (70eff460) : INFO (siteBlacked:9) - Result Title: Can't Be Contained
2019-01-14 16:07:00,014 (70eff460) : INFO (siteBlacked:12) - ID: cant-be-contained
2019-01-14 16:07:00,163 (70eff460) : DEBUG (runtime:88) - Sending packed state data (23888 bytes)
2019-01-14 16:07:00,166 (70eff460) : DEBUG (runtime:924) - Response: [200] str, 692 bytes
2019-01-14 16:07:02,871 (70eff460) : DEBUG (runtime:717) - Handling request GET /:/plugins/com.plexapp.agents.phoenixadult/messaging/function/X0FnZW50S2l0OlVwZGF0ZU1ldGFkYXRh/Y2VyZWFsMQoxCmxpc3QKMApyMAo/Y2VyZWFsMQoxCmRpY3QKMTAKczIKZW5zNApsYW5nYjFzNQpmb3JjZWIwczgKcGV$
2019-01-14 16:07:02,879 (70eff460) : DEBUG (runtime:49) - Received packed state data (21252 bytes)
2019-01-14 16:07:02,948 (70eff460) : DEBUG (runtime:814) - Found route matching /:/plugins/com.plexapp.agents.phoenixadult/messaging/function/X0FnZW50S2l0OlVwZGF0ZU1ldGFkYXRh/Y2VyZWFsMQoxCmxpc3QKMApyMAo_/Y2VyZWFsMQoxCmRpY3QKMTAKczIKZW5zNApsYW5nYjFzNQpmb3JjZWIwczgKcGV$
2019-01-14 16:07:02,955 (70eff460) : DEBUG (model:32) - Loading model with GUID com.plexapp.agents.phoenixadult://_cant-be-contained|1?lang=en
2019-01-14 16:07:02,959 (70eff460) : DEBUG (model:234) - Deserializing from /home/pi/Library/Application Support/Plex Media Server/Metadata/Movies/b/1e360f546c7ad80923fbf6d33fad9035386c335.bundle/Contents/com.plexapp.agents.phoenixadult/Info.xml
2019-01-14 16:07:02,968 (70eff460) : DEBUG (networking:166) - Requesting 'http://127.0.0.1:32400/library/metadata/14519/tree'
2019-01-14 16:07:03,042 (70eff460) : INFO (init:406) - UPDATE CALLED
2019-01-14 16:07:03,045 (70eff460) : INFO (init:409) - 1
2019-01-14 16:07:03,054 (70eff460) : DEBUG (networking:166) - Requesting 'https://www.blacked.com/cant-be-contained'
2019-01-14 16:07:04,361 (70eff460) : CRITICAL (agentkit:1078) - Exception in the update function of agent named 'PhoenixAdult', called with guid 'com.plexapp.agents.phoenixadult://_cant-be-contained|1?lang=en' (most recent call last):
File "/usr/lib/plexmediaserver/Resources/Plug-ins-cc260c476/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/api/agentkit.py", line 1076, in _update
agent.update(obj, media, lang, **kwargs)
File "/home/pi/Library/Application Support/Plex Media Server/Plug-ins/PhoenixAdult.bundle/Contents/Code/init.py", line 416, in update
metadata = PAsearchSites.siteBlacked.update(metadata,siteID,movieGenres,movieActors)
File "/home/pi/Library/Application Support/Plex Media Server/Plug-ins/PhoenixAdult.bundle/Contents/Code/siteBlacked.py", line 58, in update
paragraph = detailsPageElements.xpath('//meta[@content="description"]')[0].get('content')
File "/usr/lib/plexmediaserver/Resources/Plug-ins-cc260c476/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/code/sandbox.py", line 108, in
getitem = lambda x, y: x.getitem(y),
IndexError: list index out of range

2019-01-14 16:07:04,373 (70eff460) : DEBUG (model:229) - Serializing to /home/pi/Library/Application Support/Plex Media Server/Metadata/Movies/b/1e360f546c7ad80923fbf6d33fad9035386c335.bundle/Contents/com.plexapp.agents.phoenixadult/Info.xml
2019-01-14 16:07:04,487 (70eff460) : DEBUG (runtime:88) - Sending packed state data (21284 bytes)
2019-01-14 16:07:04,490 (70eff460) : DEBUG (runtime:924) - Response: [200] str, 16 bytes
`

[bug] TrueAnal matches not working

When trying to match using any of these:

TrueAnal - Back in Gia's Ass
TrueAnal - Back in Gias Ass
True Anal - Back in Gia's Ass
True Anal - Back in Gias Ass

No results are ever found. Trying to use the search function on trueanal.com's site also fails, however, to find anything. There are only search results found if you let the site do auto-search while typing in the search bar. Putting in a search term, hitting return, never seems to find anything via:

https://tour.trueanal.com/search/back%20in%20gia's%20ass

So it seems like the search url used originally just isn't functional anymore.

Consolidate similar site code

Many of the sites share such similar layouts because they are all published by the same studio; a solid example is PornFidelity/TeenFidelity/KellyMadison. Those could pretty easily be combined into a single python file, instead of 3 separate files taking up 3 times the space, and having to make changes in 3 different files. Figure out how to combine them (and others) to consolidate the number of files left to edit.

How to know if a Site needs "studio - title" or "studio - date - models" ?

As this agent is pretty picky about file naming structure (title vs model searches), is there an easy way I can check which studio search requires which sort of name?

For background, I have a post processing script that renames downloaded media into something that PA should be able to process better, however, I need to account for the different naming conventions. Is there a master list I can reference that's up to date?

Thanks!

Work on consistency across search results

Right now it's the wild west in the search results. Try to make a common theme to all the results. I'm open to suggestions, but current thinking is something like this:

Name of the Video [Studio / Subsite] YYYY-MM-DD

First example: Brazzers results can easily add subsite and release date, Reality Kings can add subsite; will have to check on Release date for this one...

TeenFidelity broken? Malformed html in cache maybe?

I've got a file named: TeenFidelity - 2018-11-22 - Autumn Falls Falls In Lust.mp4

Despite all my combinations of date and title and model, nothing returns.

Searching pornfidelity.com for 'Autumn' turns up the result.

Looking in the agent log I see:

2018-12-11 06:01:31,972 (7fac31607700) :  DEBUG (networking:161) - Fetching 'https://www.pornfidelity.com/episodes/?search=%20Autumn%20Falls' from the HTTP cache

Which looks good.

That ends up returning the html:

2018-12-11 06:01:31,975 (7fac31607700) :  INFO (siteTeenFidelity:8) - <!DOCTYPE html>
<html lang="en">

<head>
        <!-- Google Tag Manager -->
    <script>
        (function (w, d, s, l, i) {
            w[l] = w[l] || [];
            w[l].push({
                'gtm.start': new Date().getTime(),
                event: 'gtm.js'
            });
            var f = d.getElementsByTagName(s)[0],
                j = d.createElement(s),
                dl = l != 'dataLayer' ? '&l=' + l : '';
            j.async = true;
            j.src =
                'https://www.googletagmanager.com/gtm.js?id=' + i + dl;
            f.parentNode.insertBefore(j, f);
        })(window, document, 'script', 'dataLayer', 'GTM-WQZ3KR6');
    </script>
    <!-- End Google Tag Manager -->

    <title>The Official PornFidelity Website - Home of Kelly Madison and Ryan Madison</title>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=Edge,chrome=1" />
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="csrf-token" content="qH4uFXenhfa3OpJJtdXowqyUA1WdBagbF7IzwPcs">
<meta name="_token" content="qH4uFXenhfa3OpJJtdXowqyUA1WdBagbF7IzwPcs" />

<!-- icons -->
<link rel="apple-touch-icon" sizes="180x180" href="https://www.pornfidelity.com/apple-touch-icon.png">
<link rel="icon" type="image/png" href="https://www.pornfidelity.com/favicon-32x32.png" sizes="32x32">
<link rel="icon" type="image/png" href="https://www.pornfidelity.com/favicon-16x16.png" sizes="16x16">
<link rel="manifest" href="https://www.pornfidelity.com/manifest.json">
<link rel="mask-icon" href="https://www.pornfidelity.com/safari-pinned-tab.svg" color="#5bbad5">

<!-- http://ogp.me/ -->
<meta property="og:title" content="Porn Fidelity" />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://members.kellymadisonmedia.com/" />
<meta property="og:image" content="https://www.pornfidelity.com/img/share/share_1024x1024.jpg" />
<meta property="og:description" content="Members Area - Porn Fidelity : Kelly Madison : Teen Fidelity" />
<meta property="fb:app_id" content="240289309659979" />

<!-- CSS -->
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,400,500,700|Material+Icons">
<link rel="stylesheet" type="text/css" href="https://tour-cdn.kellymadisonmedia.com/content/promos/promos.css" />
<link rel="stylesheet" href="https://unpkg.com/[email protected]/dist/css/bootstrap-material-design.min.css"
    integrity="sha384-wXznGJNEXNG1NFsbm0ugrLFMQPWswR3lds2VeinahP8N0zJw9VWSopbjv2x7WCvX" crossorigin="anonymous">
<link rel="stylesheet" href="/css/jquery.bxslider.min.css">
<link rel="stylesheet" href="https://vjs.zencdn.net/6.2.8/video-js.css">
<link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/gh/kenwheeler/[email protected]/slick/slick.css" />
<link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/gh/kenwheeler/[email protected]/slick/slick-theme.css" />
<link rel="stylesheet" href="/build/css/app-41f75d79f0.css">
<!--
<link rel="stylesheet" href="/css/bootstrap-notifications.css">
-->

<!-- JavaScript -->
<script src="https://code.jquery.com/jquery-3.2.1.min.js" crossorigin="anonymous"></script>
<script src="//code.jquery.com/ui/1.12.1/jquery-ui.js"></script>


<script src="https://unpkg.com/[email protected]/dist/umd/popper.js" integrity="sha384-fA23ZRQ3G/J53mElWqVJEGJzU0sTs+SvzG8fXVWP+kJQ1lwFAOkcUOysnlKJC33U"
    crossorigin="anonymous"></script>
<script src="https://unpkg.com/[email protected]/dist/js/bootstrap-material-design.js" integrity="sha384-CauSuKpEqAFajSpkdjv3z9t8E7RlpJ1UP0lKM/+NdtSarroVKu069AlsRPKkFBz9"
    crossorigin="anonymous"></script>
<script src="https://cdn.rawgit.com/FezVrasta/snackbarjs/1.1.0/dist/snackbar.min.js"></script>
<script src="https://vjs.zencdn.net/ie8/1.1.2/videojs-ie8.min.js"></script>

<!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->

<script>
    $(document).ready(function () {
        $('body').bootstrapMaterialDesign();
    });
</script>
</head>
<body class="dark pf ">
        <!-- Google Tag Manager (noscript) -->
    <noscript>
        <iframe src="https://www.googletagmanager.com/ns.html?id=GTM-WQZ3KR6" height="0" width="0" style="display:none;visibility:hidden"></iframe>
    </noscript>
    <!-- End Google Tag Manager (noscript) -->

        <div id="main-nav-placeholder" class="mx-auto "></div>

    <div class="container-fluid  pt-2 pt-lg-4 " style="width: 99% !important">


            <nav id="main-nav" class="navbar navbar-expand-lg fixed-top navbar-dark mx-auto justify-content-center flex-column">



    <div class="d-flex flex-wrap flex-lg-nowrap align-items-center w-100" style="max-width: 1400px;">
                <a class="navbar-brand px-1 px-md-2 mr-0 mr-md-2" href="/?nats=MC4wLjMuNTguMC4wLjAuMC4w">
            <span class="porn">PORN</span>
            <span class="fidelity">FIDELITY</span>
            <img src="/img/transparent_4k.png" class="fourk align-text-bottom  d-none d-md-inline-block ml-2"/>
        </a>
                <button class="navbar-toggler position-relative" type="button" data-toggle="collapse" data-target="#top-menu"
                aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
            <span class="navbar-toggler-icon"></span>
        </button>
        <div class="collapse w-100 justify-content-end navbar-collapse" id="top-menu">
            <ul class="navbar-nav nav">
                <li class="nav-item ">
                    <a class="nav-link" href="https://www.pornfidelity.com?nats=MC4wLjMuNTguMC4wLjAuMC4w">Home</a>
                </li>
                <li class="star d-none d-lg-inline-block">
                    <i class="material-icons">star_border</i>
                </li>
                <li class="nav-item ">
                    <a class="nav-link" href="https://www.pornfidelity.com/models?nats=MC4wLjMuNTguMC4wLjAuMC4w">Models</a>
                </li>
                <li class="star d-none d-lg-inline-block">
                    <i class="material-icons">star_border</i>
                </li>
                <li class="nav-item">
                    <a class="nav-link" href="https://www2.pornfidelity.com/signup/signup.php?nats=MC4wLjMuNTguMC4wLjAuMC4w&amp;step=2">Live Cam</a>
                </li>
                <li class="star d-none d-lg-inline-block">
                    <i class="material-icons">star_border</i>
                </li>
                <li class="nav-item dropdown ">
                    <a class="nav-link dropdown-toggle" href="#" id="nav-extras" data-toggle="dropdown"
                       aria-haspopup="true" aria-expanded="false">
                        Extras
                    </a>
                    <div class="dropdown-menu" aria-labelledby="nav-extras">
                                                <a class="dropdown-item" href="https://www2.pornfidelity.com/signup/signup.php?nats=MC4wLjMuNTguMC4wLjAuMC4w&amp;step=2">
                            KellyMadison.com<br/>
                            <small class="w-100 text-success  mt-2">Free Full Access</small>
                        </a>
                                                                                                <a class="dropdown-item" href="https://www2.pornfidelity.com/signup/signup.php?nats=MC4wLjMuNTguMC4wLjAuMC4w&amp;step=2">
                            TeenFidelity.com<br/>
                            <small class="w-100 text-success mt-2">Free Full Access</small>
                        </a>
                                                <a class="dropdown-item" href="http://kellymadisonstore.com" target="_blank">Store</a>
                        <a class="dropdown-item" href="http://thekellyblog.com" target="_blank">Kelly's Blog</a>
                        <a class="dropdown-item" href="https://www2.pornfidelity.com/signup/signup.php?nats=MC4wLjMuNTguMC4wLjAuMC4w&amp;step=2">BTS Pics</a>
                        <a class="dropdown-item" href="https://www2.pornfidelity.com/signup/signup.php?nats=MC4wLjMuNTguMC4wLjAuMC4w&amp;step=2">BTS Vids</a>
                        <a class="dropdown-item" href="https://www2.pornfidelity.com/signup/signup.php?nats=MC4wLjMuNTguMC4wLjAuMC4w&amp;step=2">
                            Bonus Videos<br/>
                            <small class="w-100 text-success mt-2">100,000+ Videos</small>
                        </a>
                        <a class="dropdown-item" href="https://www2.pornfidelity.com/signup/signup.php?nats=MC4wLjMuNTguMC4wLjAuMC4w&amp;step=2">Favorites</a>
                        <a class="dropdown-item" href="https://www2.pornfidelity.com/signup/signup.php?nats=MC4wLjMuNTguMC4wLjAuMC4w&amp;step=2">Stats</a>
                        <a class="dropdown-item" href="https://www2.pornfidelity.com/signup/signup.php?nats=MC4wLjMuNTguMC4wLjAuMC4w&amp;step=2">Polls</a>
                        <a class="dropdown-item" href="https://twitter.com/imkellymadison" target="_blank">KM
                            Twitter</a>
                        <a class="dropdown-item" href="https://twitter.com/teenfidelity" target="_blank">TF Twitter</a>
                    </div>
                </li>
                            </ul>
        </div>
        <a href="https://www2.pornfidelity.com/signup/signup.php?nats=MC4wLjMuNTguMC4wLjAuMC4w&amp;step=2" class="btn btn-pf mb-0 ml-1 ml-md-3">
            Join Now!
        </a>
        <a href="https://members.kellymadisonmedia.com" target="_blank" class="btn btn-pf mb-0 ml-1 ml-md-2">
            Members <span class="d-none d-md-inline-block"> Only</span>
        </a>
    </div>
</nav>



        <div class="col col-12 d-flex mb-lg-4 mb-0 justify-content-center align-items-middle">
                                                <div style="position: relative;">
                                                <img src="https://tour-cdn.kellymadisonmedia.com/content/hero/video_slide/2/welcome3.png" class="img-fluid" style="position: absolute;" />
                                                <div>
                            <video onloadeddata="this.play();" style="width: 100%" preload="auto" playsinline="playsinline" autoplay="autoplay" loop="loop" muted="muted">
                                <source src="https://tour-cdn.kellymadisonmedia.com/content/hero/video_slide/2/hero_banner_final.mp4" type="video/mp4">
                            </video>
                        </div>
                    </div>
                                </div>

    <div id="episodes-toolbar" class="row align-items-center mt-1">
    <div id="filtercol" class="col-auto">
        <button class="navbar-toggler align-middle" style="height: 100%" type="button" data-toggle="drawer"
                data-target="#dw-s1">
            <span class="sr-only">Toggle drawer</span>
            <i class="material-icons">filter_list</i>
            <span id="active-filters-count" class="badge badge-orange d-none"
                  style="position: absolute; top: 12px; right: 12px; font-size: 60%;">4
            </span>
        </button>
    </div>
    <div id="sitecol" class="col">
        <ul class="nav nav-tabs nav-justified justify-content-center" id="update-nav">
            <li class="nav-item kmm">
                <a class="nav-link py-2 py-lg-4 filter-site active" href="/"
                   data-site="0">
                    <span class="all">ALL</span>
                    <span class="fidelity d-none d-lg-inline-block">UPDATES</span>
                </a>
            </li>
            <li class="nav-item km">
                <a class="nav-link py-2 py-lg-4 filter-site " href="/videos/km"
                   data-site="1">
                    <span class="kelly d-lg-none">KM</span>
                    <span class="kelly d-none d-lg-inline-block">KELLY</span>
                    <span class="fidelity d-none d-lg-inline-block">MADISON</span>
                </a>
            </li>
            <li class="nav-item pf">
                <a class="nav-link py-2 py-lg-4 filter-site " href="/videos/pf"
                   data-site="2">
                    <span class="porn d-lg-none">PF</span>
                    <span class="porn d-none d-lg-inline-block">PORN</span>
                    <span class="fidelity d-none d-lg-inline-block">FIDELITY</span>
                </a>
            </li>
            <li class="nav-item tf">
                <a class="nav-link py-2 py-lg-4 filter-site " href="/videos/tf"
                   data-site="3">
                    <span class="teen d-lg-none">TF</span>
                    <span class="teen d-none d-lg-inline-block">TEEN</span>
                    <span class="fidelity d-none d-lg-inline-block">FIDELITY</span>
                </a>
            </li>
        </ul>
    </div>
    <div id="searchcol" class="col-auto">
        <div class="row searchbox" style="height: 100%">
            <div class="col p-0">
                <input type="search" placeholder="Search......" name="search" value=" Autumn Falls"
                       class="searchbox-input form-control form-control-lg p-0">
            </div>
            <div class="col-auto">
                <button class="searchbox-icon navbar-toggler align-middle" style="height: 100%" type="button">
                    <span class="sr-only">Toggle search</span>
                    <i class="material-icons search">search</i>
                    <i class="material-icons close">close</i>
                </button>
            </div>
        </div>
    </div>
</div>
    <div class="row mb-4 position-relative">
        <div class="bmd-layout-container bmd-drawer-f-l position-absolute">
    <div id="dw-s1" class="bmd-layout-drawer body-bg pt-3" style="z-index: 800;overflow:hidden;">
        <div class="container">
            <form id="episodes-filter" data-site="0" class="mb-0 pb-0" method="GET" action="https://www.pornfidelity.com/episodes/search">
                <input type="hidden" id="filter-search" name="search" value=" Autumn Falls"/>
                <input type="hidden" id="filter-site" name="site" value="0"/>
                <input type="hidden" id="filter-page" name="page" value="1"/>

                <p class="mb-0 text-center d-none" id="clear-filters">
                    <a href="javascript: clearFilters();" class="btn btn-danger">
                        <i class="material-icons align-middle">close</i>
                        <span class="align-middle">clear filters</span>
                    </a>
                </p>

                <div class="filter-group mb-3">
                    <h5>
                        Sort
                        <a href="javascript: void(0);" class="close pull-right" aria-label="Close" data-toggle="drawer"
                           data-target="#dw-s1">
                            <span aria-hidden="true">&times;</span>
                        </a>
                    </h5>
                    <select name="sort" class="custom-select">
                        <option value="newest" class="is-default"
                                 selected="selected" >Newest first
                        </option>
                        <option value="liked" >Most liked
                        </option>
                        <option value="comments" >Most
                            comments
                        </option>
                    </select>
                </div>
                <div class="filter-group mb-3">
                    <h5>
                        Anal
                    </h5>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control is-default is-all"
                               id="filter-anal" name="anal_all" value="1"
                                checked="checked" >
                        <label class="custom-control-label default" for="filter-anal"   >All</label>
                    </div>
                    <div class="custom-control custom-radio d-block ml-3">
                        <input type="radio" class="custom-control-input form-control" id="filter-anal-no" name="anal"
                               value="no" >
                        <label class="custom-control-label" for="filter-anal-no">No</label>
                    </div>
                    <div class="custom-control custom-radio d-block ml-3">
                        <input type="radio" class="custom-control-input form-control" id="filter-anal-yes" name="anal"
                               value="yes" >
                        <label class="custom-control-label" for="filter-anal-yes">Yes</label>
                    </div>
                </div>
                <div class="filter-group mb-3">
                    <h5>
                        Bust
                    </h5>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control is-default is-all"
                               id="filter-bust" name="bust[]" value="all"
                                checked="checked" >
                        <label class="custom-control-label default" for="filter-bust">All</label>
                    </div>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control" id="filter-enhanced"
                               name="bust[]" value="enhanced"
                               >
                        <label class="custom-control-label" for="filter-enhanced">Enhanced</label>
                    </div>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control" id="filter-natural"
                               name="bust[]" value="natural"
                               >
                        <label class="custom-control-label" for="filter-natural">Natural</label>
                    </div>
                </div>
                <div class="filter-group mb-3">
                    <h5>
                        Cumshot
                    </h5>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control is-default is-all"
                               id="filter-cumshot" name="cumshot[]" value="all"
                                checked="checked" >
                        <label class="custom-control-label default" for="filter-cumshot">All</label>
                    </div>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control" id="filter-face"
                               name="cumshot[]" value="face"
                               >
                        <label class="custom-control-label" for="filter-face">Face</label>
                    </div>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control" id="filter-swallow"
                               name="cumshot[]" value="swallow"
                               >
                        <label class="custom-control-label" for="filter-swallow">Swallow</label>
                    </div>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control" id="filter-creampie"
                               name="cumshot[]" value="cream_pie"
                               >
                        <label class="custom-control-label" for="filter-creampie">Cream Pie</label>
                    </div>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control" id="filter-tits"
                               name="cumshot[]" value="tits"
                               >
                        <label class="custom-control-label" for="filter-tits">Tits</label>
                    </div>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control" id="filter-eyes"
                               name="cumshot[]" value="eyes"
                               >
                        <label class="custom-control-label" for="filter-eyes">Eyes</label>
                    </div>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control" id="filter-cumswap"
                               name="cumshot[]" value="cum_swap"
                               >
                        <label class="custom-control-label" for="filter-cumswap">Cum Swap</label>
                    </div>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control" id="filter-multiple"
                               name="cumshot[]" value="multiple"
                               >
                        <label class="custom-control-label" for="filter-multiple">Multiple</label>
                    </div>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control" id="filter-other"
                               name="cumshot[]" value="other"
                               >
                        <label class="custom-control-label" for="filter-other">Other</label>
                    </div>
                </div>
                <div class="filter-group mb-3">
                    <h5>
                        Ethnicity
                    </h5>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control is-default is-all"
                               id="filter-ethnicity" name="ethnicity[]" value="all"
                                checked="checked" >
                        <label class="custom-control-label default" for="filter-ethnicity">All</label>
                    </div>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control" id="filter-asian"
                               name="ethnicity[]" value="4"
                               >
                        <label class="custom-control-label" for="filter-asian">Asian</label>
                    </div>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control" id="filter-black"
                               name="ethnicity[]" value="1"
                               >
                        <label class="custom-control-label" for="filter-black">Black</label>
                    </div>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control" id="filter-caucasian"
                               name="ethnicity[]" value="2"
                               >
                        <label class="custom-control-label" for="filter-caucasian">Caucasian</label>
                    </div>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control" id="filter-exotic"
                               name="ethnicity[]" value="5"
                               >
                        <label class="custom-control-label" for="filter-exotic">Exotic</label>
                    </div>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control" id="filter-latin"
                               name="ethnicity[]" value="3"
                               >
                        <label class="custom-control-label" for="filter-latin">Latin</label>
                    </div>
                </div>
                <div class="filter-group mb-3">
                    <h5>
                        Participants
                    </h5>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control is-default is-all"
                               id="filter-participants" name="participants[]" value="all"
                                checked="checked" >
                        <label class="custom-control-label default" for="filter-participants">All</label>
                    </div>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control" id="filter-kelly-ryan"
                               name="participants[]" value="1"
                               >
                        <label class="custom-control-label" for="filter-kelly-ryan">Kelly & Ryan</label>
                    </div>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control" id="filter-kelly"
                               name="participants[]" value="2"
                               >
                        <label class="custom-control-label" for="filter-kelly">Kelly
                            <small>(only)</small>
                        </label>
                    </div>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control" id="filter-ryan"
                               name="participants[]" value="3"
                               >
                        <label class="custom-control-label" for="filter-ryan">Ryan
                            <small>(only)</small>
                        </label>
                    </div>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control" id="filter-foursome"
                               name="participants[]" value="4"
                               >
                        <label class="custom-control-label" for="filter-foursome">Foursome
                            <small>(or more)</small>
                        </label>
                    </div>
                    <div class="custom-control custom-checkbox d-block ml-3">
                        <input type="checkbox" class="custom-control-input form-control" id="filter-participants-other"
                               name="participants[]" value="5"
                               >
                        <label class="custom-control-label" for="filter-participants-other">Other</label>
                    </div>
                </div>
            </form>
        </div>
    </div>
</div>        <div class="loader text-center">
            <div class="spinner">Loading...</div>
        </div>
        <div id="episodes-list" class="w-100" style="min-height: 250px;"></div>
    </div>






            <div class="row">
    <div class="col-12 text-center">
        <a href="https://www2.pornfidelity.com/signup/signup.php?nats=MC4wLjMuNTguMC4wLjAuMC4w&amp;step=2" class="btn btn-pf btn-xlg mb-4 mt-2 py-3 py-lg-4 px-lg-5">
        Join PornFidelity.com Now!
    </a>    </div>
</div>
<footer class="row justify-content-center px-0 py-3 px-lg-3 mb-4">
    <div class="col-12 text-center pt-4 pb-2">
        <a href="https://www.pornfidelity.com">Home</a>
        <i class="material-icons star">star_border</i>
        <a href="https://www.pornfidelity.com/models">Models</a>
        <i class="material-icons star">star_border</i>
        <a href="https://members.kellymadisonmedia.com">Member's Area</a>
        <i class="material-icons star">star_border</i>
        <a href="https://members.kellymadisonmedia.com/support">Support</a>
        <i class="material-icons star">star_border</i>
        <a href="https://www.pornfidelity.com/legal ">Terms</a>
        <i class="material-icons star">star_border</i>
        <a href="https://members.kellymadisonmedia.com/support">Contact Us</a>
    </div>
    <div class="col-sm-8 text-center border-top pt-4 mt-2">
        <a href="/2257" class="twotwofiveseven">18 U.S.C. 2257 Record-Keeping Requirements Compliance Statement</a>
                 | <a href="https://www.kellycash.com" class="twotwofiveseven">Webmasters Earn $$$</a>
                <p class="text-really-really-muted text-smaller mt-2">©1999-2018 Kelly Madison Media. All Rights Reserved.</p>
    </div>
</footer>

<!-- Return to Top -->
<a href="javascript:" id="return-to-top"><i class="material-icons">keyboard_arrow_up</i></a>


    </div>

    <script src="https://www.pornfidelity.com/js/ie10-viewport-bug-workaround.js"></script>
<script src="/build/js/app-6f054f71e2.js"></script>
<script src="https://www.pornfidelity.com/js/masonry.min.js"></script>
<script src="https://www.pornfidelity.com/js/js.cookie.js"></script>
<script src="/js/jquery.bxslider.min.js"></script>
<script type="text/javascript" src="//cdn.jsdelivr.net/gh/kenwheeler/[email protected]/slick/slick.min.js"></script> <script type="text/javascript">
    $('.js-btn-newsletter-signup').click(function (e) {
        e.preventDefault();
        if (!$('#emailInput').val()) {
            $('#subscribeform').after('<p class="alert alert-danger mt-2">Please enter an email address</p>');
            setTimeout(function () {
                $('#subscribeform').parent().find(".alert").fadeOut();
            }, 2000);
        } else {
            $('#subscribeform').submit();
        }
    });
</script>

<!-- Special Promotions -->
<script>
    var natsCode = 'MC4wLjMuNTguMC4wLjAuMC4w';
    var joinUrl = 'https://www2.pornfidelity.com/signup/signup.php?nats=MC4wLjMuNTguMC4wLjAuMC4w&amp;step=2';
</script>
<script src="/promo.js?nats=MC4wLjMuNTguMC4wLjAuMC4w" async></script>
<script type="text/javascript">
// ===== Scroll to Top ====
$(window).scroll(function() {
    if ($(this).scrollTop() >= 50) {        // If page is scrolled more than 50px
        $('#return-to-top').fadeIn(200);    // Fade in the arrow
    } else {
        $('#return-to-top').fadeOut(200);   // Else fade out the arrow
    }
});

$('#return-to-top').click(function() {      // When arrow is clicked
    $('body,html').animate({
        scrollTop : 0                       // Scroll to top of body
    }, 500);
});
</script>
 <script>
    $(document).hoverIntent({
        over: function() {
            playVideoGif($(this));
        },
        out: function() {
            $(this).parent().parent().removeClass('play loading').find('video').get(0).pause();
        },
        selector: '.card.episode .card-video:not(.mobile) a.card-link',
        interval: 150
    });

    function playVideoGif($ele) {
        var $cv =  $ele.parent().parent();
        var $vid = $cv.find('video').get(0);
        $vid.onplaying = function() {
            $cv.addClass('play').removeClass('loading');
        };
        $cv.addClass('loading');
        $vid.play();
    }

    function togglePreviewVideo(cv) {
        var $cv = $(cv);
        var $vid = $cv.find('video').get(0);



        if($cv.hasClass('playforced')) {
            $cv.removeClass('playforced loading');
            $vid.pause();
        }
        else {
            $cv.addClass('playforced');
            $vid.onplaying = function() {
                $cv.addClass('playforced').removeClass('loading');
            };
            $cv.addClass('loading');
            $vid.play();
        }
    }
</script></body>
</htm

Interesting that the "l>" is missing from the end of the document. Is is possible that because the html is malformed, it's interfering with the lxml parsing?

Nightly pipeline for quality assurance

Hi,
since all of this whole work depends strictly on the DOM of the network sites, have you ever minded about some nightly pipeline which runs every day (sorry, night) and triggers some test units in order to properly check if some website has changed?

I guess that this could be helpful to avoid random failures.

You could pick 3 or 4 scenes for each network and assert that some selectors work properly by returning the expected results, if not, the pipeline fails and sends you the notification.

HardX problem

CRITICAL (agentkit:1078) - Exception in the update function of agent named 'PhoenixAdult', called with guid 'com.plexapp.agents.phoenixadult://_en_video_Super-Cute-Workout-Scene-01_131927|278?lang=en' (most recent call last):File "/usr/pbi/plexmediaserver-amd64/share/plexmediaserver/Resources/Plug-ins-9d51fdfaa/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/api/agentkit.py", line 1076, in _update
agent.update(obj, media, lang, **kwargs)
File "/var/db/plexdata/Plex Media Server/Plug-ins/PhoenixAdult.bundle/Contents/Code/init.py", line 496, in update
metadata = PAsearchSites.siteHardX.update(metadata,siteID,movieGenres)
File "/var/db/plexdata/Plex Media Server/Plug-ins/PhoenixAdult.bundle/Contents/Code/siteHardX.py", line 45, in update
tagline = detailsPageElements.xpath('//a[@Class="dvdLink "]')[0].get('title')
File "/usr/pbi/plexmediaserver-amd64/share/plexmediaserver/Resources/Plug-ins-9d51fdfaa/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/code/sandbox.py", line 108, in
getitem = lambda x, y: x.getitem(y),
IndexError: list index out of range