Looking for folk implementing RSS Import


#1

I need to make a solution for this scenario:
1/ Most shows auto-import, but some do not. The ones that do not auto-import can be manually imported every time?
2/ Blank new Playlists are being created, I get about 6 to 10 a day. No fields are filled apart from id.
3/ Logs do not seem to show any errors.
I could go on with more symptoms… Please add yours. I need clues.


#2

This sounds like it could be tied to the dramatic thousands of empty playlists that people have communicated. Most of the time there are errors in airtime-celery.log (when logging is enabled). I think that mutagen or how the RSS is parsed has issues with built-in jpg and misidentifies them as a mp3, but I haven’t had time to confirm this. There was at least one Github issue https://github.com/LibreTime/libretime/issues/271 that had another error related to this and I just haven’t had time to fully follow up on it as far as testing. Please keep us up to date on any resolutions you come to. I’m sorry I haven’t integrated your PR. My time has been severely limited due to needing to focus on another project to get it done.


#3

Heh. Just looked in the download code/simple pie. It grabs the first enclosure. :\ and doesn’t care what sort it is. This would be easily fixed. So I will try filtering the enclosure.


#4

As promised, I have modified /usr/share/airtime/php/airtime_mvc/application/services/PodcastEpisodeService.php so that it now only tries to import audio enclosures. I can now import more feeds than I could.
It is in a pull request here: https://github.com/LibreTime/libretime/pull/464


#5

Side Note: I have been seeing ‘500’ page errors from celery DURING import from an episode click, both with the original code and the new code. Trying to hunt that down. Example of an offending call: https://x-pedia.org/api/poll-celery?_=1520779883342 however, the track does get imported. I will now clear down my blank playlist page and watch…


#6

So far, no blank playlists :slight_smile:


#7

Progress report. I am finding periodic exception Error 500 in the Zend log which seems to be stopping RSS imports.
The exception is ‘Cannot find Podcast’. This seems to be because table ‘imported_podcasts’ is being traversed and it contains references to Podcasts which have been deleted. Thus they are not found. I think that the exception stops the whole import, instead of returning nothing from the query and being handled, and that is the end of the import (and any further ones in the list).
Next, I will write a query to sanitize the table, modify the error handler to report which podcast id it cannot find and see if it then works.


#8

I have also noticed that some podcasts (not episodes) had corrupted records. Symptoms: The main description is blank and fields are shifted one, or sometimes two, fields to the left. I suspect this is because the main title and description fields (and probably other text fields) are not escaped during import. Thus the owner field is empty. I manually corrected these, but it made no difference that I could see.


#9

Ugh, yeah the podcast code is probably rife with little bugs like this. We could try find them by writing tests or through trial and error. Thanks for the update. I have had little time to work on LT recently and have been spending just enough to chime in on whatever bugs people identify.


#10

Further update: I could not understand why it worked on my old server and not on my new one.
It turned out that it was the PHP version. The new one came with PHP5.6 and the old one PHP5.31.
Why was that important? Seems it was how the versions handle Json. 5.3 doesn’t care if it gets a string or array, I guess it casts the string to an array. 5.6 was not playing nicely. If I passed it a string (the original code) it complained and threw an error. If I made it an array it complained in other places that it was expecting a string :\ sounds like my ex-wife. I have spent more than enough time on this, for now, and have other things to do.
My own improved external-code RSS importer seems stable and works fine, so I will keep with that, for the moment. Maybe it could be used as a guide to a possible re-write. It does import the item title, description and more correctly. It also tidies up and makes consistent the track title in a manner suitable for talk shows.


#11


This is what my auto imported tracks panel looks like, using my own code.