Automatic File System Import, Watched Folders & more

When we upload a pre-recorded show (Track Type=PRS) we put the show name in the “Album” field (and show host in the “Creator” field). It would be good to have Serge’s warning on different fields, depending on “Track Type”.

F’rinstance, we absolutely need “Genre” on all MUS track types, but not on the PRS track types.

I’ve created a feature request in Github: https://github.com/LibreTime/libretime/issues/1129

–Bob.

So i’m going to re activate this topic with a Solution that just works.

It is not pretty but i’m happy with it, it just does it’s job.
Based on the FTP Upload hook i wrote this PHP Script:

#!/usr/bin/php
<?php
$uploaddir = "/path/to/uploads/folder/"; //Path to folder where files will be uploaded, preferably full path ! (must have / at the end)
$api_key = "YOUR API KEY GOES HERE"; //LibreTime API Key (needed for import)

$dir = scandir($uploaddir);
array_splice($dir, 0, 2);

foreach($dir as $file)
{
 echo "Uploading " . $uploaddir . $file . "\n";
 shell_exec("curl -fL --max-time 30 http://127.0.0.1:80/rest/media -u " . $api_key . ": -X POST -F file=@" . $uploaddir . $file);
}
?>


Just set the 2 needed variables and run it, does what it should.
I have it run via a cronjob (crontab -e Settings: * 1 * * * /path/to/import.php) every hour.

Works like intended and i don’t have to use the Upload via Web UI anymore.

Maybe someone will be happy with it, i’m fairly certain it is a “bad” way of doing it, i could see how to upload the file with PHP Curl but i’m lazy so i did not :crazy_face:

Greetings from Germany :slight_smile:

1 Like

I was having trouble with your php script importing files with spaces or special characters (only tested with single quotes at the moment). I rewrote your script in Python:

import os
from pathlib import Path
import requests
from requests.auth import HTTPAuthBasic


APIKEY = "YOUR_API_KEY"
IMPORTDIR = "/path/to/import/directory"


def import_audio():
    localhost = "http://127.0.0.1:80/rest/media"
    api_key = HTTPBasicAuth(APIKEY, "")

    for files in os.listdir(IMPORTDIR):
        data = {"name": files}
        local_file = {"file": open(os.path.join(IMPORTDIR, files), 'rb')}
        response = request.post(localhost, auth=api_key, data=data, files=local_file)
        print(response.text)


if __name__ == "__main__":
    import_audio()

One thing I’d like this script to do is assign track_type upon import. I tried a few different things, but none worked. I’ll update if I figure it out.

So, I don’t like putting my API key, database username and password in plaintext hardcoded in the script. But this gets me where I wanted to be. I’m confident now, that if I scraped the source for getting config settings from /etc/airtime/airtime.conf I’d be able to make this script into a watchdog-like service.

#!/usr/bin/python3
import os
import requests

import psycopg2

from pathlib import Path
from requests.auth import HTTPBasicAuth


APIKEY = "YOUR_API_KEY" # Should be parsed from config
IMPORTDIR = Path('~/Music').expanduser() # Should be cli argument?
TRACK_ID = 'desired_track_id' # Certainly needs to be a cli argument.


def import_music() -> list:
    # localhost could also be cli argument as this part of the script could be ran from outside the server
    localhost = "http://127.0.0.1:80/rest/media"
    api_key = HTTPBasicAuth(APIKEY, "")

    track_ids = []

    for files in os.listdir(IMPORTDIR):
        data = {"name": files}
        local_file = {"file": open(os.path.join(IMPORTDIR, files), 'rb')}
        response = requests.post(localhost, auth=api_key, data=data, files=local_file)
        resp_data = response.json()
        track_ids.append(data['id'])

    return track_ids


def alter_track_type(track_ids: list, track_type:str):
    psql = psycopg2.connect("dbname='DATABASE' user='USERNAME' host='localhost' password='PASSWORD'")
    cursor = psql.cursor()
    for track_id in track_ids:
        cursor.execute(f"update cc_files set track_type = \'{track_type}\' where id = {track_id}")

    psql.commit()
    cursor.close()
    psql.close()


if __name__ == "__main__":
    alter_track_type(import_music(), TRACK_ID)

While I was writing this, I was worried that I would have to be strict about tagging track_type as there was a point where I thought about just update cc_files set track_type = 'whatever' where track_type = '';. But I was pretty stoked when I realized that that requests.post returns a json, that can be parsed.

So now, in theory, you could setup a directory that ingests files according to rules. Like for instance:

Directory /music_imports/500/
This is a smb file share or whatever…
the cli or cronjob might look like: audio-ingest /music_imports/500 500
these files take the track_type 500…

I think you might be able to also query libretime to see that it has ingest audio properly with like:

def query_libretime(track_id):
    track_url = f"http://127.0.0.1:80/rest/media/{track_id}"
    ingest = requests.get(track_url, auth=api_key)
    data = ingest.json()
    if data['md5'] is not 'null':
        print('Audio has been ingested')

then you can delete the original… maybe rename the file during ingestion… like:

Prince - Raspberry Beret.mp3 # Original file
Prince - Raspberry Beret.mp3.INGEST # has been uploaded, but has not yet been hashed
Prince - Raspberry Beret.mp3.ACCEPTED # has been properly imported and will next be deleted
-- no file # has been deleted since it has been properly imported by libretime...

And in case it has been rejected, no deletion…

Prince - Raspberry Beret.mp3.REJECTED # file remains in the directory and is ignored by further scans

An entry is then made to the log as to why the file has been rejected… But I’m not sure if hash is still compiled regardless of whether or not the file has been properly imported. This will be tested later…

*Note, I’m a little blurry-eyed, so there may be mistakes in my code…

For reference, a bulk import script has already been worked on here feat: create bulk-import tool by jooola · Pull Request #1659 · libretime/libretime · GitHub

Cheers