So I’m finally getting around to understanding what we need to do to implement something akin to a filesystem based import process, but I’ve come to realize there are two distinct functionalities and I’m wanting feedback from users as to what would be the most useful for them.
Basically we could write some a python module to setup a folder that when you copy files into it, LibreTime will scan it and import them and delete them from this directory.
The other idea is a watched folder where LibreTime will periodically scan this folder and add any files to the Library but they are still located in this folder on your filesystem and if they are removed from this folder then they are unavailable from libretime. In theory this could also work with network mounted folders etc. This is the functionality that existed in Airtime 2.5. But the way to implement this isn’t as straightforward because it requires a modified import process to add the files to the library rather than using the existing import call. There is also the question of who is the file “owner”. I suppose it would be admin by default.
Basically I’m just seeing if anyone else has ideas or feedback about this.
For ouer needs, the fisrt option is enough… Provided we don’t need to edit the imported files in LibreTime thereafter. For the automatic file import system there a no informations about how save to the meta datas in file we want to import by this process. I’ve tried to find all tags but i had dificulties to find some o them ?
Title ==> {title}
Creator ==> {artist}
Album ==> {Album}
Description ==> ???
Track number ==> {tracknumber}
Genre ==> {genre}
Year ==> {date} or {year}
Label ==> {label}
Composer ==> {composer}
Conductor ==> {conductor}
Mood ==> {mood}
BPM ==> {Bpm}
copyright ==> {Copyright}
ISRC Number ==> ???
Website ==> ???
Language ==> {language}
Encoded by ==> ???
The import process doesn’t currently alter any of the file metadata although that has been something I was considering implementing so that files put in a certain directory automatically get assigned to a specific owner or genre. But this probably won’t happen in the first round of things.
You should be able to edit the metadata inside of the LibreTime UI on a per track basis and access most of those fields or otherwise edit the id3 tags before importing them. Description might exist only in the database. Some other fields such as owner_id are database only as they don’t exist as a id3 tag.
Am I right in reading your message to say that you are looking for a way of overriding the id3 tags and thus LibreTime fields during the import process ?
I further explored the option of importing in place but even after modifying the code to make this process work the core LibreTime system had issues dealing with the files ie couldn’t play or download them. So I’m going to leave this alone for now and work on setting up a python script that monitors a directory on the file system and imports and removes the files after import.
Basically it has a hardcoded in the config file folder of /srv/airtime/stor/uploads where any audio files copied into that folder will be uploaded to your LibreTime install and deleted out of that directory.
This folder will be setup upon install and can be changed with minimal effort but requires a little bit of familiarity with the command-line.
The reasoning behind this is that in its current state it could be very destructive if pointed at the wrong directory. For example if someone changed it to /srv/airtime/organized - it could create an endless loop of file shuffling and deletion that would break the system. (In fact I’m going to code against allowing this possibility).
In fact all this does currently is restore a useful property of the way media-monitor the airtime 2.5 worked in that it would import files automatically copied into the organize directory. I am open to suggestions as to additional functionality that could be added.
The first option would be great for our use as well. I thought it was already available, but after playing with it for a while, I determined that I didn’t have quite enough fortitude to fight with the PHP scripts that I thought would do it.
Yeah there were a few shell scripts that accomplished the importing of files but it was outside of the scope of the libretime install to make them work. This PR once fixed will add the service that basically makes a folder that imports files copied into it. It is working I just need to finish up adding an option to make the folder user-defined during install and change the documentation.
Just yesterday our collective discussed how we plan to migrate over to using libretime.
For 16 years the station has always kept our music library organized using folders in the operating system organized by artist and album. This has been far from ideal for various reasons. To my knowledge the automation software we’ve used, like Zara radio and Radit, scan these folders without moving files, which gives a certain security for non-tech folks who have seen buggy automation software come and go, but at least we have a backup of the music library.
My understanding from reading the documentation is that imported files are moved into sub-folders according their metadata. Is this correct?
Metadata in our music library has always been sketchy, especially because we try to give priority to local artists. We’re now in a “cleaning house” stage of selecting what we DO want from the years of accumulated music, and then improving metadata in those files with tools like Puddletag. We’ll need to be well organized and in agreement as a group to ensure that people don’t import files without metadata.
No technology can substitute for strong group consensus, but since this thread asks about needs, features that would help out collective radios include
It sure would be great if the media folder could be a “source of truth” for libretime, ie. if files are added, moved, or deleted from that folder then those changes would be reflected in the libretime interface. We’re currently thinking we’ll network mount our main music folder as you mentioned via sshfs. If however the answer to my file organization question above is yes, then we can teach that workflow in our upcoming trainings.
It would also be great to either alert a user that the files they’re importing don’t contain metadata, or even to include an optional step in the import process where users could review and add metadata.
Hey Ryan,
That part of the documentation is out of date. There is no “watched folder” functionality currently for LibreTime, before I put together this latest PR there was no easy way to import music from a filesystem at all, other than a few shell scripts. You basically had to manually upload everything through the web interface.
What I put together currently only creates a folder where anything you copy into it will be uploaded to LibreTime and deleted from the filesystem. I tried to implement the watched folder setup through modifying the code a previous contributor had put together but ran into numerous issues where it wouldn’t playback files and basically found that it wasn’t very compatible with the web API. When LibreTime was still the Airtime SaaS-dev branch (aka Airtime.pro) they refactored the code so that it could work with Amazon S3 so that they could put everyones data in a single cloud repository and not have to worry about it. But they left the filesystem as an option and that is what we have been using.
What currently happens to a file as it is imported is that it is moved into /srv/airtime/stor/imported/#/artist/track.file and the # corresponds to the user ID of the person uploading it. Everything will be put under user ID /1/ at the moment when uploaded through the PR I put together. The metadata is what is used to display and search for tracks in LibreTime, so it is pretty important that it be consistent. I believe that when you edit the track title etc it also changes the metadata on the file. But I’m not totally sure.
So I haven’t finished coding my PR. But let me provide some comments on your suggestions.
For the first 1 - I understand what you are proposing here but I’m not sure how easily it could be implemented as the watched folder functionality doesn’t currently exist. Specifically LibreTime stores all of the metadata for its files in the Postgres SQL database, and it creates this through analyzing the file upon import. LibreTime would need to rescan all of the files and see if any of them had changed and then update the database if it detects a change. If a file is deleted or moved it isn’t as straight forward either. So that is an issue.
So this is a challenge for stations using LibreTime and that is how to maintain large databases of music. Another issue with the whole remote filesystem is the source of truth plan would be what happens if someone uploads a file directly to the LibreTime web UI. The way watched folders originally worked is the files normally uploaded would be added to the local filesystem and the watched folders are separate and so nothing is ever added to them but instead just stored on the local filesystem.
The web interface currently doesn’t do any review alert and I’m not totally sure how to best implement this with any kind of filesystem import. Figuring out a way to do this seems like a good idea though.
So one additional feature I was thinking of adding to my code was the ability to scan a folder and import everything in it. This could be used with a remote filesystem but the question becomes how to determine if a file was imported already. There is no duplicate detection currently in LibreTime and so I think this would be essential before we scan a folder and import everything while leaving the files intact. Also my thought is that the files would be copied to LibreTime’s local filesystem rather than it attempting to play files from a sshfs or other network mount. This would help ensure that they are locally available for play even if there is an issue with the network or whatever. But it also makes the duplicate detection aspect of it even more important.
So I’m not sure how this would work AS IS with your current setup but when I get back to working on the PR I’ll definitely try to keep in mind the feedback you gave. I wonder how automated filesystem importing by scanning an existing folder would work for you all. I don’t think there is easy way to keep your existing filesystem as the sole source of truth if you are going to have people using the web UI to uploaded as well. I do think this is an interesting problem to try to solve, and the filesystem/folders created by LibreTime are kind of a mess in comparison to those created by humans with attention to detail.
if we need to just tell everybody not to touch the folder of music, that’s fine. we’ll be doing trainings anyway, so that can just be part of it. we could even set folder permissions to allow access to libretime but prevent the default server desktop user from making changes without sudo. The important thing is that tracks are organized within sub-folders by artist. that means we can make a backup that’s still usable, organize-able.
more soon, gotta run now to go cook dinner for my hungry 3yo and 5yo, somethign i’m sure you’re totally unable to understand i also wanna ask you about how best to contribute documentation in spanish. thank you take care!!!
Hello to all.
I am working to organize our Music Data Base for import in Libretime. We ripped about 39’000 titles since 19 years. Our handicap is that we didn’t used Metadata Tags and all the files are stored in one folder without sub-folders. I want to change this mistake. I have to tag all files. MusicBarainzPicard is very useful for this. After beginning the import I saw that it would be nice to have BPM values in tags to use all advantages of intelligent Playlists . MusicBrainzPicard has plugins for BPM analyzis, but it does not work. I found Piston BPM detector to make BPM tags, but this tool is made for Windows and I have to use it in wine. This solution is not so clean like I wished.
Maybe somebody can upgrade the Libretime import process.
It exist command line tools to analyze BPM value and write this tag into the audio file.
My question is now, will it be possible to make that the importing process in Libretime will do this automatically, when no BPM value exists or when it is = 0 ?
I am a newcomer in Libretime and all what I have tried was very, very nice. Bravo…
I want to promote Libretime for new local non commercialcommunity radio stations.
Thank you for all this nice work.
I have worked on the airtime_analyzer code and it is currently written in python 2.7 - if there is a CLI tool that works under Linux it could probably be added to the import process.
Hello robbt.
I installed bpm tools in my ubuntu 18.04 PC and made first test.
Bpm tools detected BPM of indicated “audio-file” but didn’t write the BPM tag in to the files.
I got the return that it needs sox.
So I installed sox from official bionic archive. https://launchpad.net/ubuntu/bionic/amd64/libsox-fmt-base
But this doesn’t work.
It wasn’t enough.
When I tried to tag my audio mp3 file I got the return that mp3 couldn’t be treated.
The sudo apt-get install libsox-fmt-mp3 was the solution.
Now I can analyze and write with simple command line the BPM value in the mp3 file without changing the other existing tags. Maybe we need to install libsox-fmt-XXX for other .xxx formats too. I think it will be useful to add this function in the audio file import process. This means too that the installation of Libretime should add this tools and their dependencies too and maybe add a bottom to choice this function in Libretime.
Hi, coming back to BPM tools. I think if BPM will be part of analyzing during importing audio files we can increase Libretime. I made some experimentation and found following way.
I installed bpm tools in my ubuntu 18.04 PC and made first test.
Bpm tools detected BPM of indicated “audio-file” but didn’t write the BPM tag in to the files.
I got the return that it needs sox.
So I installed sox from official bionic archive.
But this doesn’t work.
It wasn’t enough.
When I tried to tag my audio mp3 file I got the return that mp3 couldn’t be treated.
The sudo apt-get install libsox-fmt-mp3 was the solution.
Now I can analyze and write with simple command line the BPM value in the mp3 file without changing the other existing tags. Maybe we need to install libsox-fmt-XXX for other .xxx formats too. I think it will be useful to add this function in the audio file import process. This means too that the installation of Libretime should add this tools and their dependencies too and maybe add a bottom to choice this function in Libretime.
Just following up about the mp3 import question. This is a good idea, I don’t think anyone would be opposed to it but I also don’t know if it will be remembered without a feature request on github. Nobody is working on that part of LibreTime too much at the moment but thanks for sharing an idea for a possible upgrade.
i also wanna ask you about how best to contribute documentation in spanish. thank you take care!!!