owen/lms
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Removed AcousticBrainz support. fixes #221

Browse Source
This commit is contained in:
emeric
2023-01-15 15:03:12 +01:00
parent b681a20b5a
commit 69f7b89d33
8 changed files with 19 additions and 214 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+7 -13
View File
@@ -9,7 +9,8 @@ A [demo instance](http://lms-demo.poupon.dev) is available. Note the administrat
## Main features
* Recommendation engine
* Audio transcode for maximum interoperability and low bandwith requirements
* Multi-value tags: `artist`, `genre`, `composer`, `lyricist`, `mood`, `performer`, ...
* Multi-value tags: `genre`, `albumgenre`, `mood`, `albummood`, `albumgrouping`, ...
* Artist relationships: `composer`, `conductor`, `lyricist`, `mixer`, `performer`, `producer`, `remixer`
* [MusicBrainz Identifier](https://musicbrainz.org/doc/MusicBrainz_Identifier) support to handle duplicated artist and release names
* [ListenBrainz](https://listenbrainz.org) support for:
* Scrobbling and synchronizing listens
@@ -22,7 +23,7 @@ A [demo instance](http://lms-demo.poupon.dev) is available. Note the administrat
_LMS_ provides several ways to help you find the music you like:
* Tag-based filters (ex: _Rock_, _Metal_ and _Aggressive_, _Electronic_ and _Relaxed_, ...)
* Recommendations for similar artists and albums
* Radio mode, based on what is in the current playqueue
* Radio mode, with endless filling of the play queue with tracks similar to what is there
* Searches in album, artist and track names (including sort names)
* Starred Albums/Artists/Tracks
* Various tags to help you filter your music: _mood_, _albummood_, _albumgenre_, _albumgrouping_, ...
@@ -33,17 +34,8 @@ _LMS_ provides several ways to help you find the music you like:
* Starred _Jazz_ albums
* ...
The recommendation engine uses two different sources:
1. Tags that are present in the audio files
2. Acoustic similarities of the audio files, using a trained [Self-Organizing Map](https://en.wikipedia.org/wiki/Self-organizing_map)
__Notes on the self-organizing map__:
* training the map requires significant computation time on large collections (ex: half an hour for 40k tracks using a Core i5)
* audio acoustic data is pulled from [AcousticBrainz](https://acousticbrainz.org/). Therefore your audio files _must_ contain the [recording](https://musicbrainz.org/doc/Recording) [MusicBrainz Identifier](https://musicbrainz.org/doc/MusicBrainz_Identifier).
* to enable the audio similarity source, you have to enable it first in the administration panel.
## Subsonic API
The API version implemented is 1.16.0 and has been tested on _Android_ using _Subsonic Player_, _Ultrasonic_ and _DSub_.
The API version implemented is 1.16.0 and has been tested on _Android_ using _Subsonic Player_, _Ultrasonic_, _Symfonium_, and _DSub_.
Since _LMS_ uses metadata tags to organize music, a compatibility mode is used to browse the collection when using the directory browsing commands.
The Subsonic API is enabled by default.
@@ -53,9 +45,11 @@ __Note__: since _LMS_ may store hashed and salted passwords or may forward authe
_LMS_ relies exclusively on tags to organize your music collection.
### Filtering
You can specify the tags you want to be used to filter your collection. By default, `GENRE`, `ALBUMGROUPING`, `MOOD` and `ALBUMMOOD` tags are used.
You can specify the tags you want to use to filter your collection. By default, `genre`, `albumgrouping`, `mood` and `albummood` tags are used.
In the administration panel, you can set whatever tags you want, even custom tags.
__Note__: you can use the `lms-metadata` tool to have an idea of the tags parsed by _LMS_ using [TagLib](https://github.com/taglib/taglib).
### Multiple album artists
_LMS_ requires the `ALBUMARTISTS` and `ALBUMARTISTSSORT` tags to properly handle multiple album artists on the same album. As they are a custom tags, you may need to setup your favorite tagger to add them.
approot/admin-database.xml
+2
View File
@@ -34,6 +34,7 @@
</div>
</div>
<legend>${tr:Lms.Admin.Database.scan-options}</legend>
${<if-has-recommendation-engine>}
<div class="col-lg-6">
<label class="form-label" for="${id:recommendation-engine-type}">
${tr:Lms.Admin.Database.recommendation-engine-type}
@@ -43,6 +44,7 @@
${recommendation-engine-type-info}
</div>
</div>
${</if-has-recommendation-engine>}
<div class="col-12">
<label class="form-label" for="${id:clusters}">
${tr:Lms.clusters}
src/libs/services/scanner/CMakeLists.txt
-1
View File
@@ -1,6 +1,5 @@
add_library(lmsscanner SHARED
impl/AcousticBrainzUtils.cpp
impl/ScannerService.cpp
impl/ScannerStats.cpp
)
src/libs/services/scanner/impl/AcousticBrainzUtils.cpp
-87
View File
@@ -1,87 +0,0 @@
/*
* Copyright (C) 2018 Emeric Poupon
*
* This file is part of LMS.
*
* LMS is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* LMS is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with LMS. If not, see <http://www.gnu.org/licenses/>.
*/
#include "AcousticBrainzUtils.hpp"
#include <boost/property_tree/ptree.hpp>
#include <boost/property_tree/json_parser.hpp>
#include <Wt/WIOService.h>
#include <Wt/Http/Client.h>
#include "utils/IConfig.hpp"
#include "utils/Logger.hpp"
#include "utils/Service.hpp"
#include "utils/UUID.hpp"
namespace AcousticBrainz
{
static
std::string
getJsonData(const UUID& mbid)
{
static constexpr std::string_view defaultAPIURL {"https://acousticbrainz.org"};
const std::string url {std::string {Service<IConfig>::get()->getString("acousticbrainz-api-base-url", defaultAPIURL)} + "/api/v1/" + std::string {mbid.getAsString()} + "/low-level"};
boost::asio::io_service ioService;
Wt::Http::Client client {ioService};
client.setFollowRedirect(true);
client.setSslCertificateVerificationEnabled(true);
client.setMaximumResponseSize(256*1024);
if (!client.get(url))
{
LMS_LOG(DBUPDATER, ERROR) << "Cannot perform a GET request to url '" << url << "'";
return {};
}
std::string response;
client.done().connect([&](Wt::AsioWrapper::error_code ec, const Wt::Http::Message &msg)
{
if (ec)
{
LMS_LOG(DBUPDATER, ERROR) << "GET request to url '" << url << "' failed: " << ec.message();
return;
}
if (msg.status() != 200)
{
LMS_LOG(DBUPDATER, ERROR) << "GET request to url '" << url << "' failed: status = " << msg.status() << ", body = " << msg.body();
return;
}
response = msg.body();
});
ioService.run();
return response;
}
std::string
extractLowLevelFeatures(const UUID& recordingMBID)
{
return getJsonData(recordingMBID);
}
} // namespace Scanner::AcousticBrainz
src/libs/services/scanner/impl/AcousticBrainzUtils.hpp
-30
View File
@@ -1,30 +0,0 @@
/*
* Copyright (C) 2018 Emeric Poupon
*
* This file is part of LMS.
*
* LMS is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* LMS is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with LMS. If not, see <http://www.gnu.org/licenses/>.
*/
#pragma once
#include <string>
class UUID;
namespace AcousticBrainz
{
std::string extractLowLevelFeatures(const UUID& recordingMBID);
}
src/libs/services/scanner/impl/ScannerService.cpp
-81
View File
@@ -38,7 +38,6 @@
#include "utils/Logger.hpp"
#include "utils/Path.hpp"
#include "utils/UUID.hpp"
#include "AcousticBrainzUtils.hpp"
using namespace Database;
@@ -526,7 +525,6 @@ ScannerService::scan(bool forceScan)
if (!_abortScan)
{
checkDuplicatedAudioFiles(stats);
fetchTrackFeatures(stats);
reloadSimilarityEngine(stats);
}
@@ -560,85 +558,6 @@ ScannerService::scan(bool forceScan)
}
}
bool
ScannerService::fetchTrackFeatures(TrackId trackId, const UUID& recordingMBID)
{
std::map<std::string, double> features;
LMS_LOG(DBUPDATER, INFO) << "Fetching low level features for recording '" << recordingMBID.getAsString() << "'";
const std::string data {AcousticBrainz::extractLowLevelFeatures(recordingMBID)};
if (data.empty())
{
LMS_LOG(DBUPDATER, ERROR) << "Track " << trackId.getValue() << ", recording MBID = '" << recordingMBID.getAsString() << "': cannot extract features using AcousticBrainz";
return false;
}
{
auto uniqueTransaction {_dbSession.createUniqueTransaction()};
Track::pointer track {Track::find(_dbSession, trackId)};
if (!track)
return false;
_dbSession.create<TrackFeatures>(track, data);
}
return true;
}
void
ScannerService::fetchTrackFeatures(ScanStats& stats)
{
if (_recommendationServiceType != ScanSettings::RecommendationEngineType::Features)
return;
ScanStepStats stepStats{stats.startTime, ScanProgressStep::FetchingTrackFeatures};
LMS_LOG(DBUPDATER, INFO) << "Fetching missing track features...";
struct TrackInfo
{
TrackId id;
UUID recordingMBID;
};
const auto tracksToFetch {[&]()
{
std::vector<TrackInfo> res;
auto transaction {_dbSession.createSharedTransaction()};
auto trackIds {Track::findWithRecordingMBIDAndMissingFeatures(_dbSession, Range {})};
for (const TrackId trackId : trackIds.results)
{
const Track::pointer track {Track::find(_dbSession, trackId)};
res.emplace_back(TrackInfo {track->getId(), *track->getRecordingMBID()});
}
return res;
}()};
stepStats.totalElems = tracksToFetch.size();
notifyInProgress(stepStats);
LMS_LOG(DBUPDATER, INFO) << "Found " << tracksToFetch.size() << " track(s) to fetch!";
for (const TrackInfo& trackToFetch : tracksToFetch)
{
if (_abortScan)
return;
if (fetchTrackFeatures(trackToFetch.id, trackToFetch.recordingMBID))
stats.featuresFetched++;
stepStats.processedElems++;
notifyInProgressIfNeeded(stepStats);
}
notifyInProgress(stepStats);
LMS_LOG(DBUPDATER, INFO) << "Track features fetched!";
}
void
ScannerService::refreshScanSettings()
{
src/libs/services/scanner/impl/ScannerService.hpp
-2
View File
@@ -77,8 +77,6 @@ namespace Scanner
void scan(bool force);
void scanMediaDirectory( const std::filesystem::path& mediaDirectory, bool forceScan, ScanStats& stats);
bool fetchTrackFeatures(Database::TrackId trackId, const UUID& MBID);
void fetchTrackFeatures(ScanStats& stats);
// Helpers
void refreshScanSettings();
src/lms/ui/admin/DatabaseSettingsView.cpp
+10
View File
@@ -142,6 +142,13 @@ class DatabaseSettingsModel : public Wt::WFormModel
scanSettings.modify()->setClusterTypes(LmsApp->getDbSession(), std::set<std::string>(clusterTypes.begin(), clusterTypes.end()));
}
static ScanSettings::RecommendationEngineType getCurrentRecommendationEngine()
{
auto transaction {LmsApp->getDbSession().createSharedTransaction()};
const ScanSettings::pointer scanSettings {ScanSettings::get(LmsApp->getDbSession())};
return scanSettings->getRecommendationEngineType();
}
private:
void initializeModels()
{
@@ -211,6 +218,9 @@ DatabaseSettingsView::refreshView()
t->setFormWidget(DatabaseSettingsModel::UpdateStartTimeField, std::move(updateStartTime));
// recommendation engine type
// Hide the settings if the engine is set to clusters, as we don't want users to switch to acoustic features (currently broken)
// Otherwise, give the user a way to switch back to clusters
t->setCondition("if-has-recommendation-engine", model->getCurrentRecommendationEngine() == ScanSettings::RecommendationEngineType::Features);
auto recommendationEngineType {std::make_unique<Wt::WComboBox>()};
recommendationEngineType->setModel(model->recommendationEngineTypeModel());
t->setFormWidget(DatabaseSettingsModel::RecommendationEngineTypeField, std::move(recommendationEngineType));