mirror/lmms
1
0
Fork 0
mirror of https://github.com/LMMS/lmms.git synced 2026-05-17 11:18:30 -04:00
Code Issues Packages Projects Releases Wiki Activity

Adds support for local paths and project bundles (#5735)

Browse Source 
* Adds a baseDir for the local path

	Adds a new Base for paths called "local:", which will translate to the dir where the currently opened project file is at. In the future this will allow us to make project bundles and make it easier to export and transfer projects to other people without breaking the paths to samples, presets, plugins and others.

* Starts implementing the makeBundle functionality

	For now, to make a bundle LMMS has to be run through CLI with the makeBundle/--makeBundle command, followed by an input file and an output file ('lmms --makeBundle input.mmp output.mmp'). DataFile::writeBundle() is then called. For now, it only saves the mmp/mmpz file normally and also creates a "resources" folder if it doesn't exists. Later it will also manipulate the DataFile so all paths are local and copy all files to the resources folder.

TODO:
	-Remove warnings.
	-Implement the logic to manipulate the DataFile and copy files.

* Starts implementing logic to go through resources

	Starts implementing the logic that will go through all the resources of the project file and add them to the bundle. We use a std::map of QString to std::vector<QString>: The first string is the DOM element tagname that is going to be searched for. The vector of strings holds all attributes this element can have that accesses resources. For now we just print those to the screen.

* Adds logic to copy files and update the project

	The raw logic for creating the bundle is finished. It now copies the resource files and update the project to use "local:" paths to the new file now.
	Now it's a matter of organizing things and adding safety checks for file operation errors basically.

* Makes the writeBundle method more organized

	Improves comments and debugging warnings to make the writeBundle a bit more organized for review.

* Adds a project bundle folder

	Adds a project bundle folder, inside which the bundles will be created. Instead of receiving an output project file name, the makeBundle command now receives a bundle name that will be used as the name of the bundle's folder.
	Uses a typedef for the std::map with the tags and attributes with resources.

TODO:
	- Fix the local: prefix so it works when we don't have the project file open (for CLI usage, or find another way to deal with it).
	- Sanitize the bundle name.
	- Allow overwriting bundles?

* Handles local paths when a project isn't open

	The PathUtil base prefix conversion for "local:" uses the loaded song file name. When we are running the makebundle command from the CLI there isn't a loaded project, so those prefixes aren't converted properly. Now, when there isn't a project open PathUtil will return "local:" again when it tries to convert this base prefix. DataFile can then check if the base prefix is still there, and if it is it knows the conversion wasn't possible, so it does the conversion itself. To do that, a member called m_fileName was added to DataFile, which will hold the file being manipulated if that's where the DataFile originated from, and the local path can be retrieved from it.

* Sanitizes the bundle name

	The bundle name is now sanitized. Since it's going to be used as a folder name, we need to keep the user from giving invalid folder names as a bundle's name. The rules for the name are:
	1) It must start with a word character (either a digit or letter)
	2) It can be followed by any number of letters, digits, whitespaces or hyphens
	3) It must end with a word character (either a digit or letter)

	A Regexp is used to check for the name validity.

* Moves away from projectbundle folder concept

	This commit regresses some functionality. Project bundles will be saved just as any other project, except they will also have the resources folder. It will be up to the user to organize the bundles on their own folders. It's currently not allowed to save a bundle on a folder where there's one already though (if there's a resources folder already). Later it might be allowed to overwrite bundles in that case.
	The projectbundles folder was dropped. The user can save project bundles anywhere in the system.
	The DataFile::writeBundle was removed. It's functionality was merged into the DataFile::writeFile method, by adding a boolean on the parameters defining whether it should be saved with resources. The logic of copying the resource files and changing the paths inside the project DataFile was moved to DataFile::copyResources, making the methods a little bit less dense.

* Adds an option to save project as bundle

	The "Save As" dialog now has an option to save project as a project bundle (with resources), which will save the file as a bundle.

Known bug:
	- Because the "local:" base prefix is translated to the filename from the Engine::getSong(), it breaks when Song::guiSaveProjectAs is called, because that method changes the project name before saving. Urgent fix!

* Fix local: prefix saving bug

	There was a bug where "local:" prefixes weren't resolved properly during saving because Song::guiSaveProjectAs() changed the project name to the destiny file name before saving. This resulted in the local paths using the destination file as a reference. Both Song::guiSaveProject() and Song::guiSaveProjectAs() were rewritten, and now they only rename the project after it's saved.

* Adds a warning message box

	When the user tries to save a project bundle on a folder that already has a project bundle (contains a resources folder) a message box pops up telling the user it's not permitted and that another path should be chosen.

* Removes unused header

	Forgot to remove <QRegExp> header when I removed the code that used it.

* Removes Vestige plugins bundling

	For safety reasons, remove the possibility to bundle VSTs loaded
through vestige. Also runs a safety check during the project being
loaded (Song::loadProject) to check if either Vestige plugins or effect
plugins are using local paths, and abort the project load if so. That is
to avoid malicious code being run because of bad DLLs being shipped with
a project file.

* Extracts code from loadProject to another method

	Extracts code that checks if a DataFile contains local paths to
plugins to another method inside DataFile.

* Removes debug warnings

	Removes warnings previously used for debugging. Improves a
warning message on PathUtil.

* Fixes small bug with error logging

	Fixes small bug, where a QMessageBox was being used to prompt an
error without checking if the gui is loaded first. Now we check for the
GUI and if we are in CLI mode we use a QTextStream instead.

* Saves the bundle in a newly created folder

	Now a folder with the project name is created inside which the
bundle will be saved. This makes the process more convenient.
	Some save errors that previously only triggered qWarnings now
trigger message boxes to warn the user of what happened (using a lambda
function that either shows message boxes or trigger qWarnings depending
whether a gui is present).
	Makes it so saving a bundle doesn't change the loaded project
path, that way the user won't be able to accidentally "Save" over a
bundle which should not be done for now.

* Enhances the name conflict workaround

	Now, instead of replacing the resource names with meaningless
numbers, the bundle save will just append a counter to the end of
filenames that have been repeated.

* Starts addressing Johannes review

* Adds makebundle action to bash completion file

	Adds the bash completion code for the made bundle action.

* Improves safety check on project files

	Now, instead of checking certain XML tags for local paths,
DataFile::hasLocalPlugin() will return true if ANY tag that isn't on the
RESOURCE_ELEMENTS list contains an attribute that starts with "local:".
	The method is now recursive so it can go through all XML tags
during this check.

* Addresses Spekular change request

	Uses basePrefix(Base::LocalDir) instead of "local:" on the
return of unresolved local paths.

* Makes hasLocalPlugins method const

* Replaces literal uses of "local:"

	Instead of using "local:" we are now retrieving the base prefix
from PathUtil, so if we change the prefix on the future we don't need to
replace every mention to it as well.

* Fix some comments on the header and cpp file

* Changes variable on PathUtil to const

	Changes the retrieved pointer to the song object to a const
pointer.

* Leave doxygen comment on CPP file only

	There was 2 doxygen comments for the same method, on the header
and CPP file. The latter was kept since it goes into more details about
the functionality of the method.

* Fix doxygen comment @param

	Fixes the doxygen comment from hasLocalPlugin().

* Remove assert statements

	Some assert statements were being done wrong and are probably
even unnecessary for that piece of code, so they were removed.

* Skips local paths when looking for shortest path

	PathUtil::toShortestRelative() was including the local paths on
the candidate paths, which could lead to a unallowed resource (i.e.:
vst plugin) to be assigned a local path even on a regular save.
	The local paths are now skipped when looking for the shortest
relative path, since they should only be used by the bundle save on the
allowed resources.

* Address Spekular's review

	Changes some of the PathUtil methods to allow a boolean pointer
to be used to return the status of the method, setting it to false if it
failed somewhere.
	Also adds a parameter to toShortestRelative to either allow or
forbid local paths in the search for the shortest relative path.

* Replaces "ok" with "error"
This commit is contained in:
IanCaio
2021-04-02 12:12:16 -03:00
committed by GitHub
parent 3ab86fa381
commit e16dca389e
10 changed files with 418 additions and 65 deletions
Download Patch File Download Diff File Expand all files Collapse all files

257
src/core/DataFile.cpp
View File

@@ -27,10 +27,12 @@
#include "DataFile.h"
#include <math.h>
#include <map>
#include <QDebug>
#include <QFile>
#include <QFileInfo>
#include <QDir>
#include <QMessageBox>
#include "base64.h"
@@ -43,12 +45,19 @@
#include "ProjectVersion.h"
#include "SongEditor.h"
#include "TextFloat.h"
#include "PathUtil.h"
#include "lmmsversion.h"
static void findIds(const QDomElement& elem, QList<jo_id_t>& idList);
// QMap with the DOM elements that access file resources
const DataFile::ResourcesMap DataFile::ELEMENTS_WITH_RESOURCES = {
{ "sampletco", {"src"} },
{ "audiofileprocessor", {"src"} },
};
// Vector with all the upgrade methods
const std::vector<DataFile::UpgradeMethod> DataFile::UPGRADE_METHODS = {
&DataFile::upgrade_0_2_1_20070501 , &DataFile::upgrade_0_2_1_20070508,
@@ -92,6 +101,7 @@ DataFile::typeDescStruct
DataFile::DataFile( Type type ) :
QDomDocument( "lmms-project" ),
m_fileName(""),
m_content(),
m_head(),
m_type( type ),
@@ -118,6 +128,7 @@ DataFile::DataFile( Type type ) :
DataFile::DataFile( const QString & _fileName ) :
QDomDocument(),
m_fileName(_fileName),
m_content(),
m_head(),
m_fileVersion( UPGRADE_METHODS.size() )
@@ -147,6 +158,7 @@ DataFile::DataFile( const QString & _fileName ) :
DataFile::DataFile( const QByteArray & _data ) :
QDomDocument(),
m_fileName(""),
m_content(),
m_head(),
m_fileVersion( UPGRADE_METHODS.size() )
@@ -271,25 +283,90 @@ void DataFile::write( QTextStream & _strm )
bool DataFile::writeFile( const QString& filename )
bool DataFile::writeFile(const QString& filename, bool withResources)
{
const QString fullName = nameWithExtension( filename );
// Small lambda function for displaying errors
auto showError = [this](QString title, QString body){
if (gui)
{
QMessageBox mb;
mb.setWindowTitle(title);
mb.setText(body);
mb.setIcon(QMessageBox::Warning);
mb.setStandardButtons(QMessageBox::Ok);
mb.exec();
}
else
{
qWarning() << body;
}
};
// If we are saving without resources, filename is just the file we are
// saving to. If we are saving with resources (project bundle), filename
// will be used (discarding extensions) to create a folder where the
// bundle will be saved in
QFileInfo fInfo(filename);
const QString bundleDir = fInfo.path() + "/" + fInfo.fileName().section('.', 0, 0);
const QString resourcesDir = bundleDir + "/resources";
const QString fullName = withResources
? nameWithExtension(bundleDir + "/" + fInfo.fileName())
: nameWithExtension(filename);
const QString fullNameTemp = fullName + ".new";
const QString fullNameBak = fullName + ".bak";
QFile outfile( fullNameTemp );
if( !outfile.open( QIODevice::WriteOnly | QIODevice::Truncate ) )
// If we are saving with resources, setup the bundle folder first
if (withResources)
{
if( gui )
// First check if there's a bundle folder with the same name in
// the path already. If so, warns user that we can't overwrite a
// project bundle.
if (QDir(bundleDir).exists())
{
QMessageBox::critical( NULL,
SongEditor::tr( "Could not write file" ),
SongEditor::tr( "Could not open %1 for writing. You probably are not permitted to "
"write to this file. Please make sure you have write-access to "
"the file and try again." ).arg( fullName ) );
showError(SongEditor::tr("Operation denied"),
SongEditor::tr("A bundle folder with that name already eists on the "
"selected path. Can't overwrite a project bundle. Please select a different "
"name."));
return false;
}
// Create bundle folder
if (!QDir().mkdir(bundleDir))
{
showError(SongEditor::tr("Error"),
SongEditor::tr("Couldn't create bundle folder."));
return false;
}
// Create resources folder
if (!QDir().mkdir(resourcesDir))
{
showError(SongEditor::tr("Error"),
SongEditor::tr("Couldn't create resources folder."));
return false;
}
// Copy resources to folder and update paths
if (!copyResources(resourcesDir))
{
showError(SongEditor::tr("Error"),
SongEditor::tr("Failed to copy resources."));
return false;
}
}
QFile outfile (fullNameTemp);
if (!outfile.open(QIODevice::WriteOnly | QIODevice::Truncate))
{
showError(SongEditor::tr("Could not write file"),
SongEditor::tr("Could not open %1 for writing. You probably are not permitted to"
"write to this file. Please make sure you have write-access to "
"the file and try again.").arg(fullName));
return false;
}
@@ -336,6 +413,164 @@ bool DataFile::writeFile( const QString& filename )
bool DataFile::copyResources(const QString& resourcesDir)
{
// List of filenames used so we can append a counter to any
// repeating filenames
std::list<QString> namesList;
ResourcesMap::const_iterator it = ELEMENTS_WITH_RESOURCES.begin();
// Copy resources and manipulate the DataFile to have local paths to them
while (it != ELEMENTS_WITH_RESOURCES.end())
{
QDomNodeList list = elementsByTagName(it->first);
// Go through all elements with the tagname from our map
for (int i = 0; !list.item(i).isNull(); ++i)
{
QDomElement el = list.item(i).toElement();
std::vector<QString>::const_iterator res = it->second.begin();
// Search for attributes that point to resources
while (res != it->second.end())
{
// If the element has that attribute
if (el.hasAttribute(*res))
{
// Get absolute path to resource
bool error;
QString resPath = PathUtil::toAbsolute(el.attribute(*res), &error);
// If we are running without the project loaded (from CLI), "local:" base
// prefixes aren't converted, so we need to convert it ourselves
if (error)
{
resPath = QFileInfo(m_fileName).path() + "/" + resPath.remove(0,
PathUtil::basePrefix(PathUtil::Base::LocalDir).length());
}
// Check if we need to add a counter to the filename
QString finalFileName = QFileInfo(resPath).fileName();
QString extension = resPath.section('.', -1);
int repeatedNames = 0;
for (QString name : namesList)
{
if (finalFileName == name)
{
++repeatedNames;
}
}
// Add the name to the list before modifying it
namesList.push_back(finalFileName);
if (repeatedNames)
{
// Remove the extension, add the counter and add the
// extension again to get the final file name
finalFileName.truncate(finalFileName.lastIndexOf('.'));
finalFileName = finalFileName + "-" + QString::number(repeatedNames) + "." + extension;
}
// Final path is our resources dir + the new file name
QString finalPath = resourcesDir + "/" + finalFileName;
// Copy resource file to the resources folder
if(!QFile::copy(resPath, finalPath))
{
qWarning("ERROR: Failed to copy resource");
return false;
}
// Update attribute path to point to the bundle file
QString newAtt = PathUtil::basePrefix(PathUtil::Base::LocalDir) + "resources/" + finalFileName;
el.setAttribute(*res, newAtt);
}
++res;
}
}
++it;
}
return true;
}
/**
* @brief This recursive method will go through all XML nodes of the DataFile
* and check whether any of them have local paths. If they are not on
* our list of elements that can have local paths we return true,
* indicating that we potentially have plugins with local paths that
* would be a security issue. The Song class can then abort loading
* this project.
* @param parent The parent node being iterated. When called
* without arguments, this will be an empty element that will be
* ignored (since the second parameter will be true).
* @param firstCall Defaults to true, and indicates to this recursive
* method whether this is the first call. If it is it will use the
* root element as the parent.
*/
bool DataFile::hasLocalPlugins(QDomElement parent /* = QDomElement()*/, bool firstCall /* = true*/) const
{
// If this is the first iteration of the recursion we use the root element
if (firstCall) { parent = documentElement(); }
auto children = parent.childNodes();
for (int i = 0; i < children.size(); ++i)
{
QDomNode child = children.at(i);
QDomElement childElement = child.toElement();
bool skipNode = false;
// Skip the nodes allowed to have "local:" attributes, but
// still check its children
for
(
ResourcesMap::const_iterator it = ELEMENTS_WITH_RESOURCES.begin();
it != ELEMENTS_WITH_RESOURCES.end();
++it
)
{
if (childElement.tagName() == it->first)
{
skipNode = true;
break;
}
}
// Check if they have "local:" attribute (unless they are allowed to
// and skipNode is true)
if (!skipNode)
{
auto attributes = childElement.attributes();
for (int i = 0; i < attributes.size(); ++i)
{
QDomNode attribute = attributes.item(i);
QDomAttr attr = attribute.toAttr();
if (attr.value().startsWith(PathUtil::basePrefix(PathUtil::Base::LocalDir),
Qt::CaseInsensitive))
{
return true;
}
}
}
// Now we check the children of this node (recursively)
// and if any return true we return true.
if (hasLocalPlugins(childElement, false))
{
return true;
}
}
// If we got here none of the nodes had the "local:" path.
return false;
}
DataFile::Type DataFile::type( const QString& typeName )
{
for( int i = 0; i < TypeCount; ++i )