Files
lmms/include/SampleThumbnail.h
Cas Pascal 786088baec Improve performance when rendering sample waveforms (#7366)
This PR attempts a number of improvements to the sample rendering (in the song editor, automation editor, AudioFileProcessor, SlicerT, etc) in LMMS:

Thumbnails: Samples are aggregated into a set of thumbnails of multiple resolutions. The smallest larger thumbnail is then chosen for rendering during sample drawing. Each set of thumbnails is stored with its sample file metadata in an unordered_map, so that duplicate samples will use the same set of thumbnails.

Partial rendering: additionally, this PR only renders the portions of the sample clips visible to the viewer to save rendering time.

* Experimental sample thumbnail

* Rename some classes and type aliases. Make some type declarations explicit

* Use a combination of audioFile name and shared_ptrs to track samples; refactor some loops

* That weird line at the end of the sample is now gone

* Small changes to the code; Add comments

* Add missing word to comment; Fix typo

* Track `SharedSampleThumbnailList`s instead

* Major refactor; implement thumbnailing for SlicerT, AFP and Automation editor

* Code clean up, renames and documenting

* Add the namespace lmms comments

* More code updates and documentation

* Fix error in comment

* Comment out `qDebug`s

* Fix formatting

* Use alternative initialization of `SampleThumbnailVisualizeParameters`

* Remove commented code

* Fix style and simplify code

* Use auto

* Draw lines using floating point

* Merge the classes into one nested class
+ Replace while loop with std::find_if

* Fix comparison of different signedness

* Include memory header

* Fix a logic error when selecting samples; Rename a const

* Fix more issues with signedness

* Fix sample drawing error in `visualizeOriginal`

* Only render regions in view of the sample

* Allow partial repaints of sample clips

* Remove unused variable

* Limit most of the painting to the visible region

* Revert back to using rect() in some places

* Partial rendering for AutomationEditor

* Don't redraw painted regions; allowHighResolution; remove `visualizeOriginal`; Remove s_sampleThumbnailCacheMap

* Add s_sampleThumbnailCacheMap back for testing convenience

* Minor change to the way `thumbnailSizeDivisor` is calculated

* Extend update region by an amount

* forgot about this

* Adapt to master; Redesign VisualizeParameters; Don't rely entirely on needsUpdate()

* Don't try to preserve painted regions

* Allow for a bit more thumbnails; Fix incorrect rendering when vertically scrolling

* Fix missing include statement

* Remove the unused variable

* Code optimization; Remove RMS member from Bit; Rename viewRect to drawRect

* More code optimizations

* Fix formatting

* Use begin instead of cbegin

* Improve generation of thumbnails

* Improve expressiveness of the draw code

* Add support for reversing

* Fix drawing code

* Fix draw code (part 2)

* Apply more fixes and simplifications

* Undo some out of scope changes

* Remove SampleWaveform

* Improve documentation

* Use size_t for some counters

* Change width parameter to be size_t

* Remove temporary aggregated peak variable

* Bump up AggregationPerZoomStep to 10

* Zoom out only requested range of thumbnail instead of clipping it separately

* Rename targetSampleWidth to targetThumbnailWidth

* Handle reversing for AFP; Iterate in reverse instead of reversing the entire thumbnail

* Change names to be more accurate

* Improve implementation of sample thumbnail cache map

* Move AggregationPerZoomStep back down to 2, do not cap smallest thumbnail width to display width
To improve performance with especially long
samples (~20 minutes)

* Simplify sample thumbnail cache handling in constructor

* Call drawLines instead of drawLine in a loop

QPainter::drawLine calls drawLines with a line count of 1. Therefore, calling drawLine in a loop means we are simply calling drawLines a lot more times than necessary.

* Bump up AggregationPerZoomStep to 10 again
Thought using 2 would help performance (when rendering). Maybe it does, but it's still quite slow when rendering a bunch of thumbnails at once.

* Fix off-by-one error when constructing Thumbnail from buffer

* Fix crash when viewport is not in bounds

* Apply performance improvements

Performance in the zoomOut function was bad because of the dynamic memory allocation. Huge chunks of memory were being allocated quite often, casuing a ton of cache misses and all around slower performance. To fix this, all the necessary downsampling is now done within the for loop and handled one pixel after another, instead of all at once.

To further avoid allocations in the draw call, the change to use drawLines has been reverted.

We now pass VisualizeParameters by value (it is only 64 bytes, which will fit quite nicely in a cache line, which is more benefical than reaching for that data by reference to some other part of the code).

After applying these changes, we are now practically only bounded by the painting done by Qt (according to profiling done by Valgrind). Proper use of OpenGL could resolve this issue, which should finally make the performance quite optimal in  variety of situations.

* Apply minor changes

Update copyright and unused functions. Move in newly created thumbnail into the cache instead of copying it.

* Use C++20's designated initializers

* Create param right before visualizing

* Fix regressions with reversing

* Fix incorrect rendering in AFP and SlicerT

* Move MaxSampleThumbnailCacheSize and AggregationPerZoomStep into implementation file

* Remove static keyword

* Remove getter and setter for peak data

---------

Co-authored-by: Sotonye Atemie <sakertooth@gmail.com>
2025-02-08 20:05:19 -05:00

144 lines
4.2 KiB
C++

/*
* SampleThumbnail.h
*
* Copyright (c) 2024 Khoi Dau <casboi86@gmail.com>
* Copyright (c) 2024 Sotonye Atemie <sakertooth@gmail.com>
*
* This file is part of LMMS - https://lmms.io
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public
* License as published by the Free Software Foundation; either
* version 2 of the License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* General Public License for more details.
*
* You should have received a copy of the GNU General Public
* License along with this program (see COPYING); if not, write to the
* Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
* Boston, MA 02110-1301 USA.
*
*/
#ifndef LMMS_SAMPLE_THUMBNAIL_H
#define LMMS_SAMPLE_THUMBNAIL_H
#include <QDateTime>
#include <QPainter>
#include <QRect>
#include <memory>
#include "Sample.h"
#include "lmms_export.h"
namespace lmms {
/**
Allows for visualizing sample data.
On construction, thumbnails will be generated
at logarathmic intervals of downsampling. Those cached thumbnails will then be further downsampled on the fly and
transformed in various ways to create the desired waveform.
Given that we are dealing with far less data to generate
the visualization however (i.e., we are not reading from original sample data when drawing), this provides a
significant performance boost that wouldn't be possible otherwise.
*/
class LMMS_EXPORT SampleThumbnail
{
public:
struct VisualizeParameters
{
QRect sampleRect; //!< A rectangle that covers the entire range of samples.
QRect drawRect; //!< Specifies the location in `sampleRect` where the waveform will be drawn. Equals
//!< `sampleRect` when null.
QRect viewportRect; //!< Clips `drawRect`. Equals `drawRect` when null.
float amplification = 1.0f; //!< The amount of amplification to apply to the waveform.
float sampleStart = 0.0f; //!< Where the sample begins for drawing.
float sampleEnd = 1.0f; //!< Where the sample ends for drawing.
bool reversed = false; //!< Determines if the waveform is drawn in reverse or not.
};
SampleThumbnail() = default;
SampleThumbnail(const Sample& sample);
void visualize(VisualizeParameters parameters, QPainter& painter) const;
private:
class Thumbnail
{
public:
struct Peak
{
Peak() = default;
Peak(float min, float max)
: min(min)
, max(max)
{
}
Peak(const SampleFrame& frame)
: min(std::min(frame.left(), frame.right()))
, max(std::max(frame.left(), frame.right()))
{
}
Peak operator+(const Peak& other) const { return Peak(std::min(min, other.min), std::max(max, other.max)); }
Peak operator+(const SampleFrame& frame) const { return *this + Peak{frame}; }
float min = std::numeric_limits<float>::max();
float max = std::numeric_limits<float>::min();
};
Thumbnail() = default;
Thumbnail(std::vector<Peak> peaks, double samplesPerPeak);
Thumbnail(const float* buffer, size_t size, size_t width);
Thumbnail zoomOut(float factor) const;
Peak& operator[](size_t index) { return m_peaks[index]; }
const Peak& operator[](size_t index) const { return m_peaks[index]; }
int width() const { return m_peaks.size(); }
double samplesPerPeak() const { return m_samplesPerPeak; }
private:
std::vector<Peak> m_peaks;
double m_samplesPerPeak = 0.0;
};
struct SampleThumbnailEntry
{
QString filePath;
QDateTime lastModified;
friend bool operator==(const SampleThumbnailEntry& first, const SampleThumbnailEntry& second)
{
return first.filePath == second.filePath && first.lastModified == second.lastModified;
}
};
struct Hash
{
std::size_t operator()(const SampleThumbnailEntry& entry) const noexcept { return qHash(entry.filePath); }
};
using ThumbnailCache = std::vector<Thumbnail>;
std::shared_ptr<ThumbnailCache> m_thumbnailCache = std::make_shared<ThumbnailCache>();
inline static std::unordered_map<SampleThumbnailEntry, std::shared_ptr<ThumbnailCache>, Hash> s_sampleThumbnailCacheMap;
};
} // namespace lmms
#endif // LMMS_SAMPLE_THUMBNAIL_H