From 5a561b25589d2340e56c671f4fbd8ac2ce3688a3 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 19 Jul 2019 16:39:18 -0400 Subject: [PATCH 0001/1277] Some doc updates to matches changes in UI. --- docs/faq.rst | 38 +++++++++++++++++++++++++++----------- 1 file changed, 27 insertions(+), 11 deletions(-) diff --git a/docs/faq.rst b/docs/faq.rst index a703fd065..c80d00a8a 100644 --- a/docs/faq.rst +++ b/docs/faq.rst @@ -8,18 +8,21 @@ How can I stop ZoneMinder filling up my disk? --------------------------------------------- Recent versions of ZoneMinder come with a filter you can use for this purpose already included. -The filter is called **PurgeWhenFull** and to find it, choose one of the event counts from the console page, for instance events in the last hour, for one of your monitors. **Note** that this filter is automatically enabled if you do a fresh install of ZoneMinder including creating a new database. If you already have an existing database and are upgrading ZoneMinder, it will retain the settings of the filter (which in earlier releases was disabled by default). So you may want to check if PurgeWhenFull is enabled and if not, enable it. +The filter is called **PurgeWhenFull** and to find it, click on the word **Filters** in the header. +**Note** that this filter is automatically enabled if you do a fresh install of ZoneMinder including creating a new database. If you already have an existing database and are upgrading ZoneMinder, it will retain the settings of the filter (which in earlier releases was disabled by default). So you may want to check if PurgeWhenFull is enabled and if not, enable it. -To enable it, go to Web Console, click on any of your Events of any of your monitors. -This will bring up an event listing and a filter window. +To enable it, go to Web Console, click on the word **Filters** in the UI header. -In the filter window there is a drop down select box labeled 'Use Filter', that lets your select a saved filter. Select 'PurgeWhenFull' and it will load that filter. +In the filter window there is a drop down select box labeled 'Use Filter', that lets you select a saved filter. Select 'PurgeWhenFull' and it will load that filter. Make any modifications you might want, such as the percentage full you want it to kick in, or how many events to delete at a time (it will repeat the filter as many times as needed to clear the space, but will only delete this many events each time to get there). -Then click on 'Save' which will bring up a new window. Make sure the 'Automatically delete' box is checked and press save to save your filter. This will then run in the background to keep your disk within those limits. +Ensure that the Run filter in background checkbox is checked. +Ensure that the Delete all matches checkbox is checked. -After you've done that, you changes will automatically be loaded into zmfilter within a few minutes. +Then click on 'Save'. The filter will immediately begin executing in the background to keep your disk within those limits. + +Please note that that this filter will only affect the default storage location. If you have added other storage areas, you must create a PurgeWhenFull filter for each one, and specify the Storage Area as one of the parameters in the filter. You can duplicate the existing PurgeWhenFull filter by using Save As instead of Save. Check the ``zmfilter.log`` file to make sure it is running as sometimes missing perl modules mean that it never runs but people don't always realize. @@ -42,7 +45,7 @@ Normally an event created as the result of an alarm consists of entries in one o ZM_RUN_AUDIT: -The zmaudit daemon exists to check that the saved information in the database and on the file system match and are consistent with each other. If an error occurs or if you are using 'fast deletes' it may be that database records are deleted but files remain. In this case, and similar, zmaudit will remove redundant information to synchronize the two data stores. This option controls whether zmaudit is run in the background and performs these checks and fixes continuously. This is recommended for most systems however if you have a very large number of events the process of scanning the database and file system may take a long time and impact performance. In this case you may prefer to not have zmaudit running unconditionally and schedule occasional checks at other, more convenient, times. +The zmaudit daemon exists to check that the saved information in the database and on the file system match and are consistent with each other. If an error occurs or if you are using 'fast deletes' it may be that database records are deleted but files remain. In this case, and similar, zmaudit will remove redundant information to synchronize the two data stores. This option controls whether zmaudit is run in the background and performs these checks and fixes continuously. This is not recommended for most systems, as zmaudit.pl is very resource intensive. @@ -280,14 +283,27 @@ Also, please check the value of the ZM_PATH_ZMS setting under the Paths Options Lastly, please look for errors created by the zmc processes. If zmc isn't running, then zms will not be able to get an image from it and will exit. -I have several monitors configured but when I load the Montage view in FireFox why can I only see two? or, Why don't all my cameras display when I use the Montage view in FireFox? +I have several monitors configured but when I load the Montage view why can I only see two? or, Why don't all my cameras display when I use the Montage view? -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -By default FireFox only supports a small number of simultaneous connections. Using the montage view usually requires one persistent connection for each camera plus intermittent connections for other information such as statuses. +By default most browsers only support a small number of simultaneous connections to any given server. Using the montage view usually requires one persistent connection for each camera plus intermittent connections for other information such as statuses. -You will need to increase the number of allowed connections to use the montage view with more than a small number of cameras. Certain FireFox extensions such as FasterFox may also help to achieve the same result. +In firefox you can increase the limit, but other browsers are not configurable in this way. -To resolve this situation, follow the instructions below: +A solution for all browsers is something we call multi-port. We reconfigure apache to operate on ports other than the default of 80(http) or 443(https). You need to pick a range, let's say 30000 to 30010 in order to support 10 cameras. We add lines to your zoneminder apache config file as follows: + +Listen 30000 +Listen 30001 +Listen 30002 +Listen 30003 +etc +Listen 30010 + +If you are using virtualhosts, you will have to add these to the VirtualHost directive as well. + +Then in ZoneMinder config, Go Options -> Network and set MIN_STREAMING_PORT to 30000. Now when generating urls to stream images from ZoneMinder a port will be appended that is 30000 + MonitorId, so Monitor 1 will stream from 30001 and so on. This will allow Montage to stream from all monitors. + +Alternatively if you are in fact using only Firefox, you can increase the limit as follows: Enter ``about:config`` in the address bar From 8aecf89466b01c3274dab013d462ac590e0fd46f Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 24 Sep 2019 10:12:08 -0400 Subject: [PATCH 0002/1277] documentation updates: add missing filter substitutions, rough in storage areas documentation --- docs/userguide/filterevents.rst | 31 ++++++++++++++++--------------- docs/userguide/options.rst | 2 ++ 2 files changed, 18 insertions(+), 15 deletions(-) diff --git a/docs/userguide/filterevents.rst b/docs/userguide/filterevents.rst index d3c4955c9..973755641 100644 --- a/docs/userguide/filterevents.rst +++ b/docs/userguide/filterevents.rst @@ -4,7 +4,7 @@ Filtering Events Filters allow you to define complex conditions with associated actions in ZoneMinder. Examples could include: * Send an email each time a new event occurs for a specific monitor -* Delete events that are more than 10 days old +* Delete events that are more than 10 days old And many more. @@ -27,11 +27,16 @@ Here is what the filter window looks like * Archive all matches: sets the archive field to 1 in the Database for the matched events. Think of 'archiving' as grouping them under a special category - you can view archived events later and also make sure archived events don't get deleted, for example + * Update used disk space: calculates how much disk space is taken by the event and updates the db record. + * Create video for all matches: ffmpeg will be used to create a video file out of all the stored jpgs if using jpeg storage. * Email details of all matches: Sends an email to the configured address with details about the event. The email can be customized as per TBD - * Execute command on all matches: Allows you to execute any arbitrary command on the matched events. You can use replacement tokens as subsequent arguents to the command, the last argument will be the absolute path to the event, preceeded by replacement arguents. eg: /usr/bin/script.sh %MN% will excecute as /usr/bin/script.sh MonitorName /path/to/event. + * Message details of all matches: Uses an email to SMS gateway to send an SMS message for each match. + * Execute command on all matches: Allows you to execute any arbitrary command on the matched events. You can use replacement tokens as subsequent arguents to the command, the last argument will be the absolute path to the event, preceeded by replacement arguents. eg: /usr/bin/script.sh %MN% will excecute as /usr/bin/script.sh MonitorName /path/to/event. Please note that urls may contain characters like & that need quoting. So you may need to put quotes around them like /usr/bin/scrupt.sh "%MN%". * Delete all matches: Deletes all the matched events * *E*: Use 'Submit' to 'test' your matching conditions. This will just match and show you what filters match. Use 'Execute' to actually execute the action after matching your conditions. Use 'Save' to save the filter for future use and 'Reset' to clear your settings + * Copy all matches: copies the event files to another location, specified in the Copy To dropdown. The other location must be setup in the Storage Tab under options. + * Move all matches: copies the event files to another location, specified in the Move To dropdown. The other location must be setup in the Storage Tab under options. The files will be delete from the original location. .. NOTE:: More details on filter conditions: @@ -63,8 +68,12 @@ Here is what the filter window looks like * %EPI1% Path to the first alarmed event image * %EPIM% Path to the (first) event image with the highest score * %EI1% Attach first alarmed event image + * %EI1A% Attach first alarmed event analysis image * %EIM% Attach (first) event image with the highest score + * %EIMA% Attach (first) event analysis image with the highest score + * %EIMOD% Attach event image with object detection objects * %EV% Attach event mpeg video + * %EVM% Attach event mpeg video in phone format * %MN% Name of the monitor * %MET% Total number of events for the monitor * %MEH% Number of events for the monitor in the last hour @@ -83,14 +92,7 @@ Here is what the filter window looks like Filtering is a powerful mechanism you can use to eliminate events that fit a certain pattern however in many cases modifying the zone settings will better address this. Where it really comes into its own is generally in applying time filters, so for instance events that happen during weekdays or at certain times of the day are highlighted, uploaded or deleted. Additionally using disk related terms in your filters means you can automatically create filters that delete the oldest events when your disk gets full. Be warned however that if you use this strategy then you should limit the returned results to the amount of events you want deleted in each pass until the disk usage is at an acceptable level. If you do not do this then the first pass when the disk usage is high will match, and then delete, all events unless you have used other criteria inside of limits. ZoneMinder ships with a sample filter already installed, though disabled. The PurgeWhenFull filter can be used to delete the oldest events when your disk starts filling up. To use it you should select and load it in the filter interface, modify it to your requirements, and then save it making you sure you check the ‘Delete all matches’ option. This will then run in the background and ensure that your disk does not fill up with events. - -Saving filters ------------------ - -.. image:: images/filter-save.png - :width: 400px - -When saving filters, if you want the filter to run in the background make sure you select the "Run filter in background" option. When checked, ZoneMinder will make sure the filter is checked regularly. For example, if you want to be notified of new events by email, you should make sure this is checked. Filters that are configured to run in the background have a "*" next to it. + When saving filters, if you want the filter to run in the background make sure you select the "Run filter in background" option. When checked, ZoneMinder will make sure the filter is checked regularly. For example, if you want to be notified of new events by email, you should make sure this is checked. Filters that are configured to run in the background have a "*" next to it in the dropdown. For example: @@ -101,12 +103,11 @@ How filters actually work -------------------------- It is useful to know how filters actually work behind the scenes in ZoneMinder, in the event you find your filter not functioning as intended: -* the primary filter processing process in ZoneMinder is a perl file called ``zmfilter.pl`` which retrieves filters from the Filters database table +* Each filter set to run in the background will be run in it's own process called ``zmfilter.pl`` which retrieves filters from the Filters database table * zmfilter.pl runs every FILTER_EXECUTE_INTERVAL seconds (default is 20s, can be changed in Options->System) -* in each run, it goes through all the filters which are marked as "Run in Background" and if the conditions match performs the specified action -* zmfilter.pl also reloads all the filters every FILTER_RELOAD_DELAY seconds (default is 300s/5mins, can be changed in Options->System) - * So if you have just created a new filter, zmfilter will not see it till the next FILTER_RELOAD_DELAY cycle - * This is also important if you are using "relative times" like 'now' - see :ref:`relative_caveat` +* after each interval the filter will query the database and apply the action to each matching event. +* zmfilter.pl also reloads the filter every FILTER_RELOAD_DELAY seconds (default is 300s/5mins, can be changed in Options->System) +* In previous versions of ZoneMinder filter changes would not take immediate effect, but now the web ui will start/stop/restart filters as appropriate upon editing a filter. Relative items in date strings diff --git a/docs/userguide/options.rst b/docs/userguide/options.rst index 873002a0d..a2ca67463 100644 --- a/docs/userguide/options.rst +++ b/docs/userguide/options.rst @@ -10,7 +10,9 @@ If you have changed the value of an option you should then ‘save’ it. A numb options/options_display options/options_system options/options_config + options/options_api options/options_servers + options/options_storage options/options_paths options/options_web options/options_images From 14381cc4daf891125e6472dc8cd915a18561ad89 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 24 Sep 2019 10:12:28 -0400 Subject: [PATCH 0003/1277] rough in storage documentation --- docs/userguide/options/options_storage.rst | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 docs/userguide/options/options_storage.rst diff --git a/docs/userguide/options/options_storage.rst b/docs/userguide/options/options_storage.rst new file mode 100644 index 000000000..b84f2e6f6 --- /dev/null +++ b/docs/userguide/options/options_storage.rst @@ -0,0 +1,10 @@ +Options - Storage +----------------- + +.. image:: images/Options_Storage.png + +Storage tab is used for setting up multiple storage areas for storing events. +To add a new server use the Add New Storage button. The storage area will require a name and a path. The path should be absolute and if multiple servers are in use, each should have access to it using the same path. The Url field is used for S3 type storage. S3 storage should be mounted in the filesystem using s3fs and can also be specified in the Url for more efficient access. +The Do Deletes option tell ZoneMinder whether to actually perform delete operations when deleting events. S3fs systems often do deletes in a cron job or other background task and doing the deletes can overload an S3 system. + +To delete a storage area mark that area and click the Delete button. From b7857d7a88287fb1607c46b254e7beaf00710864 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 11 Dec 2020 11:43:17 -0500 Subject: [PATCH 0004/1277] Add D1-PAL resolution --- web/skins/classic/views/monitor.php | 2 ++ 1 file changed, 2 insertions(+) diff --git a/web/skins/classic/views/monitor.php b/web/skins/classic/views/monitor.php index e3c0e4a56..090ef2ad1 100644 --- a/web/skins/classic/views/monitor.php +++ b/web/skins/classic/views/monitor.php @@ -910,11 +910,13 @@ include('_monitor_source_nvsocket.php'); '320x240'=>'320x240', '320x200'=>'320x200', '352x240'=>'352x240 CIF', + '352x480'=>'352x480', '640x480'=>'640x480', '640x400'=>'640x400', '704x240'=>'704x240 2CIF', '704x480'=>'704x480 4CIF', '720x480'=>'720x480 D1', + '720x576'=>'720x576 D1 PAL', '1280x720'=>'1280x720 720p', '1280x800'=>'1280x800', '1280x960'=>'1280x960 960p', From fd21d47bbb159863a37dee26f2e4b590115542de Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sat, 23 Jan 2021 16:28:39 -0500 Subject: [PATCH 0005/1277] Merge pull request #3113 from pliablepixels/feature-api-daemon-control new api to control daemon --- web/api/app/Controller/HostController.php | 24 +++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/web/api/app/Controller/HostController.php b/web/api/app/Controller/HostController.php index cf6e99b53..3d3a23c6e 100644 --- a/web/api/app/Controller/HostController.php +++ b/web/api/app/Controller/HostController.php @@ -21,6 +21,30 @@ class HostController extends AppController { )); } + // an interface to individually control the various ZM daemons + // invocation: https://server/zm/api/host/daemonControl/.pl/.json + // note that this API is only for interaction with a specific + // daemon. zmdc also allows other functions like logrot/etc + public function daemonControl($daemon_name, $command) { + global $user; + if ($command == 'check' || $command == 'status') { + $permission = 'View'; + } else { + $permission = 'Edit'; + } + $allowed = (!$user) || ($user['System'] == $permission ); + if ( !$allowed ) { + throw new UnauthorizedException(__("Insufficient privileges")); + return; + } + $string = ZM_PATH_BIN."/zmdc.pl $command $daemon_name"; + $result = exec($string); + $this->set(array( + 'result' => $result, + '_serialize' => array('result') + )); + } + function getLoad() { $load = sys_getloadavg(); From 73c1264771b4f63320f11af1dbf8b21a842baacf Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sat, 23 Jan 2021 17:31:19 -0500 Subject: [PATCH 0006/1277] bump version for release --- distros/redhat/zoneminder.spec | 2 +- version | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/distros/redhat/zoneminder.spec b/distros/redhat/zoneminder.spec index e0de33c96..59b7cd1bb 100644 --- a/distros/redhat/zoneminder.spec +++ b/distros/redhat/zoneminder.spec @@ -28,7 +28,7 @@ %global _hardened_build 1 Name: zoneminder -Version: 1.34.22 +Version: 1.34.23 Release: 1%{?dist} Summary: A camera monitoring and analysis tool Group: System Environment/Daemons diff --git a/version b/version index 9bda6b214..a0e1e3085 100644 --- a/version +++ b/version @@ -1 +1 @@ -1.34.22 +1.34.23 From 86fd86a72356f3f9d1d7362fd9d64e3114d205bc Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sun, 24 Jan 2021 12:29:16 -0500 Subject: [PATCH 0007/1277] Switch to FriendsOfCake branch 3.0 --- web/api/app/Plugin/Crud | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/web/api/app/Plugin/Crud b/web/api/app/Plugin/Crud index 2ab7e23fb..14292374c 160000 --- a/web/api/app/Plugin/Crud +++ b/web/api/app/Plugin/Crud @@ -1 +1 @@ -Subproject commit 2ab7e23fb4f5b31a2f043ba9b9f607a518007f75 +Subproject commit 14292374ccf1328f2d5db20897bd06f99ba4d938 From 7dd9188b3bb4ee4cd77df88ea7c7325ecfbfb4ba Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 26 Jan 2021 18:09:19 -0500 Subject: [PATCH 0008/1277] fix eslint --- web/skins/classic/views/js/watch.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/web/skins/classic/views/js/watch.js b/web/skins/classic/views/js/watch.js index ed4f370ae..820150457 100644 --- a/web/skins/classic/views/js/watch.js +++ b/web/skins/classic/views/js/watch.js @@ -858,7 +858,7 @@ function initPage() { } else if ( monitorRefresh > 0 ) { setInterval(reloadWebSite, monitorRefresh*1000); } -} // initPage +} // initPage // Kick everything off window.addEventListener('DOMContentLoaded', initPage); From 01c74fdac3c01c2d8de1837b279707c9605707f1 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sat, 27 Feb 2021 12:24:01 -0500 Subject: [PATCH 0009/1277] Add RTSPStreamName to Monitors table. Bump version to 1.35.19 --- db/zm_create.sql.in | 1 + db/zm_update-1.35.19.sql | 11 +++++++++++ distros/redhat/zoneminder.spec | 2 +- version | 2 +- 4 files changed, 14 insertions(+), 2 deletions(-) create mode 100644 db/zm_update-1.35.19.sql diff --git a/db/zm_create.sql.in b/db/zm_create.sql.in index 7a87f2403..f70ccc55d 100644 --- a/db/zm_create.sql.in +++ b/db/zm_create.sql.in @@ -535,6 +535,7 @@ CREATE TABLE `Monitors` ( `Latitude` DECIMAL(10,8), `Longitude` DECIMAL(10,8), `RTSPServer` BOOLEAN NOT NULL DEFAULT FALSE, + `RTSPStreamName` varchar(255) NOT NULL default '', PRIMARY KEY (`Id`) ) ENGINE=@ZM_MYSQL_ENGINE@; diff --git a/db/zm_update-1.35.19.sql b/db/zm_update-1.35.19.sql new file mode 100644 index 000000000..8772491db --- /dev/null +++ b/db/zm_update-1.35.19.sql @@ -0,0 +1,11 @@ +SET @s = (SELECT IF( + (SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS WHERE table_schema = DATABASE() + AND table_name = 'Monitors' + AND column_name = 'RTSPStreamName' + ) > 0, +"SELECT 'Column RTSPStreamName already exists in Monitors'", +"ALTER TABLE `Monitors` ADD `RTSPStreamName` varchar(255) NOT NULL default '' AFTER `RTSPServer`" +)); + +PREPARE stmt FROM @s; +EXECUTE stmt; diff --git a/distros/redhat/zoneminder.spec b/distros/redhat/zoneminder.spec index b0b3c1abe..d1da6e852 100644 --- a/distros/redhat/zoneminder.spec +++ b/distros/redhat/zoneminder.spec @@ -28,7 +28,7 @@ %global _hardened_build 1 Name: zoneminder -Version: 1.35.18 +Version: 1.35.19 Release: 1%{?dist} Summary: A camera monitoring and analysis tool Group: System Environment/Daemons diff --git a/version b/version index 2e155ef29..71fcfce81 100644 --- a/version +++ b/version @@ -1 +1 @@ -1.35.18 +1.35.19 From 430f83995459553cc4712166027f081f283e35c2 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sat, 27 Feb 2021 12:24:38 -0500 Subject: [PATCH 0010/1277] add video_fifo filename strings to shmem. --- scripts/ZoneMinder/lib/ZoneMinder/Memory.pm.in | 2 ++ src/zm_monitor.h | 9 ++++++++- 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/scripts/ZoneMinder/lib/ZoneMinder/Memory.pm.in b/scripts/ZoneMinder/lib/ZoneMinder/Memory.pm.in index 4b6c6531c..1e4b565cc 100644 --- a/scripts/ZoneMinder/lib/ZoneMinder/Memory.pm.in +++ b/scripts/ZoneMinder/lib/ZoneMinder/Memory.pm.in @@ -170,6 +170,8 @@ our $mem_data = { last_read_time => { type=>'time_t64', seq=>$mem_seq++ }, control_state => { type=>'uint8[256]', seq=>$mem_seq++ }, alarm_cause => { type=>'int8[256]', seq=>$mem_seq++ }, + video_fifo => { type=>'int8[64]', seq=>$mem_seq++ }, + audio_fifo => { type=>'int8[64]', seq=>$mem_seq++ }, } }, trigger_data => { type=>'TriggerData', seq=>$mem_seq++, 'contents'=> { diff --git a/src/zm_monitor.h b/src/zm_monitor.h index 54facf26b..421e14e83 100644 --- a/src/zm_monitor.h +++ b/src/zm_monitor.h @@ -102,7 +102,7 @@ protected: typedef enum { CLOSE_TIME, CLOSE_IDLE, CLOSE_ALARM } EventCloseMode; - /* sizeof(SharedData) expected to be 344 bytes on 32bit and 64bit */ + /* sizeof(SharedData) expected to be 472 bytes on 32bit and 64bit */ typedef struct { uint32_t size; /* +0 */ int32_t last_write_index; /* +4 */ @@ -154,6 +154,8 @@ protected: uint8_t control_state[256]; /* +104 */ char alarm_cause[256]; + char video_fifo[64]; + char audio_fifo[64]; } SharedData; @@ -317,6 +319,7 @@ protected: Rgb signal_check_colour; // The colour that the camera will emit when no video signal detected bool embed_exif; // Whether to embed Exif data into each image frame or not bool rtsp_server; // Whether to include this monitor as an rtsp server stream + std::string rtsp_streamname; // path in the rtsp url for this monitor int capture_max_fps; @@ -484,6 +487,10 @@ public: AVStream *GetVideoStream() const { return camera ? camera->get_VideoStream() : nullptr; }; AVCodecContext *GetVideoCodecContext() const { return camera ? camera->get_VideoCodecContext() : nullptr; }; + const std::string GetVideoFifo() const { return shared_data ? shared_data->video_fifo : ""; }; + const std::string GetAudioFifo() const { return shared_data ? shared_data->audio_fifo : ""; }; + const std::string GetRTSPStreamName() const { return rtsp_streamname; }; + int GetImage(int32_t index=-1, int scale=100); ZMPacket *getSnapshot( int index=-1 ) const; struct timeval GetTimestamp( int index=-1 ) const; From fdf1fbd4972b112af130e39a959789a71b88d40e Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sat, 27 Feb 2021 12:26:19 -0500 Subject: [PATCH 0011/1277] Add a fifo version of the rtsp server --- src/zm_rtsp_server.cpp | 226 +++++++++++++++++ src/zm_rtsp_server_adts_fifo_source.cpp | 43 ++++ src/zm_rtsp_server_adts_fifo_source.h | 66 +++++ src/zm_rtsp_server_fifo_source.cpp | 215 ++++++++++++++++ src/zm_rtsp_server_fifo_source.h | 82 ++++++ src/zm_rtsp_server_fifo_video_source.cpp | 20 ++ src/zm_rtsp_server_fifo_video_source.h | 32 +++ src/zm_rtsp_server_h264_fifo_source.cpp | 234 ++++++++++++++++++ src/zm_rtsp_server_h264_fifo_source.h | 99 ++++++++ ...zm_rtsp_server_server_media_subsession.cpp | 37 ++- src/zm_rtsp_server_server_media_subsession.h | 3 +- src/zm_rtsp_server_thread.cpp | 62 ++++- src/zm_rtsp_server_thread.h | 6 +- ...server_unicast_server_media_subsession.cpp | 5 +- 14 files changed, 1104 insertions(+), 26 deletions(-) create mode 100644 src/zm_rtsp_server.cpp create mode 100644 src/zm_rtsp_server_adts_fifo_source.cpp create mode 100644 src/zm_rtsp_server_adts_fifo_source.h create mode 100644 src/zm_rtsp_server_fifo_source.cpp create mode 100644 src/zm_rtsp_server_fifo_source.h create mode 100644 src/zm_rtsp_server_fifo_video_source.cpp create mode 100644 src/zm_rtsp_server_fifo_video_source.h create mode 100644 src/zm_rtsp_server_h264_fifo_source.cpp create mode 100644 src/zm_rtsp_server_h264_fifo_source.h diff --git a/src/zm_rtsp_server.cpp b/src/zm_rtsp_server.cpp new file mode 100644 index 000000000..d31ec50fe --- /dev/null +++ b/src/zm_rtsp_server.cpp @@ -0,0 +1,226 @@ +// +// ZoneMinder RTSP Daemon +// Copyright (C) 2021 Isaac Connor +// +// This program is free software; you can redistribute it and/or +// modify it under the terms of the GNU General Public License +// as published by the Free Software Foundation; either version 2 +// of the License, or (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. +// + +/* + +=head1 NAME + +zm_rtsp_server - The ZoneMinder Server + +=head1 SYNOPSIS + + zmc -m + zmc --monitor + zmc -h + zmc --help + zmc -v + zmc --version + +=head1 DESCRIPTION + +This binary's job is to connect to fifo's provided by local zmc processes +and provide that stream over rtsp + +=head1 OPTIONS + + -m, --monitor_id - ID of a monitor to stream + -h, --help - Display usage information + -v, --version - Print the installed version of ZoneMinder + +=cut + +*/ + +#include "zm.h" +#include "zm_db.h" +#include "zm_define.h" +#include "zm_monitor.h" +#include "zm_rtsp_server_thread.h" +#include "zm_rtsp_server_fifo_video_source.h" +#include "zm_signal.h" +#include "zm_time.h" +#include "zm_utils.h" +#include +#include + +void Usage() { + fprintf(stderr, "zm_rtsp_server -m \n"); + + fprintf(stderr, "Options:\n"); + fprintf(stderr, " -m, --monitor : We default to all monitors use this to specify just one\n"); + fprintf(stderr, " -h, --help : This screen\n"); + fprintf(stderr, " -v, --version : Report the installed version of ZoneMinder\n"); + exit(0); +} + +int main(int argc, char *argv[]) { + self = argv[0]; + + srand(getpid() * time(nullptr)); + + int monitor_id = -1; + + static struct option long_options[] = { + {"monitor", 1, nullptr, 'm'}, + {"help", 0, nullptr, 'h'}, + {"version", 0, nullptr, 'v'}, + {nullptr, 0, nullptr, 0} + }; + + while (1) { + int option_index = 0; + + int c = getopt_long(argc, argv, "m:h:v", long_options, &option_index); + if ( c == -1 ) { + break; + } + + switch (c) { + case 'm': + monitor_id = atoi(optarg); + break; + case 'h': + case '?': + Usage(); + break; + case 'v': + std::cout << ZM_VERSION << "\n"; + exit(0); + default: + // fprintf(stderr, "?? getopt returned character code 0%o ??\n", c); + break; + } + } + + if (optind < argc) { + fprintf(stderr, "Extraneous options, "); + while (optind < argc) + printf("%s ", argv[optind++]); + printf("\n"); + Usage(); + } + + const char *log_id_string = "zm_rtsp_server"; + ///std::string log_id_string = std::string("zm_rtsp_server"); + ///if ( monitor_id > 0 ) log_id_string += stringtf("_m%d", monitor_id); + + logInit(log_id_string); + zmLoadStaticConfig(); + zmDbConnect(); + zmLoadDBConfig(); + logInit(log_id_string); + + hwcaps_detect(); + + std::string where = "`Function` != 'None' AND `RTSPServer` != false"; + if (staticConfig.SERVER_ID) + where += stringtf(" AND `ServerId`=%d", staticConfig.SERVER_ID); + if (monitor_id > 0) + where += stringtf(" AND `Id`=%d", monitor_id); + + std::vector> monitors = Monitor::LoadMonitors(where, Monitor::QUERY); + + if (monitors.empty()) { + Error("No monitors found"); + exit(-1); + } else { + Debug(2, "%d monitors loaded", monitors.size()); + } + + Info("Starting RTSP Server version %s", ZM_VERSION); + zmSetDefaultHupHandler(); + zmSetDefaultTermHandler(); + zmSetDefaultDieHandler(); + + sigset_t block_set; + sigemptyset(&block_set); + + sigaddset(&block_set, SIGHUP); + sigaddset(&block_set, SIGUSR1); + sigaddset(&block_set, SIGUSR2); + + int result = 0; + + while (!zm_terminate) { + result = 0; + + for (const std::shared_ptr &monitor : monitors) { + monitor->LoadCamera(); + + while (!monitor->connect()) { + Warning("Couldn't connect to monitor %d", monitor->Id()); + sleep(1); + } + } // end foreach monitor + + RTSPServerThread ** rtsp_server_threads = nullptr; + if (config.min_rtsp_port) { + rtsp_server_threads = new RTSPServerThread *[monitors.size()]; + Debug(1, "Starting RTSP server because min_rtsp_port is set"); + } else { + Debug(1, "Not starting RTSP server because min_rtsp_port not set"); + exit(-1); + } + + for (size_t i = 0; i < monitors.size(); i++) { + rtsp_server_threads[i] = new RTSPServerThread(monitors[i]); + std::string streamname = monitors[i]->GetRTSPStreamName(); + ServerMediaSession *sms = rtsp_server_threads[i]->addSession(streamname); + ZoneMinderFifoVideoSource *video_source = static_cast(rtsp_server_threads[i]->addFifo(sms, monitors[i]->GetVideoFifo())); + video_source->setWidth(monitors[i]->Width()); + video_source->setHeight(monitors[i]->Height()); + FramedSource *audio_source = rtsp_server_threads[i]->addFifo(sms, monitors[i]->GetAudioFifo()); + rtsp_server_threads[i]->start(); + } + + while (!zm_terminate) { + sleep(1); + // What to do in here? Sleep mostly. Wait for reload commands maybe watch for dead monitors. + if ((result < 0) or zm_reload) { + // Failure, try reconnecting + break; + } + } // end while ! zm_terminate and connected + + for (size_t i = 0; i < monitors.size(); i++) { + rtsp_server_threads[i]->stop(); + rtsp_server_threads[i]->join(); + delete rtsp_server_threads[i]; + rtsp_server_threads[i] = nullptr; + } + + delete[] rtsp_server_threads; + rtsp_server_threads = nullptr; + + if (zm_reload) { + for (std::shared_ptr &monitor : monitors) { + monitor->Reload(); + } + logTerm(); + logInit(log_id_string); + zm_reload = false; + } // end if zm_reload + } // end while ! zm_terminate outer connection loop + + Image::Deinitialise(); + logTerm(); + zmDbClose(); + + return zm_terminate ? 0 : result; +} diff --git a/src/zm_rtsp_server_adts_fifo_source.cpp b/src/zm_rtsp_server_adts_fifo_source.cpp new file mode 100644 index 000000000..50677b519 --- /dev/null +++ b/src/zm_rtsp_server_adts_fifo_source.cpp @@ -0,0 +1,43 @@ +/* --------------------------------------------------------------------------- +** +** ADTS_FifoSource.cpp +** +** ADTS Live555 source +** +** -------------------------------------------------------------------------*/ + +#include "zm_rtsp_server_adts_fifo_source.h" + +#include + +#if HAVE_RTSP_SERVER + +static unsigned const samplingFrequencyTable[16] = { + 96000, 88200, 64000, 48000, + 44100, 32000, 24000, 22050, + 16000, 12000, 11025, 8000, + 7350, 0, 0, 0 +}; +// --------------------------------- +// ADTS ZoneMinder FramedSource +// --------------------------------- +// +ADTS_ZoneMinderFifoSource::ADTS_ZoneMinderFifoSource( + UsageEnvironment& env, + std::string fifo, + unsigned int queueSize + ) + : + ZoneMinderFifoSource(env, fifo, queueSize), + samplingFrequencyIndex(0), + channels(1) +{ + std::ostringstream os; + os << + "profile-level-id=1;" + "mode=AAC-hbr;sizelength=13;indexlength=3;" + "indexdeltalength=3" + << "\r\n"; + m_auxLine.assign(os.str()); +} +#endif // HAVE_RTSP_SERVER diff --git a/src/zm_rtsp_server_adts_fifo_source.h b/src/zm_rtsp_server_adts_fifo_source.h new file mode 100644 index 000000000..7278f1be1 --- /dev/null +++ b/src/zm_rtsp_server_adts_fifo_source.h @@ -0,0 +1,66 @@ +/* --------------------------------------------------------------------------- +** This software is in the public domain, furnished "as is", without technical +** support, and with no warranty, express or implied, as to its usefulness for +** any purpose. +** +** ADTS_ZoneMinderFifoSource.h +** +** ADTS ZoneMinder live555 source +** +** -------------------------------------------------------------------------*/ + +#ifndef ZM_RTSP_SERVER_ADTS_FIFO_SOURCE_H +#define ZM_RTSP_SERVER_ADTS_FIFO_SOURCE_H + +#include "zm_config.h" +#include "zm_rtsp_server_fifo_source.h" + +#if HAVE_RTSP_SERVER +// --------------------------------- +// ADTS(AAC) ZoneMinder FramedSource +// --------------------------------- + +class ADTS_ZoneMinderFifoSource : public ZoneMinderFifoSource { + public: + static ADTS_ZoneMinderFifoSource* createNew( + UsageEnvironment& env, + std::string fifo, + unsigned int queueSize + ) { + return new ADTS_ZoneMinderFifoSource(env, fifo, queueSize); + }; + protected: + ADTS_ZoneMinderFifoSource( + UsageEnvironment& env, + std::string fifo, + unsigned int queueSize + ); + + virtual ~ADTS_ZoneMinderFifoSource() {} + + /* + virtual unsigned char* extractFrame(unsigned char* frame, size_t& size, size_t& outsize); + virtual unsigned char* findMarker(unsigned char *frame, size_t size, size_t &length); + */ + public: + int samplingFrequency() { return 8000; //m_stream->codecpar->sample_rate; + }; + const char *configStr() { return config.c_str(); }; + int numChannels() { + //Debug(1, "this %p m_stream %p channels %d", + //this, m_stream, channels); + //Debug(1, "m_stream %p codecpar %p channels %d => %d", + //m_stream, m_stream->codecpar, m_stream->codecpar->channels, channels); + return 1; + //return channels; + //return m_stream->codecpar->channels; + } + + protected: + std::string config; + int samplingFrequencyIndex; + int channels; +}; +#endif // HAVE_RTSP_SERVER + +#endif // ZM_RTSP_SERVER_ADTS_FIFO_SOURCE_H diff --git a/src/zm_rtsp_server_fifo_source.cpp b/src/zm_rtsp_server_fifo_source.cpp new file mode 100644 index 000000000..7b25058ab --- /dev/null +++ b/src/zm_rtsp_server_fifo_source.cpp @@ -0,0 +1,215 @@ +/* --------------------------------------------------------------------------- +** This software is in the public domain, furnished "as is", without technical +** support, and with no warranty, express or implied, as to its usefulness for +** any purpose. +** +** +** ZoneMinder Live555 source +** +** -------------------------------------------------------------------------*/ + +#include "zm_rtsp_server_fifo_source.h" + +#include "zm_config.h" +#include "zm_logger.h" +#include "zm_rtsp_server_frame.h" +#include "zm_signal.h" + +#include +#include + +#if HAVE_RTSP_SERVER +ZoneMinderFifoSource::ZoneMinderFifoSource( + UsageEnvironment& env, + std::string fifo, + unsigned int queueSize + ) : + FramedSource(env), + m_fifo(fifo), + m_queueSize(queueSize), + m_fd(-1) +{ + m_eventTriggerId = envir().taskScheduler().createEventTrigger(ZoneMinderFifoSource::deliverFrameStub); + memset(&m_thid, 0, sizeof(m_thid)); + memset(&m_mutex, 0, sizeof(m_mutex)); + pthread_mutex_init(&m_mutex, nullptr); + pthread_create(&m_thid, nullptr, threadStub, this); + m_buffer_ptr = &m_buffer[0]; +} + +ZoneMinderFifoSource::~ZoneMinderFifoSource() { + stop = 1; + envir().taskScheduler().deleteEventTrigger(m_eventTriggerId); + pthread_join(m_thid, nullptr); + while ( m_captureQueue.size() ) { + NAL_Frame * f = m_captureQueue.front(); + m_captureQueue.pop_front(); + delete f; + } + + pthread_mutex_destroy(&m_mutex); +} + +// thread mainloop +void* ZoneMinderFifoSource::thread() { + stop = 0; + + while (!stop) { + getNextFrame(); + } + return nullptr; +} + +// getting FrameSource callback +void ZoneMinderFifoSource::doGetNextFrame() { + deliverFrame(); +} + +// stopping FrameSource callback +void ZoneMinderFifoSource::doStopGettingFrames() { + //stop = 1; + Debug(1, "ZoneMinderFifoSource::doStopGettingFrames"); + FramedSource::doStopGettingFrames(); +} + +// deliver frame to the sink +void ZoneMinderFifoSource::deliverFrame() { + if (!isCurrentlyAwaitingData()) { + Debug(4, "not awaiting data"); + return; + } + + pthread_mutex_lock(&m_mutex); + if (m_captureQueue.empty()) { + Debug(4, "Queue is empty"); + pthread_mutex_unlock(&m_mutex); + return; + } + + NAL_Frame *frame = m_captureQueue.front(); + m_captureQueue.pop_front(); + pthread_mutex_unlock(&m_mutex); + + fDurationInMicroseconds = 0; + fFrameSize = 0; + + unsigned int nal_size = frame->size(); + + if (nal_size > fMaxSize) { + fFrameSize = fMaxSize; + fNumTruncatedBytes = nal_size - fMaxSize; + } else { + fFrameSize = nal_size; + } + Debug(2, "deliverFrame timestamp: %ld.%06ld size: %d queuesize: %d", + frame->m_timestamp.tv_sec, frame->m_timestamp.tv_usec, + fFrameSize, + m_captureQueue.size() + ); + + fPresentationTime = frame->m_timestamp; + memcpy(fTo, frame->buffer(), fFrameSize); + + if (fFrameSize > 0) { + // send Frame to the consumer + FramedSource::afterGetting(this); + } + delete frame; +} // end void ZoneMinderFifoSource::deliverFrame() + +// FrameSource callback on read event +void ZoneMinderFifoSource::incomingPacketHandler() { + if (this->getNextFrame() <= 0) { + handleClosure(this); + } +} + +// read from monitor +int ZoneMinderFifoSource::getNextFrame() { + if (zm_terminate) return -1; + + if (m_fd == -1) { + Debug(1, "Opening fifo %s", m_fifo.c_str()); + m_fd = open(m_fifo.c_str(), O_RDONLY); + if (m_fd < 0) { + Error("Can't open %s: %s", m_fifo.c_str(), strerror(errno)); + return -1; + } + } + + int bytes_in_buffer = m_buffer_ptr - &m_buffer[0]; + int bytes_read = read(m_fd, m_buffer_ptr, BUFFER_SIZE-bytes_in_buffer); + if (bytes_read == 0) + return -1; + if (bytes_read < 0) { + Error("Problem during reading: %s", strerror(errno)); + ::close(m_fd); + m_fd = -1; + return -1; + } + bytes_in_buffer += bytes_read; + unsigned int bytes_remaining = bytes_in_buffer; + + timeval tv; + gettimeofday(&tv, nullptr); + + std::list< std::pair > framesList = this->splitFrames(m_buffer, bytes_remaining); + Debug(1, "Got %d frames, bytes remaining %d", framesList.size(), bytes_remaining); + + if ( bytes_remaining > 0 ) { + memmove(&m_buffer[0], &m_buffer[0] + ( bytes_in_buffer - bytes_remaining ), bytes_remaining); + m_buffer_ptr = &m_buffer[0] + bytes_remaining; + } else { + m_buffer_ptr = &m_buffer[0]; + } + + while (framesList.size()) { + std::pair nal = framesList.front(); + framesList.pop_front(); + + NAL_Frame *frame = new NAL_Frame(nal.first, nal.second, tv); + + pthread_mutex_lock(&m_mutex); + if (m_captureQueue.size() > 10) { + NAL_Frame * f = m_captureQueue.front(); + while (m_captureQueue.size() and ((f->m_timestamp.tv_sec - tv.tv_sec) > 2)) { + m_captureQueue.pop_front(); + delete f; + f = m_captureQueue.front(); + } + } +#if 0 + while ( m_captureQueue.size() >= m_queueSize ) { + Debug(2, "Queue full dropping frame %d", m_captureQueue.size()); + NAL_Frame * f = m_captureQueue.front(); + m_captureQueue.pop_front(); + delete f; + } +#endif + m_captureQueue.push_back(frame); + pthread_mutex_unlock(&m_mutex); + + // post an event to ask to deliver the frame + envir().taskScheduler().triggerEvent(m_eventTriggerId, this); + } // end while we get frame from data + return 1; +} + +// split packet in frames +std::list< std::pair > ZoneMinderFifoSource::splitFrames(unsigned char* frame, unsigned &frameSize) { + std::list< std::pair > frameList; + if ( frame != nullptr ) { + frameList.push_back(std::pair(frame, frameSize)); + } + // We consume it all + frameSize = 0; + return frameList; +} + +// extract a frame +unsigned char* ZoneMinderFifoSource::extractFrame(unsigned char* frame, size_t& size, size_t& outsize) { + outsize = size; + size = 0; + return frame; +} +#endif // HAVE_RTSP_SERVER diff --git a/src/zm_rtsp_server_fifo_source.h b/src/zm_rtsp_server_fifo_source.h new file mode 100644 index 000000000..142946de2 --- /dev/null +++ b/src/zm_rtsp_server_fifo_source.h @@ -0,0 +1,82 @@ +/* --------------------------------------------------------------------------- +** +** FifoSource.h +** +** live555 source +** +** -------------------------------------------------------------------------*/ + +#ifndef ZM_RTSP_SERVER_FIFO_SOURCE_H +#define ZM_RTSP_SERVER_FIFO_SOURCE_H + +#include "zm_config.h" +#include "zm_define.h" +#include +#include +#include + +#if HAVE_RTSP_SERVER +#include + +class NAL_Frame; + +class ZoneMinderFifoSource: public FramedSource { + + public: + static ZoneMinderFifoSource* createNew( + UsageEnvironment& env, + std::string fifo, + unsigned int queueSize + ) { + return new ZoneMinderFifoSource(env, fifo, queueSize); + }; + std::string getAuxLine() { return m_auxLine; }; + int getWidth() { return m_width; }; + int getHeight() { return m_height; }; + int setWidth(int width) { return m_width=width; }; + int setHeight(int height) { return m_height=height; }; + + protected: + ZoneMinderFifoSource(UsageEnvironment& env, std::string fifo, unsigned int queueSize); + virtual ~ZoneMinderFifoSource(); + + protected: + static void* threadStub(void* clientData) { return ((ZoneMinderFifoSource*) clientData)->thread();}; + void* thread(); + static void deliverFrameStub(void* clientData) {((ZoneMinderFifoSource*) clientData)->deliverFrame();}; + void deliverFrame(); + static void incomingPacketHandlerStub(void* clientData, int mask) { ((ZoneMinderFifoSource*) clientData)->incomingPacketHandler(); }; + void incomingPacketHandler(); + int getNextFrame(); + void processFrame(char * frame, int frameSize, const timeval &ref); + void queueFrame(char * frame, int frameSize, const timeval &tv); + + // split packet in frames + virtual std::list< std::pair > splitFrames(unsigned char* frame, unsigned &frameSize); + + // overide FramedSource + virtual void doGetNextFrame(); + virtual void doStopGettingFrames(); + virtual unsigned char *extractFrame(unsigned char *data, size_t& size, size_t& outsize); + + protected: + std::list m_captureQueue; + EventTriggerId m_eventTriggerId; + std::string m_fifo; + + int m_width; + int m_height; + unsigned int m_queueSize; + pthread_t m_thid; + pthread_mutex_t m_mutex; + std::string m_auxLine; + int stop; + + int m_fd; + #define BUFFER_SIZE 65536 + unsigned char m_buffer[BUFFER_SIZE]; + unsigned char *m_buffer_ptr; +}; +#endif // HAVE_RTSP_SERVER + +#endif // ZM_RTSP_SERVER_FIFO_SOURCE_H diff --git a/src/zm_rtsp_server_fifo_video_source.cpp b/src/zm_rtsp_server_fifo_video_source.cpp new file mode 100644 index 000000000..bcf1f8841 --- /dev/null +++ b/src/zm_rtsp_server_fifo_video_source.cpp @@ -0,0 +1,20 @@ +/* --------------------------------------------------------------------------- +** This software is in the public domain, furnished "as is", without technical +** support, and with no warranty, express or implied, as to its usefulness for +** any purpose. +** +** +** ZoneMinder Live555 source +** +** -------------------------------------------------------------------------*/ + +#include "zm_rtsp_server_fifo_video_source.h" + +#if HAVE_RTSP_SERVER +ZoneMinderFifoVideoSource::ZoneMinderFifoVideoSource( + UsageEnvironment& env, std::string fifo, unsigned int queueSize) : + ZoneMinderFifoSource(env,fifo,queueSize) +{ +} + +#endif // HAVE_RTSP_SERVER diff --git a/src/zm_rtsp_server_fifo_video_source.h b/src/zm_rtsp_server_fifo_video_source.h new file mode 100644 index 000000000..444d0ca04 --- /dev/null +++ b/src/zm_rtsp_server_fifo_video_source.h @@ -0,0 +1,32 @@ +/* --------------------------------------------------------------------------- +** +** FifoSource.h +** +** live555 source +** +** -------------------------------------------------------------------------*/ + +#ifndef ZM_RTSP_SERVER_FIFO_VIDEO_SOURCE_H +#define ZM_RTSP_SERVER_FIFO_VIDEO_SOURCE_H + +#include "zm_rtsp_server_fifo_source.h" + +#if HAVE_RTSP_SERVER + +class ZoneMinderFifoVideoSource: public ZoneMinderFifoSource { + + public: + int getWidth() { return m_width; }; + int getHeight() { return m_height; }; + int setWidth(int width) { return m_width=width; }; + int setHeight(int height) { return m_height=height; }; + + protected: + ZoneMinderFifoVideoSource(UsageEnvironment& env, std::string fifo, unsigned int queueSize); + + int m_width; + int m_height; +}; +#endif // HAVE_RTSP_SERVER + +#endif // ZM_RTSP_SERVER_FIFO_VIDEO_SOURCE_H diff --git a/src/zm_rtsp_server_h264_fifo_source.cpp b/src/zm_rtsp_server_h264_fifo_source.cpp new file mode 100644 index 000000000..cc8391f7c --- /dev/null +++ b/src/zm_rtsp_server_h264_fifo_source.cpp @@ -0,0 +1,234 @@ +/* --------------------------------------------------------------------------- +** +** H264_FifoSource.cpp +** +** H264 Live555 source +** +** -------------------------------------------------------------------------*/ + +#include "zm_rtsp_server_h264_fifo_source.h" + +#include "zm_config.h" +#include "zm_logger.h" +#include "zm_rtsp_server_frame.h" +#include +#include + +#if HAVE_RTSP_SERVER +// live555 +#include + +// --------------------------------- +// H264 ZoneMinder FramedSource +// --------------------------------- +// +H264_ZoneMinderFifoSource::H264_ZoneMinderFifoSource( + UsageEnvironment& env, + std::string fifo, + unsigned int queueSize, + bool repeatConfig, + bool keepMarker) + : H26X_ZoneMinderFifoSource(env, fifo, queueSize, repeatConfig, keepMarker) +{ + // extradata appears to simply be the SPS and PPS NAL's + //this->splitFrames(m_stream->codecpar->extradata, m_stream->codecpar->extradata_size); +} + +// split packet into frames +std::list< std::pair > H264_ZoneMinderFifoSource::splitFrames(unsigned char* frame, unsigned &frameSize) { + std::list< std::pair > frameList; + + size_t bufSize = frameSize; + size_t size = 0; + unsigned char* buffer = this->extractFrame(frame, bufSize, size); + bool updateAux = false; + while ( buffer != nullptr ) { + switch ( m_frameType & 0x1F ) { + case 7: + Debug(4, "SPS_Size: %d bufSize %d", size, bufSize); + m_sps.assign((char*)buffer, size); + updateAux = true; + break; + case 8: + Debug(4, "PPS_Size: %d bufSize %d", size, bufSize); + m_pps.assign((char*)buffer, size); + updateAux = true; + break; + case 5: + Debug(4, "IDR_Size: %d bufSize %d", size, bufSize); + if ( m_repeatConfig && !m_sps.empty() && !m_pps.empty() ) { + frameList.push_back(std::pair((unsigned char*)m_sps.c_str(), m_sps.size())); + frameList.push_back(std::pair((unsigned char*)m_pps.c_str(), m_pps.size())); + } + break; + default: + Debug(4, "Unknown frametype!? %d %d", m_frameType, m_frameType & 0x1F); + break; + } + + if ( updateAux and !m_sps.empty() and !m_pps.empty() ) { + u_int32_t profile_level_id = 0; + if ( m_sps.size() >= 4 ) profile_level_id = (m_sps[1]<<16)|(m_sps[2]<<8)|m_sps[3]; + + char* sps_base64 = base64Encode(m_sps.c_str(), m_sps.size()); + char* pps_base64 = base64Encode(m_pps.c_str(), m_pps.size()); + + std::ostringstream os; + os << "profile-level-id=" << std::hex << std::setw(6) << std::setfill('0') << profile_level_id; + os << ";sprop-parameter-sets=" << sps_base64 << "," << pps_base64; + os << "a=x-dimensions:" << m_width << "," << m_height << "\r\n"; + m_auxLine.assign(os.str()); + Debug(1, "auxLine: %s", m_auxLine.c_str()); + + delete [] sps_base64; + delete [] pps_base64; + } + frameList.push_back(std::pair(buffer, size)); + + buffer = this->extractFrame(&buffer[size], bufSize, size); + } // end while buffer + frameSize = bufSize; + return frameList; +} + +H265_ZoneMinderFifoSource::H265_ZoneMinderFifoSource( + UsageEnvironment& env, + std::string fifo, + unsigned int queueSize, + bool repeatConfig, + bool keepMarker) + : H26X_ZoneMinderFifoSource(env, fifo, queueSize, repeatConfig, keepMarker) +{ + // extradata appears to simply be the SPS and PPS NAL's + // this->splitFrames(m_stream->codecpar->extradata, m_stream->codecpar->extradata_size); +} + +// split packet in frames +std::list< std::pair > +H265_ZoneMinderFifoSource::splitFrames(unsigned char* frame, unsigned frameSize) { + std::list< std::pair > frameList; + + size_t bufSize = frameSize; + size_t size = 0; + unsigned char* buffer = this->extractFrame(frame, bufSize, size); + while ( buffer != nullptr ) { + switch ((m_frameType&0x7E)>>1) { + case 32: + Debug(4, "VPS_Size: %d bufSize %d", size, bufSize); + m_vps.assign((char*)buffer,size); + break; + case 33: + Debug(4, "SPS_Size: %d bufSize %d", size, bufSize); + m_sps.assign((char*)buffer,size); + break; + case 34: + Debug(4, "PPS_Size: %d bufSize %d", size, bufSize); + m_pps.assign((char*)buffer,size); + break; + case 19: + case 20: + Debug(4, "IDR_Size: %d bufSize %d", size, bufSize); + if ( m_repeatConfig && !m_vps.empty() && !m_sps.empty() && !m_pps.empty() ) { + frameList.push_back(std::pair((unsigned char*)m_vps.c_str(), m_vps.size())); + frameList.push_back(std::pair((unsigned char*)m_sps.c_str(), m_sps.size())); + frameList.push_back(std::pair((unsigned char*)m_pps.c_str(), m_pps.size())); + } + break; + default: + Debug(4, "Unknown frametype!? %d %d", m_frameType, ((m_frameType & 0x7E) >> 1)); + break; + } + + if ( !m_vps.empty() && !m_sps.empty() && !m_pps.empty() ) { + char* vps_base64 = base64Encode(m_vps.c_str(), m_vps.size()); + char* sps_base64 = base64Encode(m_sps.c_str(), m_sps.size()); + char* pps_base64 = base64Encode(m_pps.c_str(), m_pps.size()); + + std::ostringstream os; + os << "sprop-vps=" << vps_base64; + os << ";sprop-sps=" << sps_base64; + os << ";sprop-pps=" << pps_base64; + os << "a=x-dimensions:" << m_width << "," << m_height << "\r\n"; + m_auxLine.assign(os.str()); + Debug(1, "Assigned auxLine to %s", m_auxLine.c_str()); + + delete [] vps_base64; + delete [] sps_base64; + delete [] pps_base64; + } + frameList.push_back(std::pair(buffer, size)); + + buffer = this->extractFrame(&buffer[size], bufSize, size); + } // end while buffer + if ( bufSize ) { + Debug(1, "%d bytes remaining", bufSize); + } + frameSize = bufSize; + return frameList; +} // end H265_ZoneMinderFifoSource::splitFrames(unsigned char* frame, unsigned frameSize) + +unsigned char * H26X_ZoneMinderFifoSource::findMarker( + unsigned char *frame, size_t size, size_t &length + ) { + //Debug(1, "findMarker %p %d", frame, size); + unsigned char *start = nullptr; + for ( size_t i = 0; i < size-2; i += 1 ) { + //Debug(1, "%d: %d %d %d", i, frame[i], frame[i+1], frame[i+2]); + if ( (frame[i] == 0) and (frame[i+1]) == 0 and (frame[i+2] == 1) ) { + if ( i and (frame[i-1] == 0) ) { + start = frame + i - 1; + length = sizeof(H264marker); + } else { + start = frame + i; + length = sizeof(H264shortmarker); + } + break; + } + } + return start; +} + +// extract a frame +unsigned char* H26X_ZoneMinderFifoSource::extractFrame(unsigned char* frame, size_t& size, size_t& outsize) { + unsigned char *outFrame = nullptr; + Debug(4, "ExtractFrame: %p %d", frame, size); + outsize = 0; + size_t markerLength = 0; + size_t endMarkerLength = 0; + m_frameType = 0; + unsigned char *startFrame = nullptr; + if ( size >= 3 ) + startFrame = this->findMarker(frame, size, markerLength); + if ( startFrame != nullptr ) { + Debug(4, "startFrame: %p marker Length %d", startFrame, markerLength); + m_frameType = startFrame[markerLength]; + + int remainingSize = size-(startFrame-frame+markerLength); + unsigned char *endFrame = nullptr; + if ( remainingSize > 3 ) { + endFrame = this->findMarker(startFrame+markerLength, remainingSize, endMarkerLength); + } + Debug(4, "endFrame: %p marker Length %d, remaining size %d", endFrame, endMarkerLength, remainingSize); + + if ( m_keepMarker ) { + size -= startFrame-frame; + outFrame = startFrame; + } else { + size -= startFrame-frame+markerLength; + outFrame = &startFrame[markerLength]; + } + + if ( endFrame != nullptr ) { + outsize = endFrame - outFrame; + } else { + outsize = size; + } + size -= outsize; + Debug(4, "Have frame type: %d size %d, keepmarker %d", m_frameType, outsize, m_keepMarker); + } else if ( size >= sizeof(H264shortmarker) ) { + Info("No marker found size %d", size); + } + + return outFrame; +} +#endif // HAVE_RTSP_SERVER diff --git a/src/zm_rtsp_server_h264_fifo_source.h b/src/zm_rtsp_server_h264_fifo_source.h new file mode 100644 index 000000000..9896c6122 --- /dev/null +++ b/src/zm_rtsp_server_h264_fifo_source.h @@ -0,0 +1,99 @@ +/* --------------------------------------------------------------------------- +** This software is in the public domain, furnished "as is", without technical +** support, and with no warranty, express or implied, as to its usefulness for +** any purpose. +** +** H264_ZoneMinderFifoSource.h +** +** H264 ZoneMinder live555 source +** +** -------------------------------------------------------------------------*/ + +#ifndef ZM_RTSP_H264_FIFO_SOURCE_H +#define ZM_RTSP_H264_FIFO_SOURCE_H + +#include "zm_config.h" +#include "zm_rtsp_server_fifo_video_source.h" + +// --------------------------------- +// H264 ZoneMinder FramedSource +// --------------------------------- +#if HAVE_RTSP_SERVER +class H26X_ZoneMinderFifoSource : public ZoneMinderFifoVideoSource { + protected: + H26X_ZoneMinderFifoSource( + UsageEnvironment& env, + std::string fifo, + unsigned int queueSize, + bool repeatConfig, + bool keepMarker) + : + ZoneMinderFifoVideoSource(env, fifo, queueSize), + m_repeatConfig(repeatConfig), + m_keepMarker(keepMarker), + m_frameType(0) { } + + virtual ~H26X_ZoneMinderFifoSource() {} + + virtual unsigned char* extractFrame(unsigned char* frame, size_t& size, size_t& outsize); + virtual unsigned char* findMarker(unsigned char *frame, size_t size, size_t &length); + + protected: + std::string m_sps; + std::string m_pps; + bool m_repeatConfig; + bool m_keepMarker; + int m_frameType; +}; + +class H264_ZoneMinderFifoSource : public H26X_ZoneMinderFifoSource { + public: + static H264_ZoneMinderFifoSource* createNew( + UsageEnvironment& env, + std::string fifo, + unsigned int queueSize, + bool repeatConfig, + bool keepMarker) { + return new H264_ZoneMinderFifoSource(env, fifo, queueSize, repeatConfig, keepMarker); + } + + protected: + H264_ZoneMinderFifoSource( + UsageEnvironment& env, + std::string fifo, + unsigned int queueSize, + bool repeatConfig, + bool keepMarker); + + // overide ZoneMinderFifoSource + virtual std::list< std::pair > splitFrames(unsigned char* frame, unsigned &frameSize); +}; + +class H265_ZoneMinderFifoSource : public H26X_ZoneMinderFifoSource { + public: + static H265_ZoneMinderFifoSource* createNew( + UsageEnvironment& env, + std::string fifo, + unsigned int queueSize, + bool repeatConfig, + bool keepMarker) { + return new H265_ZoneMinderFifoSource(env, fifo, queueSize, repeatConfig, keepMarker); + } + + protected: + H265_ZoneMinderFifoSource( + UsageEnvironment& env, + std::string fifo, + unsigned int queueSize, + bool repeatConfig, + bool keepMarker); + + // overide ZoneMinderFifoSource + virtual std::list< std::pair > splitFrames(unsigned char* frame, unsigned frameSize); + + protected: + std::string m_vps; +}; +#endif // HAVE_RTSP_SERVER + +#endif // ZM_RTSP_H264_FIFO_SOURCE_H diff --git a/src/zm_rtsp_server_server_media_subsession.cpp b/src/zm_rtsp_server_server_media_subsession.cpp index 72702c02d..87f3fdfd7 100644 --- a/src/zm_rtsp_server_server_media_subsession.cpp +++ b/src/zm_rtsp_server_server_media_subsession.cpp @@ -8,6 +8,7 @@ #include "zm_config.h" #include "zm_rtsp_server_adts_source.h" +#include "zm_rtsp_server_adts_fifo_source.h" #include #if HAVE_RTSP_SERVER @@ -15,16 +16,18 @@ // BaseServerMediaSubsession // --------------------------------- FramedSource* BaseServerMediaSubsession::createSource( - UsageEnvironment& env, FramedSource* inputSource, const std::string& format) + UsageEnvironment& env, + FramedSource* inputSource, + const std::string& format) { FramedSource* source = nullptr; - if ( format == "video/MP2T" ) { + if (format == "video/MP2T") { source = MPEG2TransportStreamFramer::createNew(env, inputSource); - } else if ( format == "video/H264" ) { + } else if (format == "video/H264") { source = H264VideoStreamDiscreteFramer::createNew(env, inputSource); } #if LIVEMEDIA_LIBRARY_VERSION_INT > 1414454400 - else if ( format == "video/H265" ) { + else if (format == "video/H265") { source = H265VideoStreamDiscreteFramer::createNew(env, inputSource); } #endif @@ -49,21 +52,21 @@ RTPSink* BaseServerMediaSubsession::createSink( ) { RTPSink* sink = nullptr; - if ( format == "video/MP2T" ) { + if (format == "video/MP2T") { sink = SimpleRTPSink::createNew(env, rtpGroupsock, rtpPayloadTypeIfDynamic, 90000, "video", "MP2T", 1, true, false); - } else if ( format == "video/H264" ) { + } else if (format == "video/H264") { sink = H264VideoRTPSink::createNew(env, rtpGroupsock, rtpPayloadTypeIfDynamic); - } else if ( format == "video/VP8" ) { + } else if (format == "video/VP8") { sink = VP8VideoRTPSink::createNew(env, rtpGroupsock, rtpPayloadTypeIfDynamic); } #if LIVEMEDIA_LIBRARY_VERSION_INT > 1414454400 - else if ( format == "video/VP9" ) { + else if (format == "video/VP9") { sink = VP9VideoRTPSink::createNew(env, rtpGroupsock, rtpPayloadTypeIfDynamic); - } else if ( format == "video/H265" ) { + } else if (format == "video/H265") { sink = H265VideoRTPSink::createNew(env, rtpGroupsock, rtpPayloadTypeIfDynamic); #endif - } else if ( format == "audio/AAC" ) { - ADTS_ZoneMinderDeviceSource *adts_source = (ADTS_ZoneMinderDeviceSource *)(m_replicator->inputSource()); + } else if (format == "audio/AAC") { + ADTS_ZoneMinderFifoSource *adts_source = (ADTS_ZoneMinderFifoSource *)(m_replicator->inputSource()); sink = MPEG4GenericRTPSink::createNew(env, rtpGroupsock, rtpPayloadTypeIfDynamic, adts_source->samplingFrequency(), @@ -83,24 +86,20 @@ RTPSink* BaseServerMediaSubsession::createSink( } char const* BaseServerMediaSubsession::getAuxLine( - ZoneMinderDeviceSource* source, + ZoneMinderFifoSource* source, unsigned char rtpPayloadType ) { const char* auxLine = nullptr; - if ( source ) { + if (source) { std::ostringstream os; os << "a=fmtp:" << int(rtpPayloadType) << " "; os << source->getAuxLine(); os << "\r\n"; - int width = source->getWidth(); - int height = source->getHeight(); - if ( (width > 0) && (height>0) ) { - os << "a=x-dimensions:" << width << "," << height << "\r\n"; - } auxLine = strdup(os.str().c_str()); Debug(1, "auxLine: %s", auxLine); } else { - Error("No source auxLine: "); + Error("No source auxLine:"); + return ""; } return auxLine; } diff --git a/src/zm_rtsp_server_server_media_subsession.h b/src/zm_rtsp_server_server_media_subsession.h index 4b6166b44..d3780cd6e 100644 --- a/src/zm_rtsp_server_server_media_subsession.h +++ b/src/zm_rtsp_server_server_media_subsession.h @@ -11,6 +11,7 @@ #define ZM_RTSP_SERVER_SERVER_MEDIA_SUBSESSION_H #include "zm_config.h" +#include "zm_rtsp_server_fifo_source.h" #include #if HAVE_RTSP_SERVER @@ -36,7 +37,7 @@ class BaseServerMediaSubsession { FramedSource *source); char const* getAuxLine( - ZoneMinderDeviceSource* source, + ZoneMinderFifoSource* source, unsigned char rtpPayloadType); protected: diff --git a/src/zm_rtsp_server_thread.cpp b/src/zm_rtsp_server_thread.cpp index 1b5c2f61a..de6c9addb 100644 --- a/src/zm_rtsp_server_thread.cpp +++ b/src/zm_rtsp_server_thread.cpp @@ -2,7 +2,9 @@ #include "zm_config.h" #include "zm_rtsp_server_adts_source.h" +#include "zm_rtsp_server_adts_fifo_source.h" #include "zm_rtsp_server_h264_device_source.h" +#include "zm_rtsp_server_h264_fifo_source.h" #include "zm_rtsp_server_unicast_server_media_subsession.h" #if HAVE_RTSP_SERVER @@ -31,7 +33,6 @@ RTSPServerThread::RTSPServerThread(std::shared_ptr monitor) : return; } const char *prefix = rtspServer->rtspURLPrefix(); - Debug(1, "RTSP prefix is %s", prefix); delete[] prefix; } // end RTSPServerThread::RTSPServerThread @@ -68,7 +69,60 @@ bool RTSPServerThread::stopped() const { return terminate ? true : false; } // end RTSPServerThread::stopped() -void RTSPServerThread::addStream(AVStream *video_stream, AVStream *audio_stream) { +ServerMediaSession *RTSPServerThread::addSession(std::string &streamname) { + ServerMediaSession *sms = ServerMediaSession::createNew(*env, streamname.c_str()); + if (sms) { + rtspServer->addServerMediaSession(sms); + char *url = rtspServer->rtspURL(sms); + Debug(1, "url is %s for stream %s", url, streamname.c_str()); + delete[] url; + } + return sms; +} + +FramedSource *RTSPServerThread::addFifo( + ServerMediaSession *sms, + std::string fifo) { + if (!rtspServer) return nullptr; + + int queueSize = 30; + bool repeatConfig = true; + bool muxTS = false; + FramedSource *source = nullptr; + + if (!fifo.empty()) { + StreamReplicator* replicator = nullptr; + std::string rtpFormat; + if (fifo.find("h264")) { + rtpFormat = "video/H264"; + source = H264_ZoneMinderFifoSource::createNew(*env, fifo, queueSize, repeatConfig, muxTS); + } else if (fifo.find("h265")) { + rtpFormat = "video/H265"; + source = H265_ZoneMinderFifoSource::createNew(*env, fifo, queueSize, repeatConfig, muxTS); + } else if (fifo.find("aac")) { + rtpFormat = "audio/AAC"; + source = ADTS_ZoneMinderFifoSource::createNew(*env, fifo, queueSize); + Debug(1, "ADTS source %p", source); + } else { + Warning("Unknown format in %s", fifo.c_str()); + } + if (source == nullptr) { + Error("Unable to create source"); + } else { + replicator = StreamReplicator::createNew(*env, source, false); + } + sources.push_back(source); + + if (replicator) { + sms->addSubsession(UnicastServerMediaSubsession::createNew(*env, replicator, rtpFormat)); + } + } else { + Debug(1, "Not Adding stream"); + } + return source; +} // end void addFifo + +void RTSPServerThread::addStream(std::string &streamname, AVStream *video_stream, AVStream *audio_stream) { if ( !rtspServer ) return; @@ -101,7 +155,7 @@ void RTSPServerThread::addStream(AVStream *video_stream, AVStream *audio_stream) // Create Unicast Session if ( videoReplicator ) { if ( !sms ) - sms = ServerMediaSession::createNew(*env, "streamname"); + sms = ServerMediaSession::createNew(*env, streamname.c_str()); sms->addSubsession(UnicastServerMediaSubsession::createNew(*env, videoReplicator, rtpFormat)); } } @@ -119,7 +173,7 @@ void RTSPServerThread::addStream(AVStream *video_stream, AVStream *audio_stream) } if ( replicator ) { if ( !sms ) - sms = ServerMediaSession::createNew(*env, "streamname"); + sms = ServerMediaSession::createNew(*env, streamname.c_str()); sms->addSubsession(UnicastServerMediaSubsession::createNew(*env, replicator, rtpFormat)); } } else { diff --git a/src/zm_rtsp_server_thread.h b/src/zm_rtsp_server_thread.h index 57fc33bc9..97edb05de 100644 --- a/src/zm_rtsp_server_thread.h +++ b/src/zm_rtsp_server_thread.h @@ -4,6 +4,8 @@ #include "zm_config.h" #include "zm_ffmpeg.h" #include "zm_thread.h" +#include "zm_rtsp_server_server_media_subsession.h" +#include "zm_rtsp_server_fifo_source.h" #include #include @@ -28,7 +30,9 @@ class RTSPServerThread : public Thread { public: explicit RTSPServerThread(std::shared_ptr monitor); ~RTSPServerThread(); - void addStream(AVStream *, AVStream *); + ServerMediaSession *addSession(std::string &streamname); + void addStream(std::string &streamname, AVStream *, AVStream *); + FramedSource *addFifo(ServerMediaSession *sms, std::string fifo); int run(); void stop(); bool stopped() const; diff --git a/src/zm_rtsp_server_unicast_server_media_subsession.cpp b/src/zm_rtsp_server_unicast_server_media_subsession.cpp index 0b5566860..ede9ce527 100644 --- a/src/zm_rtsp_server_unicast_server_media_subsession.cpp +++ b/src/zm_rtsp_server_unicast_server_media_subsession.cpp @@ -11,6 +11,7 @@ #include "zm_config.h" #include "zm_rtsp_server_device_source.h" +#include "zm_rtsp_server_fifo_source.h" #if HAVE_RTSP_SERVER // ----------------------------------------- @@ -45,6 +46,8 @@ RTPSink* UnicastServerMediaSubsession::createNewRTPSink( char const* UnicastServerMediaSubsession::getAuxSDPLine( RTPSink* rtpSink, FramedSource* inputSource ) { - return this->getAuxLine(dynamic_cast(m_replicator->inputSource()), rtpSink->rtpPayloadType()); + return this->getAuxLine( + dynamic_cast(m_replicator->inputSource()), + rtpSink->rtpPayloadType()); } #endif // HAVE_RTSP_SERVER From d2e1c231408f2cfe9dafd5e55e7009a7dad7bdd9 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sat, 27 Feb 2021 12:26:37 -0500 Subject: [PATCH 0012/1277] Add zm_rtsp_server build target and source files --- src/CMakeLists.txt | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index 46c1a0368..0bc9f8608 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -50,10 +50,15 @@ set(ZM_BIN_SRC_FILES zm_rtp_source.cpp zm_rtsp.cpp zm_rtsp_auth.cpp + zm_rtsp_server.cpp zm_rtsp_server_thread.cpp zm_rtsp_server_adts_source.cpp + zm_rtsp_server_adts_fifo_source.cpp zm_rtsp_server_h264_device_source.cpp + zm_rtsp_server_h264_fifo_source.cpp zm_rtsp_server_device_source.cpp + zm_rtsp_server_fifo_source.cpp + zm_rtsp_server_fifo_video_source.cpp zm_rtsp_server_server_media_subsession.cpp zm_rtsp_server_unicast_server_media_subsession.cpp zm_sdp.cpp @@ -85,8 +90,9 @@ target_link_libraries(zm zm-core-interface) add_executable(zmc zmc.cpp) -add_executable(zmu zmu.cpp) +add_executable(zm_rtsp_server zm_rtsp_server.cpp) add_executable(zms zms.cpp) +add_executable(zmu zmu.cpp) target_link_libraries(zmc PRIVATE @@ -96,7 +102,7 @@ target_link_libraries(zmc ${ZM_BIN_LIBS} ${CMAKE_DL_LIBS}) -target_link_libraries(zmu +target_link_libraries(zm_rtsp_server PRIVATE zm-core-interface zm @@ -112,6 +118,15 @@ target_link_libraries(zms ${ZM_BIN_LIBS} ${CMAKE_DL_LIBS}) +target_link_libraries(zmu + PRIVATE + zm-core-interface + zm + ${ZM_EXTRA_LIBS} + ${ZM_BIN_LIBS} + ${CMAKE_DL_LIBS} + bcrypt) + # Generate man files for the binaries destined for the bin folder if(BUILD_MAN) foreach(CBINARY zmc zmu) @@ -119,7 +134,7 @@ if(BUILD_MAN) endforeach(CBINARY zmc zmu) endif() -install(TARGETS zmc zmu RUNTIME DESTINATION "${CMAKE_INSTALL_FULL_BINDIR}" PERMISSIONS OWNER_WRITE OWNER_READ OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE) +install(TARGETS zmc zm_rtsp_server zmu RUNTIME DESTINATION "${CMAKE_INSTALL_FULL_BINDIR}" PERMISSIONS OWNER_WRITE OWNER_READ OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE) install(TARGETS zms RUNTIME DESTINATION "${ZM_CGIDIR}" PERMISSIONS OWNER_WRITE OWNER_READ OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE) install(CODE "execute_process(COMMAND ln -sf zms nph-zms WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})") install(FILES ${CMAKE_CURRENT_BINARY_DIR}/nph-zms DESTINATION "${ZM_CGIDIR}") From 78062a03efa92c55725bb5ec4e05d96a9f861b5a Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sat, 27 Feb 2021 12:27:10 -0500 Subject: [PATCH 0013/1277] Add writePacket and write static function to write h264 packet stream and audio stream to a fifo for the zm_rtsp_server to read from --- src/zm_fifo.cpp | 62 +++++++++++++++++++++++++++++++++++++++++++++++++ src/zm_fifo.h | 3 +++ 2 files changed, 65 insertions(+) diff --git a/src/zm_fifo.cpp b/src/zm_fifo.cpp index 3e09f8886..8cbe020ed 100644 --- a/src/zm_fifo.cpp +++ b/src/zm_fifo.cpp @@ -269,3 +269,65 @@ void FifoStream::runStream() { } close(fd_lock); } + +bool FifoStream::writePacket(std::string filename, ZMPacket &packet) { + bool on_blocking_abort = true; + FILE *outfile = nullptr; + int raw_fd = 0; + + if ( !on_blocking_abort ) { + if ( (outfile = fopen(filename.c_str(), "wb")) == nullptr ) { + Error("Can't open %s for writing: %s", filename, strerror(errno)); + return false; + } + } else { + raw_fd = open(filename.c_str(), O_WRONLY|O_NONBLOCK|O_CREAT|O_TRUNC,S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH); + if (raw_fd < 0) + return false; + outfile = fdopen(raw_fd, "wb"); + if (outfile == nullptr) { + close(raw_fd); + return false; + } + } + + if (fwrite(packet.packet.data, packet.packet.size, 1, outfile) != 1) { + Debug(1, "Unable to write to '%s': %s", filename.c_str(), strerror(errno)); + fclose(outfile); + return false; + } + + fclose(outfile); + return true; +} + +bool FifoStream::write(std::string filename, uint8_t *data, size_t bytes) { + bool on_blocking_abort = true; + FILE *outfile = nullptr; + int raw_fd = 0; + + if ( !on_blocking_abort ) { + if ( (outfile = fopen(filename.c_str(), "wb")) == nullptr ) { + Error("Can't open %s for writing: %s", filename, strerror(errno)); + return false; + } + } else { + raw_fd = open(filename.c_str(), O_WRONLY|O_NONBLOCK|O_CREAT|O_TRUNC,S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH); + if (raw_fd < 0) + return false; + outfile = fdopen(raw_fd, "wb"); + if (outfile == nullptr) { + close(raw_fd); + return false; + } + } + + if (fwrite(data, bytes, 1, outfile) != 1) { + Debug(1, "Unable to write to '%s': %s", filename.c_str(), strerror(errno)); + fclose(outfile); + return false; + } + + fclose(outfile); + return true; +} diff --git a/src/zm_fifo.h b/src/zm_fifo.h index e99d56dd2..92fa5139b 100644 --- a/src/zm_fifo.h +++ b/src/zm_fifo.h @@ -20,6 +20,7 @@ #define ZM_FIFO_H #include "zm_stream.h" +#include "zm_packet.h" class Monitor; @@ -74,5 +75,7 @@ class FifoStream : public StreamBase { void setStreamStart(const char * path); void setStreamStart(int monitor_id, const char * format); void runStream() override; + static bool writePacket(std::string filename, ZMPacket &packet); + static bool write(std::string filename, uint8_t *data, size_t size); }; #endif // ZM_FIFO_H From 57585e49868c5efb1d4fa34bf2680855a53d61e8 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sat, 27 Feb 2021 12:27:27 -0500 Subject: [PATCH 0014/1277] Not sure why raw_fd should be static... --- src/zm_image.cpp | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/zm_image.cpp b/src/zm_image.cpp index 704e2d374..a06a23c5b 100644 --- a/src/zm_image.cpp +++ b/src/zm_image.cpp @@ -1067,8 +1067,7 @@ bool Image::WriteJpeg(const char *filename, int quality_override, struct timeval struct jpeg_compress_struct *cinfo = writejpg_ccinfo[quality]; FILE *outfile = nullptr; - static int raw_fd = 0; - raw_fd = 0; + int raw_fd = 0; if ( !cinfo ) { cinfo = writejpg_ccinfo[quality] = new jpeg_compress_struct; From 3e9a2f265117adda7d9be0646c14e5570d47782f Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sat, 27 Feb 2021 12:28:01 -0500 Subject: [PATCH 0015/1277] It is actually importatnt that smInstance get cleared. The subsequent new Logger will check for it. --- src/zm_logger.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/src/zm_logger.cpp b/src/zm_logger.cpp index 16f66835c..303500e20 100644 --- a/src/zm_logger.cpp +++ b/src/zm_logger.cpp @@ -559,6 +559,7 @@ void Logger::logPrint(bool hex, const char * const filepath, const int line, con void logInit(const char *name, const Logger::Options &options) { if (Logger::smInstance) { delete Logger::smInstance; + Logger::smInstance = nullptr; } Logger::smInstance = new Logger(); From 6c6d6c51b08d16ae7ec5ee8e8294b285d13c46e9 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sat, 27 Feb 2021 12:28:27 -0500 Subject: [PATCH 0016/1277] Add RTSPSTreamName loading and write packet data to the video and audio fifos --- src/zm_monitor.cpp | 158 ++++++++++++++++++++++++++++++--------------- 1 file changed, 105 insertions(+), 53 deletions(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 2d72f9eb6..2a53ce2c5 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -87,7 +87,9 @@ std::string load_monitor_sql = "`EventPrefix`, `LabelFormat`, `LabelX`, `LabelY`, `LabelSize`," "`ImageBufferCount`, `WarmupCount`, `PreEventCount`, `PostEventCount`, `StreamReplayBuffer`, `AlarmFrameCount`, " "`SectionLength`, `MinSectionLength`, `FrameSkip`, `MotionFrameSkip`, " -"`FPSReportInterval`, `RefBlendPerc`, `AlarmRefBlendPerc`, `TrackMotion`, `Exif`, `RTSPServer`, `SignalCheckPoints`, `SignalCheckColour` FROM `Monitors`"; +"`FPSReportInterval`, `RefBlendPerc`, `AlarmRefBlendPerc`, `TrackMotion`, `Exif`," +"`RTSPServer`, `RTSPStreamName`," +"`SignalCheckPoints`, `SignalCheckColour` FROM `Monitors`"; std::string CameraType_Strings[] = { "Local", @@ -227,41 +229,41 @@ bool Monitor::MonitorLink::disconnect() { shm_id = 0; - if ( shm_data.shm_nattch <= 1 ) { - if ( shmctl( shm_id, IPC_RMID, 0 ) < 0 ) { - Debug( 3, "Can't shmctl: %s", strerror(errno) ); - return( false ); + if (shm_data.shm_nattch <= 1) { + if (shmctl(shm_id, IPC_RMID, 0) < 0) { + Debug(3, "Can't shmctl: %s", strerror(errno)); + return false; } } - if ( shmdt( mem_ptr ) < 0 ) { - Debug( 3, "Can't shmdt: %s", strerror(errno) ); - return( false ); + if (shmdt(mem_ptr) < 0) { + Debug(3, "Can't shmdt: %s", strerror(errno)); + return false; } #endif // ZM_MEM_MAPPED mem_size = 0; mem_ptr = nullptr; } - return( true ); + return true; } bool Monitor::MonitorLink::isAlarmed() { - if ( !connected ) { - return( false ); + if (!connected) { + return false; } return( shared_data->state == ALARM ); } bool Monitor::MonitorLink::inAlarm() { - if ( !connected ) { - return( false ); + if (!connected) { + return false; } return( shared_data->state == ALARM || shared_data->state == ALERT ); } bool Monitor::MonitorLink::hasAlarmed() { - if ( shared_data->state == ALARM ) { + if (shared_data->state == ALARM) { return true; } last_event_id = shared_data->last_event_id; @@ -340,6 +342,7 @@ Monitor::Monitor() signal_check_colour(0), embed_exif(0), rtsp_server(0), + rtsp_streamname(""), capture_max_fps(0), purpose(QUERY), last_camera_bytes(0), @@ -361,6 +364,21 @@ Monitor::Monitor() last_analysis_fps_time(0), auto_resume_time(0), last_motion_score(0), + event_close_mode(CLOSE_IDLE), +#if ZM_MEM_MAPPED + map_fd(-1), + mem_file(""), +#else // ZM_MEM_MAPPED + shm_id(-1), +#endif // ZM_MEM_MAPPED + mem_size(0), + mem_ptr(nullptr), + shared_data(nullptr), + trigger_data(nullptr), + video_store_data(nullptr), + shared_timestamps(nullptr), + shared_images(nullptr), + camera(nullptr), event(nullptr), storage(nullptr), @@ -401,7 +419,9 @@ Monitor::Monitor() "EventPrefix, LabelFormat, LabelX, LabelY, LabelSize," "ImageBufferCount, WarmupCount, PreEventCount, PostEventCount, StreamReplayBuffer, AlarmFrameCount, " "SectionLength, MinSectionLength, FrameSkip, MotionFrameSkip, " - "FPSReportInterval, RefBlendPerc, AlarmRefBlendPerc, TrackMotion, Exif, `RTSPServer`, SignalCheckPoints, SignalCheckColour FROM Monitors"; + "FPSReportInterval, RefBlendPerc, AlarmRefBlendPerc, TrackMotion, Exif," + "`RTSPServer`,`RTSPStreamName`, + "SignalCheckPoints, SignalCheckColour FROM Monitors"; */ void Monitor::Load(MYSQL_ROW dbrow, bool load_zones=true, Purpose p = QUERY) { @@ -570,6 +590,7 @@ void Monitor::Load(MYSQL_ROW dbrow, bool load_zones=true, Purpose p = QUERY) { embed_exif = (*dbrow[col] != '0'); col++; rtsp_server = (*dbrow[col] != '0'); col++; + rtsp_streamname = dbrow[col]; col++; signal_check_points = atoi(dbrow[col]); col++; signal_check_colour = strtol(dbrow[col][0] == '#' ? dbrow[col]+1 : dbrow[col], 0, 16); col++; @@ -642,6 +663,7 @@ void Monitor::Load(MYSQL_ROW dbrow, bool load_zones=true, Purpose p = QUERY) { } } // end if purpose + Debug(1, "Loaded monitor %d(%s), %d zones", id, name, n_zones); } // Monitor::Load @@ -864,14 +886,14 @@ bool Monitor::connect() { Debug(3, "Connecting to monitor. Purpose is %d", purpose); #if ZM_MEM_MAPPED snprintf(mem_file, sizeof(mem_file), "%s/zm.mmap.%d", staticConfig.PATH_MAP.c_str(), id); - if ( purpose != CAPTURE ) { + if (purpose != CAPTURE) { map_fd = open(mem_file, O_RDWR); } else { map_fd = open(mem_file, O_RDWR|O_CREAT, (mode_t)0660); } - if ( map_fd < 0 ) { - Error("Can't open memory map file %s, probably not enough space free: %s", mem_file, strerror(errno)); + if (map_fd < 0) { + Error("Can't open memory map file %s: %s", mem_file, strerror(errno)); return false; } else { Debug(3, "Success opening mmap file at (%s)", mem_file); @@ -990,6 +1012,8 @@ bool Monitor::connect() { shared_data->format = camera->SubpixelOrder(); shared_data->imagesize = camera->ImageSize(); shared_data->alarm_cause[0] = 0; + shared_data->video_fifo[0] = 0; + shared_data->audio_fifo[0] = 0; shared_data->last_frame_score = 0; trigger_data->size = sizeof(TriggerData); trigger_data->trigger_state = TriggerState::TRIGGER_CANCEL; @@ -1260,13 +1284,10 @@ double Monitor::GetFPS() const { /* I think this returns the # of micro seconds that we should sleep in order to maintain the desired analysis rate */ useconds_t Monitor::GetAnalysisRate() { - Debug(1, "Here"); double capture_fps = get_capture_fps(); - Debug(1, "Here"); if ( !analysis_fps_limit ) { return 0; } else if ( analysis_fps_limit > capture_fps ) { - Debug(1, "Here"); if ( last_fps_time != last_analysis_fps_time ) { // At startup they are equal, should never be equal again Warning("Analysis fps (%.2f) is greater than capturing fps (%.2f)", analysis_fps_limit, capture_fps); @@ -2208,9 +2229,23 @@ bool Monitor::Analyse() { trigger_data->trigger_state = TriggerState::TRIGGER_CANCEL; } // end if ( trigger_data->trigger_state != TRIGGER_OFF ) - if ( event ) event->AddPacket(snap); + if (event) event->AddPacket(snap); + if (snap->packet.stream_index == video_stream_id) { + if (shared_data->video_fifo[0]) { + if ( snap->keyframe ) { + AVStream *stream = camera->get_VideoStream(); + FifoStream::write(shared_data->video_fifo, + static_cast(stream->codecpar->extradata), + stream->codecpar->extradata_size); + } + FifoStream::writePacket(shared_data->video_fifo, *snap); + } + } else if (snap->packet.stream_index == video_stream_id) { + if (shared_data->audio_fifo[0]) + FifoStream::writePacket(shared_data->audio_fifo, *snap); + } - if ( videowriter == PASSTHROUGH and ! savejpegs ) { + if ((videowriter == PASSTHROUGH) and !savejpegs) { // Don't need raw images anymore delete snap->image; snap->image = nullptr; @@ -2362,7 +2397,8 @@ void Monitor::ReloadLinkedMonitors(const char *p_linked_monitors) { } // end if p_linked_monitors } // end void Monitor::ReloadLinkedMonitors(const char *p_linked_monitors) -std::vector> Monitor::LoadMonitors(std::string sql, Purpose purpose) { +std::vector> Monitor::LoadMonitors(std::string where, Purpose purpose) { + std::string sql = load_monitor_sql + " WHERE " + where; Debug(1, "Loading Monitors with %s", sql.c_str()); MYSQL_RES *result = zmDbFetch(sql.c_str()); @@ -2392,48 +2428,46 @@ std::vector> Monitor::LoadMonitors(std::string sql, Pur } #if ZM_HAS_V4L -std::vector> Monitor::LoadLocalMonitors(const char *device, Purpose purpose) { +std::vector> Monitor::LoadLocalMonitors +(const char *device, Purpose purpose) { - std::string sql = load_monitor_sql + " WHERE `Function` != 'None' AND `Type` = 'Local'"; + std::string where = "`Function` != 'None' AND `Type` = 'Local'"; if ( device[0] ) - sql += " AND `Device`='" + std::string(device) + "'"; - if ( staticConfig.SERVER_ID ) - sql += stringtf(" AND `ServerId`=%d", staticConfig.SERVER_ID); - return LoadMonitors(sql, purpose); + where += " AND `Device`='" + std::string(device) + "'"; + if (staticConfig.SERVER_ID) + where += stringtf(" AND `ServerId`=%d", staticConfig.SERVER_ID); + return LoadMonitors(where, purpose); } #endif // ZM_HAS_V4L -std::vector> Monitor::LoadRemoteMonitors(const char *protocol, const char *host, const char *port, const char *path, Purpose purpose) { - std::string sql = load_monitor_sql + " WHERE `Function` != 'None' AND `Type` = 'Remote'"; - if ( staticConfig.SERVER_ID ) - sql += stringtf(" AND `ServerId`=%d", staticConfig.SERVER_ID); - - if ( protocol ) - sql += stringtf(" AND `Protocol` = '%s' AND `Host` = '%s' AND `Port` = '%s' AND `Path` = '%s'", protocol, host, port, path); - return LoadMonitors(sql, purpose); +std::vector> Monitor::LoadRemoteMonitors +(const char *protocol, const char *host, const char *port, const char *path, Purpose purpose) { + std::string where = "`Function` != 'None' AND `Type` = 'Remote'"; + if (staticConfig.SERVER_ID) + where += stringtf(" AND `ServerId`=%d", staticConfig.SERVER_ID); + if (protocol) + where += stringtf(" AND `Protocol` = '%s' AND `Host` = '%s' AND `Port` = '%s' AND `Path` = '%s'", protocol, host, port, path); + return LoadMonitors(where, purpose); } std::vector> Monitor::LoadFileMonitors(const char *file, Purpose purpose) { - std::string sql = load_monitor_sql + " WHERE `Function` != 'None' AND `Type` = 'File'"; - if ( file[0] ) - sql += " AND `Path`='" + std::string(file) + "'"; - if ( staticConfig.SERVER_ID ) { - sql += stringtf(" AND `ServerId`=%d", staticConfig.SERVER_ID); - } - return LoadMonitors(sql, purpose); + std::string where = "`Function` != 'None' AND `Type` = 'File'"; + if (file[0]) + where += " AND `Path`='" + std::string(file) + "'"; + if (staticConfig.SERVER_ID) + where += stringtf(" AND `ServerId`=%d", staticConfig.SERVER_ID); + return LoadMonitors(where, purpose); } #if HAVE_LIBAVFORMAT std::vector> Monitor::LoadFfmpegMonitors(const char *file, Purpose purpose) { - std::string sql = load_monitor_sql + " WHERE `Function` != 'None' AND `Type` = 'Ffmpeg'"; - if ( file[0] ) - sql += " AND `Path` = '" + std::string(file) + "'"; - - if ( staticConfig.SERVER_ID ) { - sql += stringtf(" AND `ServerId`=%d", staticConfig.SERVER_ID); - } - return LoadMonitors(sql, purpose); + std::string where = "`Function` != 'None' AND `Type` = 'Ffmpeg'"; + if (file[0]) + where += " AND `Path` = '" + std::string(file) + "'"; + if (staticConfig.SERVER_ID) + where += stringtf(" AND `ServerId`=%d", staticConfig.SERVER_ID); + return LoadMonitors(where, purpose); } #endif // HAVE_LIBAVFORMAT @@ -2914,6 +2948,24 @@ int Monitor::PrimeCapture() { Debug(2, "Video stream id is %d, audio is %d, minimum_packets to keep in buffer %d", video_stream_id, audio_stream_id, pre_event_buffer_count); + + if (rtsp_server) { + if (video_stream_id >= 0) { + AVStream *videoStream = camera->get_VideoStream(); + snprintf(shared_data->video_fifo, sizeof(shared_data->video_fifo)-1, "%s/video_fifo_%d.%s", + staticConfig.PATH_SOCKS.c_str(), + id, + avcodec_get_name(videoStream->codecpar->codec_id)); + FifoStream::fifo_create_if_missing(shared_data->video_fifo); + } + if (record_audio and (audio_stream_id >= 0)) { + AVStream *audioStream = camera->get_AudioStream(); + snprintf(shared_data->audio_fifo, sizeof(shared_data->audio_fifo)-1, "%s/video_fifo_%d.%s", + staticConfig.PATH_SOCKS.c_str(), id, + avcodec_get_name(audioStream->codecpar->codec_id)); + FifoStream::fifo_create_if_missing(shared_data->audio_fifo); + } + } // end if rtsp_server } else { Debug(2, "Failed to prime %d", ret); } From cafdd02307b123cb430b44559e8858f7a6e6a5c8 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sat, 27 Feb 2021 12:28:44 -0500 Subject: [PATCH 0017/1277] Remote rtsp server code. It got moved to zm_rtsp_server.cpp --- src/zmc.cpp | 38 +------------------------------------- 1 file changed, 1 insertion(+), 37 deletions(-) diff --git a/src/zmc.cpp b/src/zmc.cpp index b197e5019..268aca845 100644 --- a/src/zmc.cpp +++ b/src/zmc.cpp @@ -58,6 +58,7 @@ possible, this should run at more or less constant speed. #include "zm_camera.h" #include "zm_db.h" #include "zm_define.h" +#include "zm_fifo.h" #include "zm_monitor.h" #include "zm_rtsp_server_thread.h" #include "zm_signal.h" @@ -269,16 +270,6 @@ int main(int argc, char *argv[]) { } // end foreach monitor if (zm_terminate) break; -#if HAVE_RTSP_SERVER - RTSPServerThread ** rtsp_server_threads = nullptr; - if (config.min_rtsp_port and monitors[0]->RTSPServer()) { - rtsp_server_threads = new RTSPServerThread *[monitors.size()]; - Debug(1, "Starting RTSP server because min_rtsp_port is set"); - } else { - Debug(1, "Not starting RTSP server because min_rtsp_port not set"); - } -#endif - std::vector> analysis_threads = std::vector>(); int *capture_delays = new int[monitors.size()]; @@ -297,13 +288,6 @@ int main(int argc, char *argv[]) { Debug(1, "Starting an analysis thread for monitor (%d)", monitors[i]->Id()); analysis_threads.emplace_back(ZM::make_unique(monitors[i])); } -#if HAVE_RTSP_SERVER - if (rtsp_server_threads) { - rtsp_server_threads[i] = new RTSPServerThread(monitors[i]); - rtsp_server_threads[i]->addStream(monitors[i]->GetVideoStream(), monitors[i]->GetAudioStream()); - rtsp_server_threads[i]->start(); - } -#endif } struct timeval now; @@ -369,32 +353,12 @@ int main(int argc, char *argv[]) { analysis_thread->Stop(); for (size_t i = 0; i < monitors.size(); i++) { -#if HAVE_RTSP_SERVER - if (rtsp_server_threads) { - rtsp_server_threads[i]->stop(); - } -#endif - monitors[i]->Close(); - -#if HAVE_RTSP_SERVER - if (rtsp_server_threads) { - rtsp_server_threads[i]->join(); - delete rtsp_server_threads[i]; - rtsp_server_threads[i] = nullptr; - } -#endif } // Killoff the analysis threads. Don't need them spinning while we try to reconnect analysis_threads.clear(); -#if HAVE_RTSP_SERVER - if (rtsp_server_threads) { - delete[] rtsp_server_threads; - rtsp_server_threads = nullptr; - } -#endif delete [] alarm_capture_delays; delete [] capture_delays; delete [] last_capture_times; From bb747bfc6d66519439a2f347094ba1a65f64852a Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sat, 27 Feb 2021 12:28:59 -0500 Subject: [PATCH 0018/1277] Add RTSPStreamName fields --- web/api/app/Plugin/Crud | 2 +- web/includes/Monitor.php | 1 + web/skins/classic/views/monitor.php | 4 ++++ 3 files changed, 6 insertions(+), 1 deletion(-) diff --git a/web/api/app/Plugin/Crud b/web/api/app/Plugin/Crud index 14292374c..0bd63fb46 160000 --- a/web/api/app/Plugin/Crud +++ b/web/api/app/Plugin/Crud @@ -1 +1 @@ -Subproject commit 14292374ccf1328f2d5db20897bd06f99ba4d938 +Subproject commit 0bd63fb464957080ead342db58ca9e01532cf1ef diff --git a/web/includes/Monitor.php b/web/includes/Monitor.php index 10e7c8f37..ad118b73f 100644 --- a/web/includes/Monitor.php +++ b/web/includes/Monitor.php @@ -122,6 +122,7 @@ class Monitor extends ZM_Object { 'Latitude' => null, 'Longitude' => null, 'RTSPServer' => array('type'=>'boolean', 'default'=>0), + 'RTSPStreamName' => '', ); private $status_fields = array( 'Status' => null, diff --git a/web/skins/classic/views/monitor.php b/web/skins/classic/views/monitor.php index 62801c6de..d63a57b21 100644 --- a/web/skins/classic/views/monitor.php +++ b/web/skins/classic/views/monitor.php @@ -1241,6 +1241,10 @@ echo htmlSelect('newMonitor[ReturnLocation]', $return_options, $monitor->ReturnL RTSPServer() ? ' checked="checked"' : '' ?>/> + + + + Date: Sat, 27 Feb 2021 12:39:24 -0500 Subject: [PATCH 0019/1277] Fix cmake to only build zm_rtsp_server if it is turned on --- src/CMakeLists.txt | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index 0bc9f8608..83a4a888e 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -50,7 +50,6 @@ set(ZM_BIN_SRC_FILES zm_rtp_source.cpp zm_rtsp.cpp zm_rtsp_auth.cpp - zm_rtsp_server.cpp zm_rtsp_server_thread.cpp zm_rtsp_server_adts_source.cpp zm_rtsp_server_adts_fifo_source.cpp @@ -90,7 +89,6 @@ target_link_libraries(zm zm-core-interface) add_executable(zmc zmc.cpp) -add_executable(zm_rtsp_server zm_rtsp_server.cpp) add_executable(zms zms.cpp) add_executable(zmu zmu.cpp) @@ -134,8 +132,20 @@ if(BUILD_MAN) endforeach(CBINARY zmc zmu) endif() -install(TARGETS zmc zm_rtsp_server zmu RUNTIME DESTINATION "${CMAKE_INSTALL_FULL_BINDIR}" PERMISSIONS OWNER_WRITE OWNER_READ OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE) +install(TARGETS zmc zmu RUNTIME DESTINATION "${CMAKE_INSTALL_FULL_BINDIR}" PERMISSIONS OWNER_WRITE OWNER_READ OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE) install(TARGETS zms RUNTIME DESTINATION "${ZM_CGIDIR}" PERMISSIONS OWNER_WRITE OWNER_READ OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE) install(CODE "execute_process(COMMAND ln -sf zms nph-zms WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})") install(FILES ${CMAKE_CURRENT_BINARY_DIR}/nph-zms DESTINATION "${ZM_CGIDIR}") +if(HAVE_RTSP_SERVER) + add_executable(zm_rtsp_server zm_rtsp_server.cpp) + target_link_libraries(zm_rtsp_server + PRIVATE + zm-core-interface + zm + ${ZM_EXTRA_LIBS} + ${ZM_BIN_LIBS} + ${CMAKE_DL_LIBS} + bcrypt) + install(TARGETS zm_rtsp_server RUNTIME DESTINATION "${CMAKE_INSTALL_FULL_BINDIR}" PERMISSIONS OWNER_WRITE OWNER_READ OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE) +endif() From 836ef731c1ffb56af5c6c9bd0a5a692f8d96f3eb Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sat, 27 Feb 2021 12:58:45 -0500 Subject: [PATCH 0020/1277] Don't use libjquery-js --- distros/ubuntu1604/zoneminder.linktrees | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/distros/ubuntu1604/zoneminder.linktrees b/distros/ubuntu1604/zoneminder.linktrees index 8a9ca2723..61edb4173 100644 --- a/distros/ubuntu1604/zoneminder.linktrees +++ b/distros/ubuntu1604/zoneminder.linktrees @@ -2,5 +2,5 @@ #replace /usr/share/php/Cake /usr/share/zoneminder/www/api/lib/Cake ## libjs-jquery -replace /usr/share/javascript/jquery/jquery.min.js /usr/share/zoneminder/www/skins/classic/js/jquery-3.5.1.min.js -replace /usr/share/javascript/jquery/jquery.min.js /usr/share/zoneminder/www/skins/flat/js/jquery-3.5.1.min.js +#replace /usr/share/javascript/jquery/jquery.min.js /usr/share/zoneminder/www/skins/classic/js/jquery-3.5.1.min.js +#replace /usr/share/javascript/jquery/jquery.min.js /usr/share/zoneminder/www/skins/flat/js/jquery-3.5.1.min.js From b70cd0bc765ec8bfd53e4dd7ae32c88d9649bd2c Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 13:33:58 -0500 Subject: [PATCH 0021/1277] Add zm_rtsp_server to list of daemons we can start --- scripts/zmdc.pl.in | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/zmdc.pl.in b/scripts/zmdc.pl.in index c5eec566d..5cf866e56 100644 --- a/scripts/zmdc.pl.in +++ b/scripts/zmdc.pl.in @@ -101,11 +101,12 @@ my @daemons = ( 'zmstats.pl', 'zmtrack.pl', 'zmcontrol.pl', + 'zm_rtsp_server', 'zmtelemetry.pl' ); if ( $Config{ZM_OPT_USE_EVENTNOTIFICATION} ) { - push @daemons,'zmeventnotification.pl'; + push @daemons, 'zmeventnotification.pl'; } my $command = shift @ARGV; From 2a4723090b5331e7da344e4051fb00efb175860c Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 13:34:26 -0500 Subject: [PATCH 0022/1277] Start zm_rtsp_server if ZM_MIN_RTSP_PORT is set --- scripts/zmpkg.pl.in | 3 +++ 1 file changed, 3 insertions(+) diff --git a/scripts/zmpkg.pl.in b/scripts/zmpkg.pl.in index b1647a26f..bca93c1b3 100644 --- a/scripts/zmpkg.pl.in +++ b/scripts/zmpkg.pl.in @@ -295,6 +295,9 @@ if ( $command =~ /^(?:start|restart)$/ ) { } else { runCommand('zmdc.pl start zmstats.pl'); } + if ( $Config{ZM_MIN_RTSP_PORT} ) { + runCommand('zmdc.pl start zm_rtsp_server'); + } } else { $retval = 1; } From 3ad76f18ec92ee6bb2f21e9b10fc3fe41401b346 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 13:48:24 -0500 Subject: [PATCH 0023/1277] code style updates --- src/zm_buffer.h | 84 +++++++++++++++++++++++++++---------------------- 1 file changed, 47 insertions(+), 37 deletions(-) diff --git a/src/zm_buffer.h b/src/zm_buffer.h index 1f9a0183a..d4df93a83 100644 --- a/src/zm_buffer.h +++ b/src/zm_buffer.h @@ -23,28 +23,35 @@ #include "zm_logger.h" #include -class Buffer -{ -protected: +class Buffer { + protected: unsigned char *mStorage; unsigned int mAllocation; unsigned int mSize; unsigned char *mHead; unsigned char *mTail; -public: - Buffer() : mStorage(nullptr), mAllocation(0), mSize(0), mHead(nullptr), mTail(nullptr) { + public: + Buffer() : + mStorage(nullptr), + mAllocation(0), + mSize(0), + mHead(nullptr), + mTail(nullptr) { } explicit Buffer(unsigned int pSize) : mAllocation(pSize), mSize(0) { mHead = mStorage = new unsigned char[mAllocation]; mTail = mHead; } - Buffer(const unsigned char *pStorage, unsigned int pSize) : mAllocation(pSize), mSize(pSize) { + Buffer(const unsigned char *pStorage, unsigned int pSize) : + mAllocation(pSize), mSize(pSize) { mHead = mStorage = new unsigned char[mSize]; std::memcpy(mStorage, pStorage, mSize); mTail = mHead + mSize; } - Buffer(const Buffer &buffer) : mAllocation(buffer.mSize), mSize(buffer.mSize) { + Buffer(const Buffer &buffer) : + mAllocation(buffer.mSize), + mSize(buffer.mSize) { mHead = mStorage = new unsigned char[mSize]; std::memcpy(mStorage, buffer.mHead, mSize); mTail = mHead + mSize; @@ -62,7 +69,7 @@ public: } return mSize; } - //unsigned int Allocation() const { return( mAllocation ); } + // unsigned int Allocation() const { return( mAllocation ); } void clear() { mSize = 0; @@ -77,7 +84,8 @@ public: // Trim from the front of the buffer unsigned int consume(unsigned int count) { if ( count > mSize ) { - Warning("Attempt to consume %d bytes of buffer, size is only %d bytes", count, mSize); + Warning("Attempt to consume %d bytes of buffer, size is only %d bytes", + count, mSize); count = mSize; } mHead += count; @@ -87,23 +95,25 @@ public: } // Trim from the end of the buffer unsigned int shrink(unsigned int count) { - if ( count > mSize ) { - Warning("Attempt to shrink buffer by %d bytes, size is only %d bytes", count, mSize); + if (count > mSize) { + Warning("Attempt to shrink buffer by %d bytes, size is only %d bytes", + count, mSize); count = mSize; } mSize -= count; - if ( mTail > (mHead + mSize) ) + if (mTail > (mHead + mSize)) mTail = mHead + mSize; tidy(0); return count; } // Add to the end of the buffer - unsigned int expand( unsigned int count ); + unsigned int expand(unsigned int count); // Return pointer to the first pSize bytes and advance the head unsigned char *extract(unsigned int pSize) { - if ( pSize > mSize ) { - Warning("Attempt to extract %d bytes of buffer, size is only %d bytes", pSize, mSize); + if (pSize > mSize) { + Warning("Attempt to extract %d bytes of buffer, size is only %d bytes", + pSize, mSize); pSize = mSize; } unsigned char *oldHead = mHead; @@ -126,13 +136,13 @@ public: unsigned int append(const Buffer &buffer) { return append(buffer.mHead, buffer.mSize); } - void tidy( bool level=0 ) { - if ( mHead != mStorage ) { - if ( mSize == 0 ) + void tidy(bool level=0) { + if (mHead != mStorage) { + if (mSize == 0) { mHead = mTail = mStorage; - else if ( level ) { - if ( ((uintptr_t)mHead-(uintptr_t)mStorage) > mSize ) { - std::memcpy( mStorage, mHead, mSize ); + } else if (level) { + if (((uintptr_t)mHead-(uintptr_t)mStorage) > mSize) { + std::memcpy(mStorage, mHead, mSize); mHead = mStorage; mTail = mHead + mSize; } @@ -140,38 +150,38 @@ public: } } - Buffer &operator=( const Buffer &buffer ) { - assign( buffer ); - return( *this ); + Buffer &operator=(const Buffer &buffer) { + assign(buffer); + return *this; } - Buffer &operator+=( const Buffer &buffer ) { - append( buffer ); - return( *this ); + Buffer &operator+=(const Buffer &buffer) { + append(buffer); + return *this; } - Buffer &operator+=( unsigned int count ) { - expand( count ); - return( *this ); + Buffer &operator+=(unsigned int count) { + expand(count); + return *this; } - Buffer &operator-=( unsigned int count ) { + Buffer &operator-=(unsigned int count) { consume(count); return *this; } operator unsigned char *() const { - return( mHead ); + return mHead; } operator char *() const { - return( (char *)mHead ); + return reinterpret_cast(mHead); } unsigned char *operator+(int offset) const { - return( (unsigned char *)(mHead+offset) ); + return (unsigned char *)(mHead+offset); } unsigned char operator[](int index) const { - return( *(mHead+index) ); + return *(mHead+index); } operator int () const { - return( (int)mSize ); + return static_cast(mSize); } - int read_into( int sd, unsigned int bytes ); + int read_into(int sd, unsigned int bytes); }; #endif // ZM_BUFFER_H From 1d73087e5b8696098d4959eaad17b6cc95f219a6 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 13:49:17 -0500 Subject: [PATCH 0024/1277] Split zm_fifo up into zm_fifo, zm_fifo_debug and zm_fifo_stream. Implement a Fifo class to keep the filehandle open. --- src/CMakeLists.txt | 2 + src/zm_fifo.cpp | 277 ++++++++++------------------------------- src/zm_fifo.h | 80 ++++++------ src/zm_fifo_debug.cpp | 98 +++++++++++++++ src/zm_fifo_debug.h | 42 +++++++ src/zm_fifo_stream.cpp | 169 +++++++++++++++++++++++++ src/zm_fifo_stream.h | 57 +++++++++ 7 files changed, 471 insertions(+), 254 deletions(-) create mode 100644 src/zm_fifo_debug.cpp create mode 100644 src/zm_fifo_debug.h create mode 100644 src/zm_fifo_stream.cpp create mode 100644 src/zm_fifo_stream.h diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index 83a4a888e..f2e899afb 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -21,6 +21,8 @@ set(ZM_BIN_SRC_FILES zm_eventstream.cpp zm_exception.cpp zm_fifo.cpp + zm_fifo_debug.cpp + zm_fifo_stream.cpp zm_file_camera.cpp zm_font.cpp zm_frame.cpp diff --git a/src/zm_fifo.cpp b/src/zm_fifo.cpp index 8cbe020ed..e1cb57625 100644 --- a/src/zm_fifo.cpp +++ b/src/zm_fifo.cpp @@ -26,106 +26,8 @@ #include #define RAW_BUFFER 512 -static bool zm_fifodbg_inited = false; -FILE *zm_fifodbg_log_fd = nullptr; -char zm_fifodbg_log[PATH_MAX] = ""; -static bool zmFifoDbgOpen() { - if ( zm_fifodbg_log_fd ) - fclose(zm_fifodbg_log_fd); - zm_fifodbg_log_fd = nullptr; - signal(SIGPIPE, SIG_IGN); - FifoStream::fifo_create_if_missing(zm_fifodbg_log); - int fd = open(zm_fifodbg_log, O_WRONLY|O_NONBLOCK|O_TRUNC); - if ( fd < 0 ) - return false; - int res = flock(fd, LOCK_EX | LOCK_NB); - if ( res < 0 ) { - close(fd); - return false; - } - zm_fifodbg_log_fd = fdopen(fd, "wb"); - if ( zm_fifodbg_log_fd == nullptr ) { - close(fd); - return false; - } - return true; -} - -int zmFifoDbgInit(Monitor *monitor) { - zm_fifodbg_inited = true; - snprintf(zm_fifodbg_log, sizeof(zm_fifodbg_log), "%s/dbgpipe-%d.log", - staticConfig.PATH_SOCKS.c_str(), monitor->Id()); - zmFifoDbgOpen(); - return 1; -} - -void zmFifoDbgOutput( - int hex, - const char * const file, - const int line, - const int level, - const char *fstring, - ... - ) { - char dbg_string[8192]; - int str_size = sizeof(dbg_string); - - va_list arg_ptr; - if ( (!zm_fifodbg_inited) || ( !zm_fifodbg_log_fd && !zmFifoDbgOpen() ) ) - return; - - char *dbg_ptr = dbg_string; - va_start(arg_ptr, fstring); - if ( hex ) { - unsigned char *data = va_arg(arg_ptr, unsigned char *); - int len = va_arg(arg_ptr, int); - dbg_ptr += snprintf(dbg_ptr, str_size-(dbg_ptr-dbg_string), "%d:", len); - for ( int i = 0; i < len; i++ ) { - dbg_ptr += snprintf(dbg_ptr, str_size-(dbg_ptr-dbg_string), " %02x", data[i]); - } - } else { - dbg_ptr += vsnprintf(dbg_ptr, str_size-(dbg_ptr-dbg_string), fstring, arg_ptr); - } - va_end(arg_ptr); - strncpy(dbg_ptr++, "\n", 2); - int res = fwrite(dbg_string, dbg_ptr-dbg_string, 1, zm_fifodbg_log_fd); - if ( res != 1 ) { - fclose(zm_fifodbg_log_fd); - zm_fifodbg_log_fd = nullptr; - } else { - fflush(zm_fifodbg_log_fd); - } -} - -bool FifoStream::sendRAWFrames() { - static unsigned char buffer[RAW_BUFFER]; - int fd = open(stream_path, O_RDONLY); - if ( fd < 0 ) { - Error("Can't open %s: %s", stream_path, strerror(errno)); - return false; - } - while ( (bytes_read = read(fd, buffer, RAW_BUFFER)) ) { - if ( bytes_read == 0 ) - continue; - if ( bytes_read < 0 ) { - Error("Problem during reading: %s", strerror(errno)); - close(fd); - return false; - } - if ( fwrite(buffer, bytes_read, 1, stdout) != 1 ) { - if ( !zm_terminate ) - Error("Problem during writing: %s", strerror(errno)); - close(fd); - return false; - } - fflush(stdout); - } - close(fd); - return true; -} - -void FifoStream::file_create_if_missing( +void Fifo::file_create_if_missing( const char * path, bool is_fifo, bool delete_fake_fifo @@ -138,139 +40,77 @@ void FifoStream::file_create_if_missing( unlink(path); } int fd; - if ( !is_fifo ) { + if (!is_fifo) { Debug(5, "Creating non fifo file as requested: %s", path); - fd = open(path, O_CREAT|O_WRONLY, S_IRUSR|S_IWUSR); - close(fd); + fd = ::open(path, O_CREAT|O_WRONLY, S_IRUSR|S_IWUSR); + ::close(fd); return; } Debug(5, "Making fifo file of: %s", path); mkfifo(path, S_IRUSR|S_IWUSR); } -void FifoStream::fifo_create_if_missing( +void Fifo::fifo_create_if_missing( const char * path, bool delete_fake_fifo ) { file_create_if_missing(path, true, delete_fake_fifo); } -bool FifoStream::sendMJEGFrames() { - static unsigned char buffer[ZM_MAX_IMAGE_SIZE]; - int fd = open(stream_path, O_RDONLY); - if ( fd < 0 ) { - Error("Can't open %s: %s", stream_path, strerror(errno)); - return false; - } - total_read = 0; - while ( - (bytes_read = read(fd, buffer+total_read, ZM_MAX_IMAGE_SIZE-total_read)) - ) { - if ( bytes_read < 0 ) { - Error("Problem during reading: %s", strerror(errno)); - close(fd); +Fifo::~Fifo() { + close(); +} +bool Fifo::open() { + if (!on_blocking_abort) { + if ( (outfile = fopen(path.c_str(), "wb")) == nullptr ) { + Error("Can't open %s for writing: %s", path.c_str(), strerror(errno)); + return false; + } + } else { + raw_fd = ::open(path.c_str(), O_WRONLY|O_NONBLOCK|O_CREAT|O_TRUNC,S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH); + if (raw_fd < 0) + return false; + outfile = fdopen(raw_fd, "wb"); + if (outfile == nullptr) { + ::close(raw_fd); + raw_fd = -1; return false; } - total_read += bytes_read; } - close(fd); + int ret = fcntl(raw_fd, F_SETPIPE_SZ, 1024 * 1024); + if (ret < 0) { + Error("set pipe size failed."); + } + long pipe_size = (long)fcntl(raw_fd, F_GETPIPE_SZ); + if (pipe_size == -1) { + perror("get pipe size failed."); + } + Debug(1, "default pipe size: %ld\n", pipe_size); + return true; +} - if ( (total_read == 0) || (frame_count%frame_mod != 0) ) - return true; +bool Fifo::close() { + fclose(outfile); + return true; +} - if ( fprintf(stdout, - "--" BOUNDARY "\r\n" - "Content-Type: image/jpeg\r\n" - "Content-Length: %d\r\n\r\n", - total_read) < 0 ) { +bool Fifo::writePacket(ZMPacket &packet) { + if (!(outfile or open())) return false; + + Debug(1, "Writing header ZM %u %" PRId64, packet.packet.size, packet.pts); + // Going to write a brief header + if ( fprintf(outfile, "ZM %u %" PRId64 "\n", packet.packet.size, packet.pts) < 0 ) { Error("Problem during writing: %s", strerror(errno)); return false; } - if ( fwrite(buffer, total_read, 1, stdout) != 1 ) { - Error("Problem during reading: %s", strerror(errno)); + if (fwrite(packet.packet.data, packet.packet.size, 1, outfile) != 1) { + Debug(1, "Unable to write to '%s': %s", path.c_str(), strerror(errno)); return false; } - fprintf(stdout, "\r\n\r\n"); - fflush(stdout); - last_frame_sent = TV_2_FLOAT(now); - frame_count++; return true; } - -void FifoStream::setStreamStart(const char * path) { - stream_path = strdup(path); -} - -void FifoStream::setStreamStart(int monitor_id, const char * format) { - char diag_path[PATH_MAX]; - std::shared_ptr monitor = Monitor::Load(monitor_id, false, Monitor::QUERY); - - if ( !strcmp(format, "reference") ) { - snprintf(diag_path, sizeof(diag_path), "%s/diagpipe-r-%d.jpg", - staticConfig.PATH_SOCKS.c_str(), monitor->Id()); - stream_type = MJPEG; - } else if ( !strcmp(format, "delta") ) { - snprintf(diag_path, sizeof(diag_path), "%s/diagpipe-d-%d.jpg", - staticConfig.PATH_SOCKS.c_str(), monitor->Id()); - stream_type = MJPEG; - } else { - if ( strcmp(format, "raw") ) { - Warning("Unknown or unspecified format. Defaulting to raw"); - } - snprintf(diag_path, sizeof(diag_path), "%s/dbgpipe-%d.log", - staticConfig.PATH_SOCKS.c_str(), monitor->Id()); - stream_type = RAW; - } - - setStreamStart(diag_path); -} - -void FifoStream::runStream() { - if ( stream_type == MJPEG ) { - fprintf(stdout, "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n\r\n"); - } else { - fprintf(stdout, "Content-Type: text/html\r\n\r\n"); - } - - /* only 1 person can read from a fifo at a time, so use a lock */ - char lock_file[PATH_MAX]; - snprintf(lock_file, sizeof(lock_file), "%s.rlock", stream_path); - file_create_if_missing(lock_file, false); - Debug(1, "Locking %s", lock_file); - - int fd_lock = open(lock_file, O_RDONLY); - if ( fd_lock < 0 ) { - Error("Can't open %s: %s", lock_file, strerror(errno)); - return; - } - int res = flock(fd_lock, LOCK_EX | LOCK_NB); - while ( (res < 0 and errno == EAGAIN) and (! zm_terminate) ) { - Warning("Flocking problem on %s: - %s", lock_file, strerror(errno)); - sleep(1); - res = flock(fd_lock, LOCK_EX | LOCK_NB); - } - if ( res < 0 ) { - Error("Flocking problem on %d != %d %s: - %s", EAGAIN, res, lock_file, strerror(errno)); - close(fd_lock); - return; - } - - while ( !zm_terminate ) { - gettimeofday(&now, nullptr); - checkCommandQueue(); - if ( stream_type == MJPEG ) { - if ( !sendMJEGFrames() ) - zm_terminate = true; - } else { - if ( !sendRAWFrames() ) - zm_terminate = true; - } - } - close(fd_lock); -} - -bool FifoStream::writePacket(std::string filename, ZMPacket &packet) { +bool Fifo::writePacket(std::string filename, ZMPacket &packet) { bool on_blocking_abort = true; FILE *outfile = nullptr; int raw_fd = 0; @@ -281,12 +121,12 @@ bool FifoStream::writePacket(std::string filename, ZMPacket &packet) { return false; } } else { - raw_fd = open(filename.c_str(), O_WRONLY|O_NONBLOCK|O_CREAT|O_TRUNC,S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH); + raw_fd = ::open(filename.c_str(), O_WRONLY|O_NONBLOCK|O_CREAT|O_TRUNC,S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH); if (raw_fd < 0) return false; outfile = fdopen(raw_fd, "wb"); if (outfile == nullptr) { - close(raw_fd); + ::close(raw_fd); return false; } } @@ -301,7 +141,22 @@ bool FifoStream::writePacket(std::string filename, ZMPacket &packet) { return true; } -bool FifoStream::write(std::string filename, uint8_t *data, size_t bytes) { +bool Fifo::write(uint8_t *data, size_t bytes, int64_t pts) { + if (!(outfile or open())) return false; + // Going to write a brief header + Debug(1, "Writing header ZM %lu %" PRId64, bytes, pts); + if ( fprintf(outfile, "ZM %lu %" PRId64 "\n", bytes, pts) < 0 ) { + Error("Problem during writing: %s", strerror(errno)); + return false; + } + if (fwrite(data, bytes, 1, outfile) != 1) { + Debug(1, "Unable to write to '%s': %s", path.c_str(), strerror(errno)); + return false; + } + return true; +} + +bool Fifo::write(std::string filename, uint8_t *data, size_t bytes) { bool on_blocking_abort = true; FILE *outfile = nullptr; int raw_fd = 0; @@ -312,12 +167,12 @@ bool FifoStream::write(std::string filename, uint8_t *data, size_t bytes) { return false; } } else { - raw_fd = open(filename.c_str(), O_WRONLY|O_NONBLOCK|O_CREAT|O_TRUNC,S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH); + raw_fd = ::open(filename.c_str(), O_WRONLY|O_NONBLOCK|O_CREAT|O_TRUNC,S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH); if (raw_fd < 0) return false; outfile = fdopen(raw_fd, "wb"); if (outfile == nullptr) { - close(raw_fd); + ::close(raw_fd); return false; } } diff --git a/src/zm_fifo.h b/src/zm_fifo.h index 92fa5139b..70dfdbb51 100644 --- a/src/zm_fifo.h +++ b/src/zm_fifo.h @@ -1,5 +1,5 @@ // -// ZoneMinder Fifo Debug +// ZoneMinder Fifo // Copyright (C) 2019 ZoneMinder LLC // // This program is free software; you can redistribute it and/or @@ -24,58 +24,52 @@ class Monitor; -#define zmFifoDbgPrintf(level, params...) {\ - zmFifoDbgOutput(0, __FILE__, __LINE__, level, ##params);\ - } - -#ifndef ZM_DBG_OFF -#define FifoDebug(level, params...) zmFifoDbgPrintf(level, ##params) -#else -#define FifoDebug(level, params...) -#endif -void zmFifoDbgOutput( - int hex, - const char * const file, - const int line, - const int level, - const char *fstring, - ...) __attribute__((format(printf, 5, 6))); -int zmFifoDbgInit(Monitor * monitor); - -class FifoStream : public StreamBase { +class Fifo { private: - char * stream_path; - int total_read; - int bytes_read; - unsigned int frame_count; - static void file_create_if_missing( - const char * path, - bool is_fifo, - bool delete_fake_fifo = true - ); - - protected: - typedef enum { UNKNOWN, MJPEG, RAW } StreamType; - StreamType stream_type; - bool sendMJEGFrames(); - bool sendRAWFrames(); - void processCommand(const CmdMsg *msg) override {} + std::string path; + int total_read; + int bytes_read; + bool on_blocking_abort; + FILE *outfile; + int raw_fd; public: - FifoStream() : - stream_path(nullptr), + static void file_create_if_missing( + const char * path, + bool is_fifo, + bool delete_fake_fifo = true + ); + + Fifo() : total_read(0), bytes_read(0), - frame_count(0), - stream_type(UNKNOWN) + on_blocking_abort(true), + outfile(nullptr), + raw_fd(-1) {} + Fifo(const char *p_path, bool p_on_blocking_abort) : + path(p_path), + total_read(0), + bytes_read(0), + on_blocking_abort(p_on_blocking_abort), + outfile(nullptr), + raw_fd(-1) + {} + ~Fifo(); + static void fifo_create_if_missing( const char * path, bool delete_fake_fifo = true); - void setStreamStart(const char * path); - void setStreamStart(int monitor_id, const char * format); - void runStream() override; + + + static bool writePacket(std::string filename, ZMPacket &packet); static bool write(std::string filename, uint8_t *data, size_t size); + + bool open(); + bool close(); + + bool writePacket(ZMPacket &packet); + bool write(uint8_t *data, size_t size, int64_t pts); }; #endif // ZM_FIFO_H diff --git a/src/zm_fifo_debug.cpp b/src/zm_fifo_debug.cpp new file mode 100644 index 000000000..9903fda2b --- /dev/null +++ b/src/zm_fifo_debug.cpp @@ -0,0 +1,98 @@ +// +// ZoneMinder Fifo Debug +// Copyright (C) 2019 ZoneMinder LLC +// +// This program is free software; you can redistribute it and/or +// modify it under the terms of the GNU General Public License +// as published by the Free Software Foundation; either version 2 +// of the License, or (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. +// + +#include "zm_fifo_debug.h" +#include "zm_fifo.h" +#include "zm_monitor.h" +#include "zm_signal.h" +#include +#include + +#define RAW_BUFFER 512 +static bool zm_fifodbg_inited = false; +FILE *zm_fifodbg_log_fd = nullptr; +char zm_fifodbg_log[PATH_MAX] = ""; + +static bool zmFifoDbgOpen() { + if ( zm_fifodbg_log_fd ) + fclose(zm_fifodbg_log_fd); + zm_fifodbg_log_fd = nullptr; + signal(SIGPIPE, SIG_IGN); + Fifo::fifo_create_if_missing(zm_fifodbg_log); + int fd = open(zm_fifodbg_log, O_WRONLY|O_NONBLOCK|O_TRUNC); + if ( fd < 0 ) + return false; + int res = flock(fd, LOCK_EX | LOCK_NB); + if ( res < 0 ) { + close(fd); + return false; + } + zm_fifodbg_log_fd = fdopen(fd, "wb"); + if ( zm_fifodbg_log_fd == nullptr ) { + close(fd); + return false; + } + return true; +} + +int zmFifoDbgInit(Monitor *monitor) { + zm_fifodbg_inited = true; + snprintf(zm_fifodbg_log, sizeof(zm_fifodbg_log), "%s/dbgpipe-%d.log", + staticConfig.PATH_SOCKS.c_str(), monitor->Id()); + zmFifoDbgOpen(); + return 1; +} + +void zmFifoDbgOutput( + int hex, + const char * const file, + const int line, + const int level, + const char *fstring, + ... + ) { + char dbg_string[8192]; + int str_size = sizeof(dbg_string); + + va_list arg_ptr; + if ( (!zm_fifodbg_inited) || ( !zm_fifodbg_log_fd && !zmFifoDbgOpen() ) ) + return; + + char *dbg_ptr = dbg_string; + va_start(arg_ptr, fstring); + if ( hex ) { + unsigned char *data = va_arg(arg_ptr, unsigned char *); + int len = va_arg(arg_ptr, int); + dbg_ptr += snprintf(dbg_ptr, str_size-(dbg_ptr-dbg_string), "%d:", len); + for ( int i = 0; i < len; i++ ) { + dbg_ptr += snprintf(dbg_ptr, str_size-(dbg_ptr-dbg_string), " %02x", data[i]); + } + } else { + dbg_ptr += vsnprintf(dbg_ptr, str_size-(dbg_ptr-dbg_string), fstring, arg_ptr); + } + va_end(arg_ptr); + strncpy(dbg_ptr++, "\n", 2); + int res = fwrite(dbg_string, dbg_ptr-dbg_string, 1, zm_fifodbg_log_fd); + if ( res != 1 ) { + fclose(zm_fifodbg_log_fd); + zm_fifodbg_log_fd = nullptr; + } else { + fflush(zm_fifodbg_log_fd); + } +} diff --git a/src/zm_fifo_debug.h b/src/zm_fifo_debug.h new file mode 100644 index 000000000..08f4c4fb5 --- /dev/null +++ b/src/zm_fifo_debug.h @@ -0,0 +1,42 @@ +// +// ZoneMinder Fifo Debug +// Copyright (C) 2019 ZoneMinder LLC +// +// This program is free software; you can redistribute it and/or +// modify it under the terms of the GNU General Public License +// as published by the Free Software Foundation; either version 2 +// of the License, or (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. +// +#ifndef ZM_FIFO_DEBUG_H +#define ZM_FIFO_DEBUG_H + +class Monitor; + +#define zmFifoDbgPrintf(level, params...) {\ + zmFifoDbgOutput(0, __FILE__, __LINE__, level, ##params);\ + } + +#ifndef ZM_DBG_OFF +#define FifoDebug(level, params...) zmFifoDbgPrintf(level, ##params) +#else +#define FifoDebug(level, params...) +#endif +void zmFifoDbgOutput( + int hex, + const char * const file, + const int line, + const int level, + const char *fstring, + ...) __attribute__((format(printf, 5, 6))); +int zmFifoDbgInit(Monitor * monitor); + +#endif // ZM_FIFO_DEBUG_H diff --git a/src/zm_fifo_stream.cpp b/src/zm_fifo_stream.cpp new file mode 100644 index 000000000..5734b142d --- /dev/null +++ b/src/zm_fifo_stream.cpp @@ -0,0 +1,169 @@ +// +// ZoneMinder Fifo Debug +// Copyright (C) 2019 ZoneMinder LLC +// +// This program is free software; you can redistribute it and/or +// modify it under the terms of the GNU General Public License +// as published by the Free Software Foundation; either version 2 +// of the License, or (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. +// + +#include "zm_fifo_stream.h" +#include "zm_fifo.h" +#include "zm_monitor.h" +#include "zm_signal.h" +#include +#include +#include + +#define RAW_BUFFER 512 +bool FifoStream::sendRAWFrames() { + static unsigned char buffer[RAW_BUFFER]; + int fd = open(stream_path, O_RDONLY); + if ( fd < 0 ) { + Error("Can't open %s: %s", stream_path, strerror(errno)); + return false; + } + while ( (bytes_read = read(fd, buffer, RAW_BUFFER)) ) { + if ( bytes_read == 0 ) + continue; + if ( bytes_read < 0 ) { + Error("Problem during reading: %s", strerror(errno)); + close(fd); + return false; + } + if ( fwrite(buffer, bytes_read, 1, stdout) != 1 ) { + if ( !zm_terminate ) + Error("Problem during writing: %s", strerror(errno)); + close(fd); + return false; + } + fflush(stdout); + } + close(fd); + return true; +} + +bool FifoStream::sendMJEGFrames() { + static unsigned char buffer[ZM_MAX_IMAGE_SIZE]; + int fd = open(stream_path, O_RDONLY); + if ( fd < 0 ) { + Error("Can't open %s: %s", stream_path, strerror(errno)); + return false; + } + total_read = 0; + while ( + (bytes_read = read(fd, buffer+total_read, ZM_MAX_IMAGE_SIZE-total_read)) + ) { + if ( bytes_read < 0 ) { + Error("Problem during reading: %s", strerror(errno)); + close(fd); + return false; + } + total_read += bytes_read; + } + close(fd); + + if ( (total_read == 0) || (frame_count%frame_mod != 0) ) + return true; + + if ( fprintf(stdout, + "--" BOUNDARY "\r\n" + "Content-Type: image/jpeg\r\n" + "Content-Length: %d\r\n\r\n", + total_read) < 0 ) { + Error("Problem during writing: %s", strerror(errno)); + return false; + } + + if ( fwrite(buffer, total_read, 1, stdout) != 1 ) { + Error("Problem during reading: %s", strerror(errno)); + return false; + } + fprintf(stdout, "\r\n\r\n"); + fflush(stdout); + last_frame_sent = TV_2_FLOAT(now); + frame_count++; + return true; +} + +void FifoStream::setStreamStart(const char * path) { + stream_path = strdup(path); +} + +void FifoStream::setStreamStart(int monitor_id, const char * format) { + char diag_path[PATH_MAX]; + std::shared_ptr monitor = Monitor::Load(monitor_id, false, Monitor::QUERY); + + if ( !strcmp(format, "reference") ) { + snprintf(diag_path, sizeof(diag_path), "%s/diagpipe-r-%d.jpg", + staticConfig.PATH_SOCKS.c_str(), monitor->Id()); + stream_type = MJPEG; + } else if ( !strcmp(format, "delta") ) { + snprintf(diag_path, sizeof(diag_path), "%s/diagpipe-d-%d.jpg", + staticConfig.PATH_SOCKS.c_str(), monitor->Id()); + stream_type = MJPEG; + } else { + if ( strcmp(format, "raw") ) { + Warning("Unknown or unspecified format. Defaulting to raw"); + } + snprintf(diag_path, sizeof(diag_path), "%s/dbgpipe-%d.log", + staticConfig.PATH_SOCKS.c_str(), monitor->Id()); + stream_type = RAW; + } + + setStreamStart(diag_path); +} + +void FifoStream::runStream() { + if ( stream_type == MJPEG ) { + fprintf(stdout, "Content-Type: multipart/x-mixed-replace;boundary=" BOUNDARY "\r\n\r\n"); + } else { + fprintf(stdout, "Content-Type: text/html\r\n\r\n"); + } + + /* only 1 person can read from a fifo at a time, so use a lock */ + char lock_file[PATH_MAX]; + snprintf(lock_file, sizeof(lock_file), "%s.rlock", stream_path); + Fifo::file_create_if_missing(lock_file, false); + Debug(1, "Locking %s", lock_file); + + int fd_lock = open(lock_file, O_RDONLY); + if ( fd_lock < 0 ) { + Error("Can't open %s: %s", lock_file, strerror(errno)); + return; + } + int res = flock(fd_lock, LOCK_EX | LOCK_NB); + while ( (res < 0 and errno == EAGAIN) and (! zm_terminate) ) { + Warning("Flocking problem on %s: - %s", lock_file, strerror(errno)); + sleep(1); + res = flock(fd_lock, LOCK_EX | LOCK_NB); + } + if ( res < 0 ) { + Error("Flocking problem on %d != %d %s: - %s", EAGAIN, res, lock_file, strerror(errno)); + close(fd_lock); + return; + } + + while ( !zm_terminate ) { + gettimeofday(&now, nullptr); + checkCommandQueue(); + if ( stream_type == MJPEG ) { + if ( !sendMJEGFrames() ) + zm_terminate = true; + } else { + if ( !sendRAWFrames() ) + zm_terminate = true; + } + } + close(fd_lock); +} diff --git a/src/zm_fifo_stream.h b/src/zm_fifo_stream.h new file mode 100644 index 000000000..446108d2c --- /dev/null +++ b/src/zm_fifo_stream.h @@ -0,0 +1,57 @@ +// +// ZoneMinder Fifo Stream +// Copyright (C) 2019 ZoneMinder LLC +// +// This program is free software; you can redistribute it and/or +// modify it under the terms of the GNU General Public License +// as published by the Free Software Foundation; either version 2 +// of the License, or (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. +// +#ifndef ZM_FIFO_STREAM_H +#define ZM_FIFO_STREAM_H + +#include "zm_stream.h" + +class Monitor; + +class FifoStream : public StreamBase { + private: + char * stream_path; + int total_read; + int bytes_read; + unsigned int frame_count; + static void file_create_if_missing( + const char * path, + bool is_fifo, + bool delete_fake_fifo = true + ); + + protected: + typedef enum { UNKNOWN, MJPEG, RAW } StreamType; + StreamType stream_type; + bool sendMJEGFrames(); + bool sendRAWFrames(); + void processCommand(const CmdMsg *msg) override {} + + public: + FifoStream() : + stream_path(nullptr), + total_read(0), + bytes_read(0), + frame_count(0), + stream_type(UNKNOWN) + {} + void setStreamStart(const char * path); + void setStreamStart(int monitor_id, const char * format); + void runStream() override; +}; +#endif // ZM_FIFO_STREAM_H From c554a22374bc26062555154779d131baf1456268 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 13:49:38 -0500 Subject: [PATCH 0025/1277] use .head() instead of casting the buffer --- src/zm_curl_camera.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/zm_curl_camera.cpp b/src/zm_curl_camera.cpp index 9baf21572..a061c91b1 100644 --- a/src/zm_curl_camera.cpp +++ b/src/zm_curl_camera.cpp @@ -210,7 +210,7 @@ int cURLCamera::Capture( ZMPacket &zm_packet ) { } /* Find crlf start */ - crlf_start = memcspn(databuffer,"\r\n",databuffer.size()); + crlf_start = memcspn(reinterpret_cast(databuffer.head()),"\r\n",databuffer.size()); if ( crlf_start == databuffer.size() ) { /* Not found, wait for more data */ need_more_data = true; From a81e3d6e195c66bcce1bca8090e02185b1e4f17a Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 13:50:03 -0500 Subject: [PATCH 0026/1277] Set zm_packet.pts scaled to AV_TIME_BASE_Q. --- src/zm_ffmpeg_camera.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/src/zm_ffmpeg_camera.cpp b/src/zm_ffmpeg_camera.cpp index bd67d5c8c..afc4f79fa 100644 --- a/src/zm_ffmpeg_camera.cpp +++ b/src/zm_ffmpeg_camera.cpp @@ -213,6 +213,7 @@ int FfmpegCamera::Capture(ZMPacket &zm_packet) { #endif bytes += packet.size; zm_packet.set_packet(&packet); + zm_packet.pts = av_rescale_q(packet.pts, mFormatContext->streams[packet.stream_index]->time_base, AV_TIME_BASE_Q); zm_av_packet_unref(&packet); return 1; } // FfmpegCamera::Capture From 6e73c08a21e30f50e5cd0e1da0c6a2dcbe27aee4 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 13:51:06 -0500 Subject: [PATCH 0027/1277] add fifo's for video and audio. Write the packet data to them. Update FifoStream->Fifo --- src/zm_monitor.cpp | 95 +++++++++++++++++++++++++++++----------------- src/zm_monitor.h | 11 ++++-- 2 files changed, 67 insertions(+), 39 deletions(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 2a53ce2c5..0e9b8e70b 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -378,7 +378,8 @@ Monitor::Monitor() video_store_data(nullptr), shared_timestamps(nullptr), shared_images(nullptr), - + video_fifo(nullptr), + audio_fifo(nullptr), camera(nullptr), event(nullptr), storage(nullptr), @@ -654,8 +655,8 @@ void Monitor::Load(MYSQL_ROW dbrow, bool load_zones=true, Purpose p = QUERY) { if ( config.record_diag_images_fifo ) { diag_path_ref = stringtf("%s/diagpipe-r-%d.jpg", staticConfig.PATH_SOCKS.c_str(), id); diag_path_delta = stringtf("%s/diagpipe-d-%d.jpg", staticConfig.PATH_SOCKS.c_str(), id); - FifoStream::fifo_create_if_missing(diag_path_ref.c_str()); - FifoStream::fifo_create_if_missing(diag_path_delta.c_str()); + Fifo::fifo_create_if_missing(diag_path_ref.c_str()); + Fifo::fifo_create_if_missing(diag_path_delta.c_str()); } else { diag_path_ref = stringtf("%s/%d/diag-r.jpg", storage->Path(), id); diag_path_delta = stringtf("%s/%d/diag-d.jpg", storage->Path(), id); @@ -900,23 +901,23 @@ bool Monitor::connect() { } struct stat map_stat; - if ( fstat(map_fd, &map_stat) < 0 ) { + if (fstat(map_fd, &map_stat) < 0) { Error("Can't stat memory map file %s: %s, is the zmc process for this monitor running?", mem_file, strerror(errno)); close(map_fd); map_fd = -1; return false; } - if ( map_stat.st_size != mem_size ) { - if ( purpose == CAPTURE ) { + if (map_stat.st_size != mem_size) { + if (purpose == CAPTURE) { // Allocate the size - if ( ftruncate(map_fd, mem_size) < 0 ) { + if (ftruncate(map_fd, mem_size) < 0) { Error("Can't extend memory map file %s to %d bytes: %s", mem_file, mem_size, strerror(errno)); close(map_fd); map_fd = -1; return false; } - } else if ( map_stat.st_size == 0 ) { + } else if (map_stat.st_size == 0) { Error("Got empty memory map file size %ld, is the zmc process for this monitor running?", map_stat.st_size, mem_size); close(map_fd); map_fd = -1; @@ -927,13 +928,13 @@ bool Monitor::connect() { map_fd = -1; return false; } - } + } // end if map_stat.st_size != mem_size Debug(3, "MMap file size is %ld", map_stat.st_size); #ifdef MAP_LOCKED mem_ptr = (unsigned char *)mmap(nullptr, mem_size, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_LOCKED, map_fd, 0); - if ( mem_ptr == MAP_FAILED ) { - if ( errno == EAGAIN ) { + if (mem_ptr == MAP_FAILED) { + if (errno == EAGAIN) { Debug(1, "Unable to map file %s (%d bytes) to locked memory, trying unlocked", mem_file, mem_size); #endif mem_ptr = (unsigned char *)mmap(nullptr, mem_size, PROT_READ|PROT_WRITE, MAP_SHARED, map_fd, 0); @@ -944,7 +945,7 @@ bool Monitor::connect() { } } #endif - if ( (mem_ptr == MAP_FAILED) or (mem_ptr == nullptr) ) { + if ((mem_ptr == MAP_FAILED) or (mem_ptr == nullptr)) { Error("Can't map file %s (%d bytes) to memory: %s(%d)", mem_file, mem_size, strerror(errno), errno); close(map_fd); map_fd = -1; @@ -953,11 +954,11 @@ bool Monitor::connect() { } #else // ZM_MEM_MAPPED shm_id = shmget((config.shm_key&0xffff0000)|id, mem_size, IPC_CREAT|0700); - if ( shm_id < 0 ) { + if (shm_id < 0) { Fatal("Can't shmget, probably not enough shared memory space free: %s", strerror(errno)); } mem_ptr = (unsigned char *)shmat(shm_id, 0, 0); - if ( mem_ptr < (void *)0 ) { + if (mem_ptr < (void *)0) { Fatal("Can't shmat: %s", strerror(errno)); } #endif // ZM_MEM_MAPPED @@ -968,29 +969,28 @@ bool Monitor::connect() { shared_timestamps = (struct timeval *)((char *)video_store_data + sizeof(VideoStoreData)); shared_images = (unsigned char *)((char *)shared_timestamps + (image_buffer_count*sizeof(struct timeval))); - if ( ((unsigned long)shared_images % 64) != 0 ) { + if (((unsigned long)shared_images % 64) != 0) { /* Align images buffer to nearest 64 byte boundary */ Debug(3, "Aligning shared memory images to the next 64 byte boundary"); shared_images = (uint8_t*)((unsigned long)shared_images + (64 - ((unsigned long)shared_images % 64))); } - if ( !camera ) - LoadCamera(); + if (!camera) LoadCamera(); Debug(3, "Allocating %d image buffers", image_buffer_count); image_buffer = new ZMPacket[image_buffer_count]; - for ( int32_t i = 0; i < image_buffer_count; i++ ) { + for (int32_t i = 0; i < image_buffer_count; i++) { image_buffer[i].image_index = i; image_buffer[i].timestamp = &(shared_timestamps[i]); image_buffer[i].image = new Image(width, height, camera->Colours(), camera->SubpixelOrder(), &(shared_images[i*camera->ImageSize()])); image_buffer[i].image->HoldBuffer(true); /* Don't release the internal buffer or replace it with another */ } - if ( deinterlacing_value == 4 ) { + if (deinterlacing_value == 4) { /* Four field motion adaptive deinterlacing in use */ /* Allocate a buffer for the next image */ - next_buffer.image = new Image( width, height, camera->Colours(), camera->SubpixelOrder()); + next_buffer.image = new Image(width, height, camera->Colours(), camera->SubpixelOrder()); } - if ( purpose == CAPTURE ) { + if (purpose == CAPTURE) { memset(mem_ptr, 0, mem_size); shared_data->size = sizeof(SharedData); shared_data->active = enabled; @@ -1012,8 +1012,8 @@ bool Monitor::connect() { shared_data->format = camera->SubpixelOrder(); shared_data->imagesize = camera->ImageSize(); shared_data->alarm_cause[0] = 0; - shared_data->video_fifo[0] = 0; - shared_data->audio_fifo[0] = 0; + shared_data->video_fifo_path[0] = 0; + shared_data->audio_fifo_path[0] = 0; shared_data->last_frame_score = 0; trigger_data->size = sizeof(TriggerData); trigger_data->trigger_state = TriggerState::TRIGGER_CANCEL; @@ -1021,11 +1021,11 @@ bool Monitor::connect() { trigger_data->trigger_cause[0] = 0; trigger_data->trigger_text[0] = 0; trigger_data->trigger_showtext[0] = 0; - shared_data->valid = true; - video_store_data->recording = {}; + video_store_data->recording = (struct timeval){0}; // Uh, why nothing? Why not nullptr? snprintf(video_store_data->event_file, sizeof(video_store_data->event_file), "nothing"); video_store_data->size = sizeof(VideoStoreData); + shared_data->valid = true; } else if ( !shared_data->valid ) { Error("Shared data not initialised by capture daemon for monitor %s", name); return false; @@ -1131,6 +1131,9 @@ Monitor::~Monitor() { delete[] linked_monitors; linked_monitors = nullptr; } + + if (video_fifo) delete video_fifo; + if (audio_fifo) delete audio_fifo; } // end Monitor::~Monitor() void Monitor::AddZones(int p_n_zones, Zone *p_zones[]) { @@ -2230,20 +2233,24 @@ bool Monitor::Analyse() { } // end if ( trigger_data->trigger_state != TRIGGER_OFF ) if (event) event->AddPacket(snap); +#if 0 if (snap->packet.stream_index == video_stream_id) { - if (shared_data->video_fifo[0]) { + if (video_fifo) { if ( snap->keyframe ) { + // avcodec strips out important nals that describe the stream and + // stick them in extradata. Need to send them along with keyframes AVStream *stream = camera->get_VideoStream(); - FifoStream::write(shared_data->video_fifo, + video_fifo->write( static_cast(stream->codecpar->extradata), stream->codecpar->extradata_size); } - FifoStream::writePacket(shared_data->video_fifo, *snap); + video_fifo->writePacket(*snap); } - } else if (snap->packet.stream_index == video_stream_id) { - if (shared_data->audio_fifo[0]) - FifoStream::writePacket(shared_data->audio_fifo, *snap); + } else if (snap->packet.stream_index == audio_stream_id) { + if (audio_fifo) + audio_fifo->writePacket(*snap); } +#endif if ((videowriter == PASSTHROUGH) and !savejpegs) { // Don't need raw images anymore @@ -2530,6 +2537,24 @@ int Monitor::Capture() { Debug(2, "Have packet stream_index:%d ?= videostream_id:(%d) q.vpktcount(%d) event?(%d) ", packet->packet.stream_index, video_stream_id, packetqueue.packet_count(video_stream_id), ( event ? 1 : 0 ) ); + if (packet->packet.stream_index == video_stream_id) { + if (video_fifo) { + if ( packet->keyframe ) { + // avcodec strips out important nals that describe the stream and + // stick them in extradata. Need to send them along with keyframes + AVStream *stream = camera->get_VideoStream(); + video_fifo->write( + static_cast(stream->codecpar->extradata), + stream->codecpar->extradata_size, + packet->pts); + } + video_fifo->writePacket(*packet); + } + } else if (packet->packet.stream_index == audio_stream_id) { + if (audio_fifo) + audio_fifo->writePacket(*packet); + } + if ( (packet->packet.stream_index != video_stream_id) and ! packet->image ) { // Only queue if we have some video packets in there. Should push this logic into packetqueue if ( record_audio and (packetqueue.packet_count(video_stream_id) or event) ) { @@ -2952,18 +2977,18 @@ int Monitor::PrimeCapture() { if (rtsp_server) { if (video_stream_id >= 0) { AVStream *videoStream = camera->get_VideoStream(); - snprintf(shared_data->video_fifo, sizeof(shared_data->video_fifo)-1, "%s/video_fifo_%d.%s", + snprintf(shared_data->video_fifo_path, sizeof(shared_data->video_fifo_path)-1, "%s/video_fifo_%d.%s", staticConfig.PATH_SOCKS.c_str(), id, avcodec_get_name(videoStream->codecpar->codec_id)); - FifoStream::fifo_create_if_missing(shared_data->video_fifo); + video_fifo = new Fifo(shared_data->video_fifo_path, true); } if (record_audio and (audio_stream_id >= 0)) { AVStream *audioStream = camera->get_AudioStream(); - snprintf(shared_data->audio_fifo, sizeof(shared_data->audio_fifo)-1, "%s/video_fifo_%d.%s", + snprintf(shared_data->audio_fifo_path, sizeof(shared_data->audio_fifo_path)-1, "%s/video_fifo_%d.%s", staticConfig.PATH_SOCKS.c_str(), id, avcodec_get_name(audioStream->codecpar->codec_id)); - FifoStream::fifo_create_if_missing(shared_data->audio_fifo); + audio_fifo = new Fifo(shared_data->audio_fifo_path, true); } } // end if rtsp_server } else { diff --git a/src/zm_monitor.h b/src/zm_monitor.h index 421e14e83..3f16785a0 100644 --- a/src/zm_monitor.h +++ b/src/zm_monitor.h @@ -23,6 +23,7 @@ #include "zm_define.h" #include "zm_camera.h" #include "zm_event.h" +#include "zm_fifo.h" #include "zm_image.h" #include "zm_packet.h" #include "zm_packetqueue.h" @@ -154,8 +155,8 @@ protected: uint8_t control_state[256]; /* +104 */ char alarm_cause[256]; - char video_fifo[64]; - char audio_fifo[64]; + char video_fifo_path[64]; + char audio_fifo_path[64]; } SharedData; @@ -366,6 +367,8 @@ protected: int video_stream_id; // will be filled in PrimeCapture int audio_stream_id; // will be filled in PrimeCapture + Fifo *video_fifo; + Fifo *audio_fifo; std::unique_ptr camera; Event *event; @@ -487,8 +490,8 @@ public: AVStream *GetVideoStream() const { return camera ? camera->get_VideoStream() : nullptr; }; AVCodecContext *GetVideoCodecContext() const { return camera ? camera->get_VideoCodecContext() : nullptr; }; - const std::string GetVideoFifo() const { return shared_data ? shared_data->video_fifo : ""; }; - const std::string GetAudioFifo() const { return shared_data ? shared_data->audio_fifo : ""; }; + const std::string GetVideoFifoPath() const { return shared_data ? shared_data->video_fifo_path : ""; }; + const std::string GetAudioFifoPath() const { return shared_data ? shared_data->audio_fifo_path : ""; }; const std::string GetRTSPStreamName() const { return rtsp_streamname; }; int GetImage(int32_t index=-1, int scale=100); From 05e7d76ccdcaf99d8129018cb3b21da28e7197bc Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 13:51:32 -0500 Subject: [PATCH 0028/1277] Add pts which will be scaled to AV_TIME_BASE_Q --- src/zm_packet.h | 1 + 1 file changed, 1 insertion(+) diff --git a/src/zm_packet.h b/src/zm_packet.h index ea609429f..6b567bc4e 100644 --- a/src/zm_packet.h +++ b/src/zm_packet.h @@ -49,6 +49,7 @@ class ZMPacket { AVMediaType codec_type; int image_index; int codec_imgsize; + int64_t pts; // pts in the packet can be in another time base. This MUST be in AV_TIME_BASE_Q public: AVPacket *av_packet() { return &packet; } From 449b547f2b6c51cdae9dbaff152266ca0bd339ea Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 13:58:24 -0500 Subject: [PATCH 0029/1277] Use buffer.head() instead of casting --- src/zm_remote_camera_http.cpp | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/zm_remote_camera_http.cpp b/src/zm_remote_camera_http.cpp index 1a4da254e..98625aab8 100644 --- a/src/zm_remote_camera_http.cpp +++ b/src/zm_remote_camera_http.cpp @@ -629,7 +629,7 @@ int RemoteCameraHttp::GetResponse() { bytes += buffer_len; char *crlf = nullptr; - char *header_ptr = (char *)buffer; + char *header_ptr = buffer; int header_len = buffer.size(); bool all_headers = false; @@ -1102,14 +1102,14 @@ int RemoteCameraHttp::Capture(ZMPacket &packet) { image->Size(), content_length); return -1; } - image->Assign(width, height, colours, subpixelorder, buffer, imagesize); + image->Assign(width, height, colours, subpixelorder, buffer.head(), imagesize); break; case X_RGBZ : if ( !image->Unzip( buffer.extract( content_length ), content_length ) ) { Error("Unable to unzip RGB image"); return -1; } - image->Assign(width, height, colours, subpixelorder, buffer, imagesize); + image->Assign(width, height, colours, subpixelorder, buffer.head(), imagesize); break; default : Error("Unexpected image format encountered"); From 7ba6e45505de3c1b05ac9b2b5ab0be68f168bea2 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 13:59:09 -0500 Subject: [PATCH 0030/1277] Rough in rtsp_server concept. Wait for fifo's to exist before connecting. Check for monitor disconnection. WIP --- src/zm_rtsp_server.cpp | 106 ++++++++++++++++++++++------------------- 1 file changed, 57 insertions(+), 49 deletions(-) diff --git a/src/zm_rtsp_server.cpp b/src/zm_rtsp_server.cpp index d31ec50fe..0e280e96b 100644 --- a/src/zm_rtsp_server.cpp +++ b/src/zm_rtsp_server.cpp @@ -58,6 +58,7 @@ and provide that stream over rtsp #include "zm_utils.h" #include #include +#include void Usage() { fprintf(stderr, "zm_rtsp_server -m \n"); @@ -155,72 +156,79 @@ int main(int argc, char *argv[]) { sigaddset(&block_set, SIGUSR1); sigaddset(&block_set, SIGUSR2); - int result = 0; + RTSPServerThread * rtsp_server_thread = nullptr; + if (config.min_rtsp_port) { + rtsp_server_thread = new RTSPServerThread(config.min_rtsp_port); + Debug(1, "Starting RTSP server because min_rtsp_port is set"); + } else { + Debug(1, "Not starting RTSP server because min_rtsp_port not set"); + exit(-1); + } + ServerMediaSession **sessions = new ServerMediaSession *[monitors.size()]; + for (size_t i = 0; i < monitors.size(); i++) sessions[i] = nullptr; + + rtsp_server_thread->start(); while (!zm_terminate) { - result = 0; - for (const std::shared_ptr &monitor : monitors) { - monitor->LoadCamera(); + for (size_t i = 0; i < monitors.size(); i++) { + std::shared_ptr monitor = monitors[i]; - while (!monitor->connect()) { + if (!(monitor->ShmValid() or monitor->connect())) { Warning("Couldn't connect to monitor %d", monitor->Id()); - sleep(1); + if (sessions[i]) { + rtsp_server_thread->removeSession(sessions[i]); + sessions[i] = nullptr; + } + continue; } + Debug(1, "monitor %d is connected", monitor->Id()); + + if (!sessions[i]) { + std::string videoFifoPath = monitor->GetVideoFifoPath(); + if (videoFifoPath.empty()) { + Debug(1, "video fifo is empty. Skipping."); + continue; + } + std::string streamname = monitor->GetRTSPStreamName(); + Debug(1, "Adding session for %s", streamname.c_str()); + ServerMediaSession *sms = sessions[i] = rtsp_server_thread->addSession(streamname); + Debug(1, "Adding video fifo %s", videoFifoPath.c_str()); + ZoneMinderFifoVideoSource *video_source = static_cast(rtsp_server_thread->addFifo(sms, videoFifoPath)); + if (video_source) { + video_source->setWidth(monitor->Width()); + video_source->setHeight(monitor->Height()); + } + Debug(1, "Adding audio fifo %s", monitor->GetAudioFifoPath().c_str()); + FramedSource *audio_source = rtsp_server_thread->addFifo(sms, monitor->GetAudioFifoPath()); + if (audio_source) { + // set frequency + } + } // end if ! sessions[i] } // end foreach monitor - - RTSPServerThread ** rtsp_server_threads = nullptr; - if (config.min_rtsp_port) { - rtsp_server_threads = new RTSPServerThread *[monitors.size()]; - Debug(1, "Starting RTSP server because min_rtsp_port is set"); - } else { - Debug(1, "Not starting RTSP server because min_rtsp_port not set"); - exit(-1); - } - - for (size_t i = 0; i < monitors.size(); i++) { - rtsp_server_threads[i] = new RTSPServerThread(monitors[i]); - std::string streamname = monitors[i]->GetRTSPStreamName(); - ServerMediaSession *sms = rtsp_server_threads[i]->addSession(streamname); - ZoneMinderFifoVideoSource *video_source = static_cast(rtsp_server_threads[i]->addFifo(sms, monitors[i]->GetVideoFifo())); - video_source->setWidth(monitors[i]->Width()); - video_source->setHeight(monitors[i]->Height()); - FramedSource *audio_source = rtsp_server_threads[i]->addFifo(sms, monitors[i]->GetAudioFifo()); - rtsp_server_threads[i]->start(); - } - - while (!zm_terminate) { - sleep(1); - // What to do in here? Sleep mostly. Wait for reload commands maybe watch for dead monitors. - if ((result < 0) or zm_reload) { - // Failure, try reconnecting - break; - } - } // end while ! zm_terminate and connected - - for (size_t i = 0; i < monitors.size(); i++) { - rtsp_server_threads[i]->stop(); - rtsp_server_threads[i]->join(); - delete rtsp_server_threads[i]; - rtsp_server_threads[i] = nullptr; - } - - delete[] rtsp_server_threads; - rtsp_server_threads = nullptr; + sleep(1); if (zm_reload) { - for (std::shared_ptr &monitor : monitors) { - monitor->Reload(); + for (size_t i = 0; i < monitors.size(); i++) { + monitors[i]->Reload(); } logTerm(); logInit(log_id_string); zm_reload = false; } // end if zm_reload - } // end while ! zm_terminate outer connection loop + } // end while ! zm_terminate + + rtsp_server_thread->stop(); + rtsp_server_thread->join(); + delete rtsp_server_thread; + rtsp_server_thread = nullptr; + + delete[] sessions; + sessions = nullptr; Image::Deinitialise(); logTerm(); zmDbClose(); - return zm_terminate ? 0 : result; + return 0; } From c716e8c149226bf991a278cbe5d7236fcd84e5fa Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 13:59:36 -0500 Subject: [PATCH 0031/1277] Add logging of audio auxLine. Default to 8000Hz --- src/zm_rtsp_server_adts_fifo_source.cpp | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/zm_rtsp_server_adts_fifo_source.cpp b/src/zm_rtsp_server_adts_fifo_source.cpp index 50677b519..757e927eb 100644 --- a/src/zm_rtsp_server_adts_fifo_source.cpp +++ b/src/zm_rtsp_server_adts_fifo_source.cpp @@ -6,6 +6,7 @@ ** ** -------------------------------------------------------------------------*/ +#include "zm_logger.h" #include "zm_rtsp_server_adts_fifo_source.h" #include @@ -29,7 +30,7 @@ ADTS_ZoneMinderFifoSource::ADTS_ZoneMinderFifoSource( ) : ZoneMinderFifoSource(env, fifo, queueSize), - samplingFrequencyIndex(0), + samplingFrequencyIndex(11), channels(1) { std::ostringstream os; @@ -39,5 +40,6 @@ ADTS_ZoneMinderFifoSource::ADTS_ZoneMinderFifoSource( "indexdeltalength=3" << "\r\n"; m_auxLine.assign(os.str()); + Debug(1, "m_auxline is %s", m_auxLine.c_str()); } #endif // HAVE_RTSP_SERVER From 53851b67c7c6f1a487d67cfba1304bb65c19f017 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 14:00:26 -0500 Subject: [PATCH 0032/1277] use a Buffer to implement the simple ZM header protocol for passing packet data from ZM to RTSP Server --- src/zm_rtsp_server_fifo_source.cpp | 154 +++++++++++++++++++---------- src/zm_rtsp_server_fifo_source.h | 5 +- 2 files changed, 106 insertions(+), 53 deletions(-) diff --git a/src/zm_rtsp_server_fifo_source.cpp b/src/zm_rtsp_server_fifo_source.cpp index 7b25058ab..d7ad62ee4 100644 --- a/src/zm_rtsp_server_fifo_source.cpp +++ b/src/zm_rtsp_server_fifo_source.cpp @@ -34,19 +34,18 @@ ZoneMinderFifoSource::ZoneMinderFifoSource( memset(&m_mutex, 0, sizeof(m_mutex)); pthread_mutex_init(&m_mutex, nullptr); pthread_create(&m_thid, nullptr, threadStub, this); - m_buffer_ptr = &m_buffer[0]; } ZoneMinderFifoSource::~ZoneMinderFifoSource() { + Debug(1, "Deleting Fifo Source"); stop = 1; envir().taskScheduler().deleteEventTrigger(m_eventTriggerId); pthread_join(m_thid, nullptr); - while ( m_captureQueue.size() ) { + while (m_captureQueue.size()) { NAL_Frame * f = m_captureQueue.front(); m_captureQueue.pop_front(); delete f; } - pthread_mutex_destroy(&m_mutex); } @@ -75,13 +74,13 @@ void ZoneMinderFifoSource::doStopGettingFrames() { // deliver frame to the sink void ZoneMinderFifoSource::deliverFrame() { if (!isCurrentlyAwaitingData()) { - Debug(4, "not awaiting data"); + Debug(5, "not awaiting data"); return; } pthread_mutex_lock(&m_mutex); if (m_captureQueue.empty()) { - Debug(4, "Queue is empty"); + Debug(5, "Queue is empty"); pthread_mutex_unlock(&m_mutex); return; } @@ -101,7 +100,8 @@ void ZoneMinderFifoSource::deliverFrame() { } else { fFrameSize = nal_size; } - Debug(2, "deliverFrame timestamp: %ld.%06ld size: %d queuesize: %d", + + Debug(4, "deliverFrame timestamp: %ld.%06ld size: %d queuesize: %d", frame->m_timestamp.tv_sec, frame->m_timestamp.tv_usec, fFrameSize, m_captureQueue.size() @@ -137,68 +137,122 @@ int ZoneMinderFifoSource::getNextFrame() { } } - int bytes_in_buffer = m_buffer_ptr - &m_buffer[0]; - int bytes_read = read(m_fd, m_buffer_ptr, BUFFER_SIZE-bytes_in_buffer); - if (bytes_read == 0) + int bytes_read = m_buffer.read_into(m_fd, 4096); + if (bytes_read == 0) { + Debug(3, "No bytes read"); + sleep(1); return -1; + } if (bytes_read < 0) { Error("Problem during reading: %s", strerror(errno)); ::close(m_fd); m_fd = -1; return -1; } - bytes_in_buffer += bytes_read; - unsigned int bytes_remaining = bytes_in_buffer; - timeval tv; - gettimeofday(&tv, nullptr); + Debug(4, "%s bytes read %d bytes, buffer size %u", m_fifo.c_str(), bytes_read, m_buffer.size()); + while (m_buffer.size()) { - std::list< std::pair > framesList = this->splitFrames(m_buffer, bytes_remaining); - Debug(1, "Got %d frames, bytes remaining %d", framesList.size(), bytes_remaining); + unsigned int data_size = 0; + int64_t pts; + unsigned char *header_end = nullptr; + unsigned char *header_start = nullptr; - if ( bytes_remaining > 0 ) { - memmove(&m_buffer[0], &m_buffer[0] + ( bytes_in_buffer - bytes_remaining ), bytes_remaining); - m_buffer_ptr = &m_buffer[0] + bytes_remaining; - } else { - m_buffer_ptr = &m_buffer[0]; - } - - while (framesList.size()) { - std::pair nal = framesList.front(); - framesList.pop_front(); - - NAL_Frame *frame = new NAL_Frame(nal.first, nal.second, tv); - - pthread_mutex_lock(&m_mutex); - if (m_captureQueue.size() > 10) { - NAL_Frame * f = m_captureQueue.front(); - while (m_captureQueue.size() and ((f->m_timestamp.tv_sec - tv.tv_sec) > 2)) { - m_captureQueue.pop_front(); - delete f; - f = m_captureQueue.front(); + if ((header_start = (unsigned char *)memmem(m_buffer.head(), m_buffer.size(), "ZM", 2))) { + // next step, look for \n + header_end = (unsigned char *)memchr(header_start, '\n', m_buffer.tail()-header_start); + if (!header_end) { + // Must not have enough data. So... keep all. + Debug(1, "Didn't find newline"); + return -1; } - } -#if 0 - while ( m_captureQueue.size() >= m_queueSize ) { - Debug(2, "Queue full dropping frame %d", m_captureQueue.size()); - NAL_Frame * f = m_captureQueue.front(); - m_captureQueue.pop_front(); - delete f; - } -#endif - m_captureQueue.push_back(frame); - pthread_mutex_unlock(&m_mutex); - // post an event to ask to deliver the frame - envir().taskScheduler().triggerEvent(m_eventTriggerId, this); - } // end while we get frame from data + unsigned int header_size = header_end-header_start; + char *header = new char[header_size+1]; + header[header_size] = '\0'; + strncpy(header, reinterpret_cast(header_start), header_end-header_start); + + char *content_length_ptr = strchr(header, ' '); + if (!content_length_ptr) { + Debug(1, "Didn't find space delineating size in %s", header); + m_buffer.consume(header_start-m_buffer.head() + 2); + delete header; + return -1; + } + *content_length_ptr = '\0'; + content_length_ptr ++; + char *pts_ptr = strchr(content_length_ptr, ' '); + if (!pts_ptr) { + m_buffer.consume(header_start-m_buffer.head() + 2); + Warning("Didn't find space delineating pts"); + delete header; + return -1; + } + *pts_ptr = '\0'; + pts_ptr ++; + data_size = atoi(content_length_ptr); + pts = strtoll(pts_ptr, nullptr, 10); + delete header; + } else { + Debug(1, "ZM header not found."); + return -1; + } + Debug(4, "ZM Packet size %u pts %" PRId64, data_size, pts); + if (header_start != m_buffer) { + Debug(4, "ZM Packet didn't start at beginning of buffer %u. %c%c", + header_start-m_buffer.head(), m_buffer[0], m_buffer[1]); + } + unsigned char *packet_start = header_end+1; + unsigned int header_size = packet_start - m_buffer.head(); // includes any bytes before header + + int bytes_needed = data_size - (m_buffer.size() - header_size); + if (bytes_needed > 0) { + Debug(4, "Need another %d bytes. Trying to read them", bytes_needed); + int bytes_read = m_buffer.read_into(m_fd, bytes_needed); + if ( bytes_read != bytes_needed ) + return -1; + } + + // splitFrames modifies so make a copy + unsigned int bytes_remaining = data_size; + std::list< std::pair > framesList = this->splitFrames(packet_start, bytes_remaining); + Debug(3, "Got %d frames, consuming %d bytes", framesList.size(), header_size + data_size); + m_buffer.consume(header_size + data_size); + + timeval tv; + tv.tv_sec = pts / 1000000; + tv.tv_usec = pts % 1000000; + + while (framesList.size()) { + std::pair nal = framesList.front(); + framesList.pop_front(); + + NAL_Frame *frame = new NAL_Frame(nal.first, nal.second, tv); + Debug(3, "Got frame, size %d, queue_size %d", frame->size(), m_captureQueue.size()); + + pthread_mutex_lock(&m_mutex); + if (m_captureQueue.size() > 25) { // 1 sec at 25 fps + NAL_Frame * f = m_captureQueue.front(); + while (m_captureQueue.size() and ((f->m_timestamp.tv_sec - tv.tv_sec) > 2)) { + m_captureQueue.pop_front(); + delete f; + f = m_captureQueue.front(); + } + } + m_captureQueue.push_back(frame); + pthread_mutex_unlock(&m_mutex); + + // post an event to ask to deliver the frame + envir().taskScheduler().triggerEvent(m_eventTriggerId, this); + } // end while we get frame from data + } // end while m_buffer.size() return 1; } // split packet in frames std::list< std::pair > ZoneMinderFifoSource::splitFrames(unsigned char* frame, unsigned &frameSize) { std::list< std::pair > frameList; - if ( frame != nullptr ) { + if (frame != nullptr) { frameList.push_back(std::pair(frame, frameSize)); } // We consume it all diff --git a/src/zm_rtsp_server_fifo_source.h b/src/zm_rtsp_server_fifo_source.h index 142946de2..990d32984 100644 --- a/src/zm_rtsp_server_fifo_source.h +++ b/src/zm_rtsp_server_fifo_source.h @@ -9,6 +9,7 @@ #ifndef ZM_RTSP_SERVER_FIFO_SOURCE_H #define ZM_RTSP_SERVER_FIFO_SOURCE_H +#include "zm_buffer.h" #include "zm_config.h" #include "zm_define.h" #include @@ -73,9 +74,7 @@ class ZoneMinderFifoSource: public FramedSource { int stop; int m_fd; - #define BUFFER_SIZE 65536 - unsigned char m_buffer[BUFFER_SIZE]; - unsigned char *m_buffer_ptr; + Buffer m_buffer; }; #endif // HAVE_RTSP_SERVER From 19d9812a3f68ce8040e225e295f062d52340b929 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 14:00:47 -0500 Subject: [PATCH 0033/1277] Add a debug function to print out the start of the nal --- src/zm_rtsp_server_frame.h | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/src/zm_rtsp_server_frame.h b/src/zm_rtsp_server_frame.h index 4afa31958..99c1223f4 100644 --- a/src/zm_rtsp_server_frame.h +++ b/src/zm_rtsp_server_frame.h @@ -52,6 +52,17 @@ class NAL_Frame { } return false; } + void debug() { + if ( m_size <= 4 ) { + Debug(1, "NAL: %d: %.2x %.2x %.2x %.2x", m_size, + m_buffer[0], m_buffer[1], m_buffer[2], m_buffer[3]); + } else { + Debug(1, "NAL: %d: %.2x %.2x %.2x %.2x %.2x %.2x %.2x %.2x ", m_size, + m_buffer[0], m_buffer[1], m_buffer[2], m_buffer[3], + m_buffer[4], m_buffer[5], m_buffer[6], m_buffer[7] + ); + } + } private: unsigned char* m_buffer; @@ -63,4 +74,4 @@ class NAL_Frame { }; #endif // HAVE_RTSP_SERVER -#endif // ZM_RTSP_SERVER_FRAME_H \ No newline at end of file +#endif // ZM_RTSP_SERVER_FRAME_H From db13b4c3b6be2c078bb6364e2a12db8d02416276 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 14:01:22 -0500 Subject: [PATCH 0034/1277] Document possible parameters, don't add a newline to auxline --- src/zm_rtsp_server_server_media_subsession.cpp | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/src/zm_rtsp_server_server_media_subsession.cpp b/src/zm_rtsp_server_server_media_subsession.cpp index 87f3fdfd7..93f94e894 100644 --- a/src/zm_rtsp_server_server_media_subsession.cpp +++ b/src/zm_rtsp_server_server_media_subsession.cpp @@ -24,7 +24,9 @@ FramedSource* BaseServerMediaSubsession::createSource( if (format == "video/MP2T") { source = MPEG2TransportStreamFramer::createNew(env, inputSource); } else if (format == "video/H264") { - source = H264VideoStreamDiscreteFramer::createNew(env, inputSource); + source = H264VideoStreamDiscreteFramer::createNew(env, inputSource + /*Boolean includeStartCodeInOutput, Boolean insertAccessUnitDelimiters*/ + ); } #if LIVEMEDIA_LIBRARY_VERSION_INT > 1414454400 else if (format == "video/H265") { @@ -94,9 +96,9 @@ char const* BaseServerMediaSubsession::getAuxLine( std::ostringstream os; os << "a=fmtp:" << int(rtpPayloadType) << " "; os << source->getAuxLine(); - os << "\r\n"; + //os << "\r\n"; auxLine = strdup(os.str().c_str()); - Debug(1, "auxLine: %s", auxLine); + Debug(1, "BaseServerMediaSubsession::auxLine: %s", auxLine); } else { Error("No source auxLine:"); return ""; From 6d938817b1e9a7321761f795d03f3194caaae07a Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 14:02:12 -0500 Subject: [PATCH 0035/1277] Rework to use a fifo source instead of Device. We no longer pass the monitor in and only open 1 port. Add addFifo function and addSession function --- src/zm_rtsp_server_thread.cpp | 28 ++++++++++++++++------------ src/zm_rtsp_server_thread.h | 3 ++- 2 files changed, 18 insertions(+), 13 deletions(-) diff --git a/src/zm_rtsp_server_thread.cpp b/src/zm_rtsp_server_thread.cpp index de6c9addb..1ed8e30e8 100644 --- a/src/zm_rtsp_server_thread.cpp +++ b/src/zm_rtsp_server_thread.cpp @@ -1,17 +1,17 @@ #include "zm_rtsp_server_thread.h" #include "zm_config.h" -#include "zm_rtsp_server_adts_source.h" +#include "zm_logger.h" #include "zm_rtsp_server_adts_fifo_source.h" -#include "zm_rtsp_server_h264_device_source.h" +#include "zm_rtsp_server_adts_source.h" #include "zm_rtsp_server_h264_fifo_source.h" +#include "zm_rtsp_server_h264_device_source.h" #include "zm_rtsp_server_unicast_server_media_subsession.h" #if HAVE_RTSP_SERVER #include -RTSPServerThread::RTSPServerThread(std::shared_ptr monitor) : - monitor_(std::move(monitor)), +RTSPServerThread::RTSPServerThread(int port) : terminate(0) { //unsigned short rtsp_over_http_port = 0; @@ -25,7 +25,7 @@ RTSPServerThread::RTSPServerThread(std::shared_ptr monitor) : //authDB = new UserAuthenticationDatabase("ZoneMinder"); //authDB->addUserRecord("username1", "password1"); // replace these with real strings - portNumBits rtspServerPortNum = config.min_rtsp_port + monitor_->Id(); + portNumBits rtspServerPortNum = port; rtspServer = RTSPServer::createNew(*env, rtspServerPortNum, authDB); if ( rtspServer == nullptr ) { @@ -37,7 +37,7 @@ RTSPServerThread::RTSPServerThread(std::shared_ptr monitor) : } // end RTSPServerThread::RTSPServerThread RTSPServerThread::~RTSPServerThread() { - if ( rtspServer ) { + if (rtspServer) { Medium::close(rtspServer); } // end if rtsp_server while ( sources.size() ) { @@ -80,26 +80,30 @@ ServerMediaSession *RTSPServerThread::addSession(std::string &streamname) { return sms; } +void RTSPServerThread::removeSession(ServerMediaSession *sms) { + rtspServer->removeServerMediaSession(sms); +} + FramedSource *RTSPServerThread::addFifo( ServerMediaSession *sms, std::string fifo) { if (!rtspServer) return nullptr; - int queueSize = 30; - bool repeatConfig = true; + int queueSize = 60; + bool repeatConfig = false; bool muxTS = false; FramedSource *source = nullptr; if (!fifo.empty()) { StreamReplicator* replicator = nullptr; std::string rtpFormat; - if (fifo.find("h264")) { + if (std::string::npos != fifo.find("h264")) { rtpFormat = "video/H264"; source = H264_ZoneMinderFifoSource::createNew(*env, fifo, queueSize, repeatConfig, muxTS); - } else if (fifo.find("h265")) { + } else if (std::string::npos != fifo.find("h265")) { rtpFormat = "video/H265"; source = H265_ZoneMinderFifoSource::createNew(*env, fifo, queueSize, repeatConfig, muxTS); - } else if (fifo.find("aac")) { + } else if (std::string::npos != fifo.find("aac")) { rtpFormat = "audio/AAC"; source = ADTS_ZoneMinderFifoSource::createNew(*env, fifo, queueSize); Debug(1, "ADTS source %p", source); @@ -117,7 +121,7 @@ FramedSource *RTSPServerThread::addFifo( sms->addSubsession(UnicastServerMediaSubsession::createNew(*env, replicator, rtpFormat)); } } else { - Debug(1, "Not Adding stream"); + Debug(1, "Not Adding stream as fifo was empty"); } return source; } // end void addFifo diff --git a/src/zm_rtsp_server_thread.h b/src/zm_rtsp_server_thread.h index 97edb05de..717ca1bdc 100644 --- a/src/zm_rtsp_server_thread.h +++ b/src/zm_rtsp_server_thread.h @@ -28,9 +28,10 @@ class RTSPServerThread : public Thread { std::list sources; public: - explicit RTSPServerThread(std::shared_ptr monitor); + explicit RTSPServerThread(int port); ~RTSPServerThread(); ServerMediaSession *addSession(std::string &streamname); + void removeSession(ServerMediaSession *sms); void addStream(std::string &streamname, AVStream *, AVStream *); FramedSource *addFifo(ServerMediaSession *sms, std::string fifo); int run(); From bfc12384f673a1f6c9ca016d0d641a8f9966a3fd Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 14:02:40 -0500 Subject: [PATCH 0036/1277] Increase debug level of auxline reporting --- src/zm_rtsp_server_h264_fifo_source.cpp | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/zm_rtsp_server_h264_fifo_source.cpp b/src/zm_rtsp_server_h264_fifo_source.cpp index cc8391f7c..3cbb6af2f 100644 --- a/src/zm_rtsp_server_h264_fifo_source.cpp +++ b/src/zm_rtsp_server_h264_fifo_source.cpp @@ -62,7 +62,6 @@ std::list< std::pair > H264_ZoneMinderFifoSource::splitF } break; default: - Debug(4, "Unknown frametype!? %d %d", m_frameType, m_frameType & 0x1F); break; } @@ -78,7 +77,7 @@ std::list< std::pair > H264_ZoneMinderFifoSource::splitF os << ";sprop-parameter-sets=" << sps_base64 << "," << pps_base64; os << "a=x-dimensions:" << m_width << "," << m_height << "\r\n"; m_auxLine.assign(os.str()); - Debug(1, "auxLine: %s", m_auxLine.c_str()); + Debug(3, "auxLine: %s", m_auxLine.c_str()); delete [] sps_base64; delete [] pps_base64; From 8af7e40ea16f3273e4f6c92608ef8fafd6fcc73d Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 14:03:08 -0500 Subject: [PATCH 0037/1277] Use fifo version of code --- src/zm_rtsp_server_unicast_server_media_subsession.cpp | 1 - 1 file changed, 1 deletion(-) diff --git a/src/zm_rtsp_server_unicast_server_media_subsession.cpp b/src/zm_rtsp_server_unicast_server_media_subsession.cpp index ede9ce527..2376748d5 100644 --- a/src/zm_rtsp_server_unicast_server_media_subsession.cpp +++ b/src/zm_rtsp_server_unicast_server_media_subsession.cpp @@ -10,7 +10,6 @@ #include "zm_rtsp_server_unicast_server_media_subsession.h" #include "zm_config.h" -#include "zm_rtsp_server_device_source.h" #include "zm_rtsp_server_fifo_source.h" #if HAVE_RTSP_SERVER From 8c2e6589ac9a438238d79197290c4281f837bd82 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 14:03:35 -0500 Subject: [PATCH 0038/1277] fifo.h got split into zm_fifo and zm_fifo_debug so update the code in zm_zone --- src/zm_zone.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/zm_zone.cpp b/src/zm_zone.cpp index 50c287ec8..749fff387 100644 --- a/src/zm_zone.cpp +++ b/src/zm_zone.cpp @@ -20,6 +20,7 @@ #include "zm_zone.h" #include "zm_fifo.h" +#include "zm_fifo_debug.h" #include "zm_monitor.h" void Zone::Setup( @@ -113,7 +114,7 @@ void Zone::Setup( "%s/diagpipe-%d-poly.jpg", staticConfig.PATH_SOCKS.c_str(), id); - FifoStream::fifo_create_if_missing(diag_path); + Fifo::fifo_create_if_missing(diag_path); } else { snprintf(diag_path, sizeof(diag_path), "%s/diag-%d-poly.jpg", monitor->getStorage()->Path(), id); From b4fc782778a0d1a85c2d68ffb179d913d4d20594 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 14:03:52 -0500 Subject: [PATCH 0039/1277] fifo.h got split into zm_fifo and zm_stream so update the code in zms --- src/zms.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/zms.cpp b/src/zms.cpp index 01df67fc7..b7d88d35b 100644 --- a/src/zms.cpp +++ b/src/zms.cpp @@ -23,7 +23,7 @@ #include "zm_signal.h" #include "zm_monitorstream.h" #include "zm_eventstream.h" -#include "zm_fifo.h" +#include "zm_fifo_stream.h" #include bool ValidateAccess(User *user, int mon_id) { From cc80ae37fb363ae03aca8c11a923d59b1808f165 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 16:42:31 -0500 Subject: [PATCH 0040/1277] Add a generic fif_audio_source class --- src/zm_rtsp_server_fifo_audio_source.cpp | 38 ++++++++++++++++ src/zm_rtsp_server_fifo_audio_source.h | 56 ++++++++++++++++++++++++ 2 files changed, 94 insertions(+) create mode 100644 src/zm_rtsp_server_fifo_audio_source.cpp create mode 100644 src/zm_rtsp_server_fifo_audio_source.h diff --git a/src/zm_rtsp_server_fifo_audio_source.cpp b/src/zm_rtsp_server_fifo_audio_source.cpp new file mode 100644 index 000000000..371ef6985 --- /dev/null +++ b/src/zm_rtsp_server_fifo_audio_source.cpp @@ -0,0 +1,38 @@ +/* --------------------------------------------------------------------------- +** +** ADTS_FifoSource.cpp +** +** ADTS Live555 source +** +** -------------------------------------------------------------------------*/ + +#include "zm_logger.h" +#include "zm_rtsp_server_fifo_audio_source.h" + +#include + +#if HAVE_RTSP_SERVER + +static unsigned const samplingFrequencyTable[16] = { + 96000, 88200, 64000, 48000, + 44100, 32000, 24000, 22050, + 16000, 12000, 11025, 8000, + 7350, 0, 0, 0 +}; +// --------------------------------- +// ADTS ZoneMinder FramedSource +// --------------------------------- +// +ZoneMinderFifoAudioSource::ZoneMinderFifoAudioSource( + UsageEnvironment& env, + std::string fifo, + unsigned int queueSize + ) + : + ZoneMinderFifoSource(env, fifo, queueSize), + samplingFrequencyIndex(-1), + frequency(-1), + channels(1) +{ +} +#endif // HAVE_RTSP_SERVER diff --git a/src/zm_rtsp_server_fifo_audio_source.h b/src/zm_rtsp_server_fifo_audio_source.h new file mode 100644 index 000000000..84827237f --- /dev/null +++ b/src/zm_rtsp_server_fifo_audio_source.h @@ -0,0 +1,56 @@ +/* --------------------------------------------------------------------------- +** This software is in the public domain, furnished "as is", without technical +** support, and with no warranty, express or implied, as to its usefulness for +** any purpose. +** +** ADTS_ZoneMinderFifoSource.h +** +** ADTS ZoneMinder live555 source +** +** -------------------------------------------------------------------------*/ + +#ifndef ZM_RTSP_SERVER_FIFO_AUDIO_SOURCE_H +#define ZM_RTSP_SERVER_FIFO_AUDIO_SOURCE_H + +#include "zm_config.h" +#include "zm_rtsp_server_fifo_source.h" + +#if HAVE_RTSP_SERVER +// --------------------------------- +// ZoneMinder AUDIO FramedSource +// --------------------------------- + +class ZoneMinderFifoAudioSource : public ZoneMinderFifoSource { + public: + static ZoneMinderFifoAudioSource* createNew( + UsageEnvironment& env, + std::string fifo, + unsigned int queueSize + ) { + return new ZoneMinderFifoAudioSource(env, fifo, queueSize); + }; + protected: + ZoneMinderFifoAudioSource( + UsageEnvironment& env, + std::string fifo, + unsigned int queueSize + ); + + virtual ~ZoneMinderFifoAudioSource() {} + public: + + void setFrequency(int p_frequency) { frequency = p_frequency; }; + int getFrequency() { return frequency; }; + const char *configStr() const { return config.c_str(); }; + void setChannels(int p_channels) { channels = p_channels; }; + int getChannels() const { return channels; }; + + protected: + std::string config; + int samplingFrequencyIndex; + int frequency; + int channels; +}; +#endif // HAVE_RTSP_SERVER + +#endif // ZM_RTSP_SERVER_FIFO_AUDIO_SOURCE_H From c5ee837b9b6fb856ad5328b193489884444f9808 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 16:42:53 -0500 Subject: [PATCH 0041/1277] Add getFrequency and getChannels --- src/zm_camera.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/src/zm_camera.h b/src/zm_camera.h index d3d7c4ce1..10797c987 100644 --- a/src/zm_camera.h +++ b/src/zm_camera.h @@ -92,6 +92,20 @@ public: unsigned int Pixels() const { return pixels; } unsigned long long ImageSize() const { return imagesize; } unsigned int Bytes() const { return bytes; }; + int getFrequency() { +#if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) + return mAudioStream ? mAudioStream->codecpar->sample_rate : -1; +#else + return mAudioStream ? mAudioStream->codec->sample_rate : -1; +#endif + } + int getChannels() { +#if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) + return mAudioStream ? mAudioStream->codecpar->channels : -1; +#else + return mAudioStream ? mAudioStream->codec->channels : -1; +#endif + } virtual int Brightness( int/*p_brightness*/=-1 ) { return -1; } virtual int Hue( int/*p_hue*/=-1 ) { return -1; } From 8500c7ee947909a02319b1c68eea411ebef67c2f Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 16:43:08 -0500 Subject: [PATCH 0042/1277] Add audio_frequency and audio_channels to shared mem --- src/zm_monitor.h | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/src/zm_monitor.h b/src/zm_monitor.h index 3f16785a0..b4cc577a0 100644 --- a/src/zm_monitor.h +++ b/src/zm_monitor.h @@ -125,32 +125,29 @@ protected: uint8_t format; /* +55 */ uint32_t imagesize; /* +56 */ uint32_t last_frame_score; /* +60 */ - // uint32_t epadding1; /* +60 */ + uint32_t audio_frequency; /* +64 */ + uint32_t audio_channels; /* +68 */ /* ** This keeps 32bit time_t and 64bit time_t identical and compatible as long as time is before 2038. ** Shared memory layout should be identical for both 32bit and 64bit and is multiples of 16. ** Because startup_time is 64bit it may be aligned to a 64bit boundary. So it's offset SHOULD be a multiple ** of 8. Add or delete epadding's to achieve this. */ - union { /* +64 */ + union { /* +72 */ time_t startup_time; /* When the zmc process started. zmwatch uses this to see how long the process has been running without getting any images */ uint64_t extrapad1; }; - union { /* +72 */ + union { /* +80 */ time_t zmc_heartbeat_time; /* Constantly updated by zmc. Used to determine if the process is alive or hung or dead */ uint64_t extrapad2; }; - union { /* +80 */ - time_t zma_heartbeat_time; /* Constantly updated by zma. Used to determine if the process is alive or hung or dead */ - uint64_t extrapad3; - }; union { /* +88 */ time_t last_write_time; - uint64_t extrapad4; + uint64_t extrapad3; }; union { /* +96 */ time_t last_read_time; - uint64_t extrapad5; + uint64_t extrapad4; }; uint8_t control_state[256]; /* +104 */ @@ -467,6 +464,9 @@ public: unsigned int Colours() const; unsigned int SubpixelOrder() const; + int GetAudioFrequency() const { return shared_data ? shared_data->audio_frequency : -1; } + int GetAudioChannels() const { return shared_data ? shared_data->audio_channels : -1; } + int GetOptSaveJPEGs() const { return savejpegs; } VideoWriter GetOptVideoWriter() const { return videowriter; } //const std::vector* GetEncoderParams() const { return &encoderparamsvec; } From a62d446f1476880d8f096a0fc756a0e7ee146d84 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 16:43:18 -0500 Subject: [PATCH 0043/1277] Add audio_frequency and audio_channels to shared mem --- scripts/ZoneMinder/lib/ZoneMinder/Memory.pm.in | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/ZoneMinder/lib/ZoneMinder/Memory.pm.in b/scripts/ZoneMinder/lib/ZoneMinder/Memory.pm.in index 1e4b565cc..5b8cc0f06 100644 --- a/scripts/ZoneMinder/lib/ZoneMinder/Memory.pm.in +++ b/scripts/ZoneMinder/lib/ZoneMinder/Memory.pm.in @@ -163,9 +163,10 @@ our $mem_data = { format => { type=>'uint8', seq=>$mem_seq++ }, imagesize => { type=>'uint32', seq=>$mem_seq++ }, last_frame_score => { type=>'uint32', seq=>$mem_seq++ }, + audio_frequency => { type=>'uint32', seq=>$mem_seq++ }, + audio_channels => { type=>'uint32', seq=>$mem_seq++ }, startup_time => { type=>'time_t64', seq=>$mem_seq++ }, zmc_heartbeat_time => { type=>'time_t64', seq=>$mem_seq++ }, - zma_heartbeat_time => { type=>'time_t64', seq=>$mem_seq++ }, last_write_time => { type=>'time_t64', seq=>$mem_seq++ }, last_read_time => { type=>'time_t64', seq=>$mem_seq++ }, control_state => { type=>'uint8[256]', seq=>$mem_seq++ }, From 9302c9506e61dff99bd4f679156b319ca1c22e1d Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 16:43:38 -0500 Subject: [PATCH 0044/1277] Add audio_frequency and audio_channels to shared mem and set them in PrimeCapture --- src/zm_monitor.cpp | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 0e9b8e70b..4564e79cd 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -1015,6 +1015,8 @@ bool Monitor::connect() { shared_data->video_fifo_path[0] = 0; shared_data->audio_fifo_path[0] = 0; shared_data->last_frame_score = 0; + shared_data->audio_frequency = -1; + shared_data->audio_channels = -1; trigger_data->size = sizeof(TriggerData); trigger_data->trigger_state = TriggerState::TRIGGER_CANCEL; trigger_data->trigger_score = 0; @@ -2968,8 +2970,11 @@ int Monitor::PrimeCapture() { packetqueue.addStreamId(video_stream_id); audio_stream_id = camera->get_AudioStreamId(); - if ( audio_stream_id >= 0 ) + if ( audio_stream_id >= 0 ) { packetqueue.addStreamId(audio_stream_id); + shared_data->audio_frequency = camera->getFrequency(); + shared_data->audio_channels = camera->getChannels(); + } Debug(2, "Video stream id is %d, audio is %d, minimum_packets to keep in buffer %d", video_stream_id, audio_stream_id, pre_event_buffer_count); From b09fe1d17c2ad8687df4662750e7f0cc1f2b86f8 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 16:43:51 -0500 Subject: [PATCH 0045/1277] Create fifo if it doesn't exist already --- src/zm_fifo.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/src/zm_fifo.cpp b/src/zm_fifo.cpp index e1cb57625..7f737426e 100644 --- a/src/zm_fifo.cpp +++ b/src/zm_fifo.cpp @@ -61,6 +61,7 @@ Fifo::~Fifo() { close(); } bool Fifo::open() { + fifo_create_if_missing(path.c_str()); if (!on_blocking_abort) { if ( (outfile = fopen(path.c_str(), "wb")) == nullptr ) { Error("Can't open %s for writing: %s", path.c_str(), strerror(errno)); From b2ff9af23836af4a10d7094f6ba43c76e8b642b4 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 16:44:09 -0500 Subject: [PATCH 0046/1277] Add setting of frequency and channels for audio stream --- src/zm_rtsp_server.cpp | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/src/zm_rtsp_server.cpp b/src/zm_rtsp_server.cpp index 0e280e96b..563a219e7 100644 --- a/src/zm_rtsp_server.cpp +++ b/src/zm_rtsp_server.cpp @@ -52,6 +52,7 @@ and provide that stream over rtsp #include "zm_define.h" #include "zm_monitor.h" #include "zm_rtsp_server_thread.h" +#include "zm_rtsp_server_fifo_audio_source.h" #include "zm_rtsp_server_fifo_video_source.h" #include "zm_signal.h" #include "zm_time.h" @@ -199,10 +200,17 @@ int main(int argc, char *argv[]) { video_source->setWidth(monitor->Width()); video_source->setHeight(monitor->Height()); } - Debug(1, "Adding audio fifo %s", monitor->GetAudioFifoPath().c_str()); - FramedSource *audio_source = rtsp_server_thread->addFifo(sms, monitor->GetAudioFifoPath()); + + std::string audioFifoPath = monitor->GetAudioFifoPath(); + if (audioFifoPath.empty()) { + Debug(1, "audio fifo is empty. Skipping."); + continue; + } + Debug(1, "Adding audio fifo %s", audioFifoPath.c_str()); + ZoneMinderFifoAudioSource *audio_source = static_cast(rtsp_server_thread->addFifo(sms, audioFifoPath)); if (audio_source) { - // set frequency + audio_source->setFrequency(monitor->GetAudioFrequency()); + audio_source->setChannels(monitor->GetAudioChannels()); } } // end if ! sessions[i] } // end foreach monitor From 495e2a48276233e4f8deaa68ed655efd1f8ac5a2 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 16:44:36 -0500 Subject: [PATCH 0047/1277] Add setting up the config string and population of mAuxLine --- src/zm_rtsp_server_adts_fifo_source.cpp | 19 +++++++++++++---- src/zm_rtsp_server_adts_fifo_source.h | 27 ++----------------------- 2 files changed, 17 insertions(+), 29 deletions(-) diff --git a/src/zm_rtsp_server_adts_fifo_source.cpp b/src/zm_rtsp_server_adts_fifo_source.cpp index 757e927eb..d62b04df1 100644 --- a/src/zm_rtsp_server_adts_fifo_source.cpp +++ b/src/zm_rtsp_server_adts_fifo_source.cpp @@ -9,6 +9,7 @@ #include "zm_logger.h" #include "zm_rtsp_server_adts_fifo_source.h" +#include #include #if HAVE_RTSP_SERVER @@ -29,17 +30,27 @@ ADTS_ZoneMinderFifoSource::ADTS_ZoneMinderFifoSource( unsigned int queueSize ) : - ZoneMinderFifoSource(env, fifo, queueSize), - samplingFrequencyIndex(11), - channels(1) + ZoneMinderFifoAudioSource(env, fifo, queueSize) { +#if 1 + int profile = 0; + + unsigned char audioSpecificConfig[2]; + u_int8_t const audioObjectType = profile + 1; + audioSpecificConfig[0] = (audioObjectType<<3) | (samplingFrequencyIndex>>1); + audioSpecificConfig[1] = (samplingFrequencyIndex<<7) | (channels<<3); + std::ostringstream os; os << "profile-level-id=1;" "mode=AAC-hbr;sizelength=13;indexlength=3;" - "indexdeltalength=3" + "indexdeltalength=3;config=" << std::hex << std::setw(2) << std::setfill('0') << audioSpecificConfig[0] + << std::hex << std::setw(2) << std::setfill('0') << audioSpecificConfig[1] << "\r\n"; + // Construct the 'AudioSpecificConfig', and from it, the corresponding ASCII string: + m_auxLine.assign(os.str()); Debug(1, "m_auxline is %s", m_auxLine.c_str()); +#endif } #endif // HAVE_RTSP_SERVER diff --git a/src/zm_rtsp_server_adts_fifo_source.h b/src/zm_rtsp_server_adts_fifo_source.h index 7278f1be1..df4abd417 100644 --- a/src/zm_rtsp_server_adts_fifo_source.h +++ b/src/zm_rtsp_server_adts_fifo_source.h @@ -13,14 +13,14 @@ #define ZM_RTSP_SERVER_ADTS_FIFO_SOURCE_H #include "zm_config.h" -#include "zm_rtsp_server_fifo_source.h" +#include "zm_rtsp_server_fifo_audio_source.h" #if HAVE_RTSP_SERVER // --------------------------------- // ADTS(AAC) ZoneMinder FramedSource // --------------------------------- -class ADTS_ZoneMinderFifoSource : public ZoneMinderFifoSource { +class ADTS_ZoneMinderFifoSource : public ZoneMinderFifoAudioSource { public: static ADTS_ZoneMinderFifoSource* createNew( UsageEnvironment& env, @@ -37,29 +37,6 @@ class ADTS_ZoneMinderFifoSource : public ZoneMinderFifoSource { ); virtual ~ADTS_ZoneMinderFifoSource() {} - - /* - virtual unsigned char* extractFrame(unsigned char* frame, size_t& size, size_t& outsize); - virtual unsigned char* findMarker(unsigned char *frame, size_t size, size_t &length); - */ - public: - int samplingFrequency() { return 8000; //m_stream->codecpar->sample_rate; - }; - const char *configStr() { return config.c_str(); }; - int numChannels() { - //Debug(1, "this %p m_stream %p channels %d", - //this, m_stream, channels); - //Debug(1, "m_stream %p codecpar %p channels %d => %d", - //m_stream, m_stream->codecpar, m_stream->codecpar->channels, channels); - return 1; - //return channels; - //return m_stream->codecpar->channels; - } - - protected: - std::string config; - int samplingFrequencyIndex; - int channels; }; #endif // HAVE_RTSP_SERVER From d33540c962093df6b7736b8a3c06c8010dc044ba Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 16:44:56 -0500 Subject: [PATCH 0048/1277] Fix logic on clearing frame queue --- src/zm_rtsp_server_fifo_source.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/zm_rtsp_server_fifo_source.cpp b/src/zm_rtsp_server_fifo_source.cpp index d7ad62ee4..3aa5f85a6 100644 --- a/src/zm_rtsp_server_fifo_source.cpp +++ b/src/zm_rtsp_server_fifo_source.cpp @@ -233,7 +233,8 @@ int ZoneMinderFifoSource::getNextFrame() { pthread_mutex_lock(&m_mutex); if (m_captureQueue.size() > 25) { // 1 sec at 25 fps NAL_Frame * f = m_captureQueue.front(); - while (m_captureQueue.size() and ((f->m_timestamp.tv_sec - tv.tv_sec) > 2)) { + while (m_captureQueue.size() and ((tv.tv_sec - f->m_timestamp.tv_sec) > 2)) { + Debug(1, "Deleting Front NAL is %d seconds old", (tv.tv_sec - f->m_timestamp.tv_sec)); m_captureQueue.pop_front(); delete f; f = m_captureQueue.front(); From b86e7eefbfda1a64374b2af7ef7c1362198f718e Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 16:45:34 -0500 Subject: [PATCH 0049/1277] fix needed line break before dimensions. Update h265 auxline processing to match h264 --- src/zm_rtsp_server_h264_fifo_source.cpp | 16 ++++++++++------ src/zm_rtsp_server_h264_fifo_source.h | 2 +- 2 files changed, 11 insertions(+), 7 deletions(-) diff --git a/src/zm_rtsp_server_h264_fifo_source.cpp b/src/zm_rtsp_server_h264_fifo_source.cpp index 3cbb6af2f..979bca18d 100644 --- a/src/zm_rtsp_server_h264_fifo_source.cpp +++ b/src/zm_rtsp_server_h264_fifo_source.cpp @@ -75,7 +75,7 @@ std::list< std::pair > H264_ZoneMinderFifoSource::splitF std::ostringstream os; os << "profile-level-id=" << std::hex << std::setw(6) << std::setfill('0') << profile_level_id; os << ";sprop-parameter-sets=" << sps_base64 << "," << pps_base64; - os << "a=x-dimensions:" << m_width << "," << m_height << "\r\n"; + os << "\r\n" << "a=x-dimensions:" << m_width << "," << m_height << "\r\n"; m_auxLine.assign(os.str()); Debug(3, "auxLine: %s", m_auxLine.c_str()); @@ -104,25 +104,29 @@ H265_ZoneMinderFifoSource::H265_ZoneMinderFifoSource( // split packet in frames std::list< std::pair > -H265_ZoneMinderFifoSource::splitFrames(unsigned char* frame, unsigned frameSize) { +H265_ZoneMinderFifoSource::splitFrames(unsigned char* frame, unsigned &frameSize) { std::list< std::pair > frameList; size_t bufSize = frameSize; size_t size = 0; unsigned char* buffer = this->extractFrame(frame, bufSize, size); + bool updateAux = false; while ( buffer != nullptr ) { switch ((m_frameType&0x7E)>>1) { case 32: Debug(4, "VPS_Size: %d bufSize %d", size, bufSize); m_vps.assign((char*)buffer,size); + updateAux = true; break; case 33: Debug(4, "SPS_Size: %d bufSize %d", size, bufSize); m_sps.assign((char*)buffer,size); + updateAux = true; break; case 34: Debug(4, "PPS_Size: %d bufSize %d", size, bufSize); m_pps.assign((char*)buffer,size); + updateAux = true; break; case 19: case 20: @@ -134,11 +138,11 @@ H265_ZoneMinderFifoSource::splitFrames(unsigned char* frame, unsigned frameSize) } break; default: - Debug(4, "Unknown frametype!? %d %d", m_frameType, ((m_frameType & 0x7E) >> 1)); + //Debug(4, "Unknown frametype!? %d %d", m_frameType, ((m_frameType & 0x7E) >> 1)); break; } - if ( !m_vps.empty() && !m_sps.empty() && !m_pps.empty() ) { + if ( updateAux and !m_vps.empty() and !m_sps.empty() and !m_pps.empty() ) { char* vps_base64 = base64Encode(m_vps.c_str(), m_vps.size()); char* sps_base64 = base64Encode(m_sps.c_str(), m_sps.size()); char* pps_base64 = base64Encode(m_pps.c_str(), m_pps.size()); @@ -147,9 +151,9 @@ H265_ZoneMinderFifoSource::splitFrames(unsigned char* frame, unsigned frameSize) os << "sprop-vps=" << vps_base64; os << ";sprop-sps=" << sps_base64; os << ";sprop-pps=" << pps_base64; - os << "a=x-dimensions:" << m_width << "," << m_height << "\r\n"; + os << "\r\n" << "a=x-dimensions:" << m_width << "," << m_height << "\r\n"; m_auxLine.assign(os.str()); - Debug(1, "Assigned auxLine to %s", m_auxLine.c_str()); + Debug(1, "Assigned h265 auxLine to %s", m_auxLine.c_str()); delete [] vps_base64; delete [] sps_base64; diff --git a/src/zm_rtsp_server_h264_fifo_source.h b/src/zm_rtsp_server_h264_fifo_source.h index 9896c6122..27ca2ef71 100644 --- a/src/zm_rtsp_server_h264_fifo_source.h +++ b/src/zm_rtsp_server_h264_fifo_source.h @@ -89,7 +89,7 @@ class H265_ZoneMinderFifoSource : public H26X_ZoneMinderFifoSource { bool keepMarker); // overide ZoneMinderFifoSource - virtual std::list< std::pair > splitFrames(unsigned char* frame, unsigned frameSize); + virtual std::list< std::pair > splitFrames(unsigned char* frame, unsigned &frameSize); protected: std::string m_vps; From 53059aebf3b63ddb79ede83536a1d84cfcf772df Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 16:46:02 -0500 Subject: [PATCH 0050/1277] rename numChannels to getChannels and samplingFrequency to getFrequency --- src/zm_rtsp_server_server_media_subsession.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/zm_rtsp_server_server_media_subsession.cpp b/src/zm_rtsp_server_server_media_subsession.cpp index 93f94e894..141e8dad1 100644 --- a/src/zm_rtsp_server_server_media_subsession.cpp +++ b/src/zm_rtsp_server_server_media_subsession.cpp @@ -71,10 +71,10 @@ RTPSink* BaseServerMediaSubsession::createSink( ADTS_ZoneMinderFifoSource *adts_source = (ADTS_ZoneMinderFifoSource *)(m_replicator->inputSource()); sink = MPEG4GenericRTPSink::createNew(env, rtpGroupsock, rtpPayloadTypeIfDynamic, - adts_source->samplingFrequency(), + adts_source->getFrequency(), "audio", "AAC-hbr", adts_source->configStr(), - adts_source->numChannels() + adts_source->getChannels() ); } else { Error("unknown format"); From 8f001413f2a6687bd2420fdd3d3cecc00189471b Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 16:46:15 -0500 Subject: [PATCH 0051/1277] Handle hevc as the codec name as well as h265 --- src/zm_rtsp_server_thread.cpp | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/zm_rtsp_server_thread.cpp b/src/zm_rtsp_server_thread.cpp index 1ed8e30e8..30e1bd0d9 100644 --- a/src/zm_rtsp_server_thread.cpp +++ b/src/zm_rtsp_server_thread.cpp @@ -100,7 +100,10 @@ FramedSource *RTSPServerThread::addFifo( if (std::string::npos != fifo.find("h264")) { rtpFormat = "video/H264"; source = H264_ZoneMinderFifoSource::createNew(*env, fifo, queueSize, repeatConfig, muxTS); - } else if (std::string::npos != fifo.find("h265")) { + } else if ( + std::string::npos != fifo.find("hevc") + or + std::string::npos != fifo.find("h265")) { rtpFormat = "video/H265"; source = H265_ZoneMinderFifoSource::createNew(*env, fifo, queueSize, repeatConfig, muxTS); } else if (std::string::npos != fifo.find("aac")) { From 82dcd87a2218519241aa8d1c0da0a81da3a5112c Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 16:46:29 -0500 Subject: [PATCH 0052/1277] Add generic fifo_audio_source --- src/CMakeLists.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index f2e899afb..4f4fe9546 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -59,6 +59,7 @@ set(ZM_BIN_SRC_FILES zm_rtsp_server_h264_fifo_source.cpp zm_rtsp_server_device_source.cpp zm_rtsp_server_fifo_source.cpp + zm_rtsp_server_fifo_audio_source.cpp zm_rtsp_server_fifo_video_source.cpp zm_rtsp_server_server_media_subsession.cpp zm_rtsp_server_unicast_server_media_subsession.cpp From 51c19912f8bcb16c3bd649e175375705b43f1671 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 16:56:47 -0500 Subject: [PATCH 0053/1277] Use analysis_image_count in opening new event log line --- src/zm_monitor.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 4564e79cd..5170a3de7 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -2102,7 +2102,7 @@ bool Monitor::Analyse() { video_store_data->recording = event->StartTime(); shared_data->state = state = ALARM; - Info("%s: %03d - Opening new event %" PRIu64 ", alarm start", name, image_count, event->Id()); + Info("%s: %03d - Opening new event %" PRIu64 ", alarm start", name, analysis_image_count, event->Id()); } // end if no event, so start it if ( alarm_frame_count ) { Debug(1, "alarm frame count so SavePreAlarmFrames"); From 7c71c1c54342f505ae533af6e1477ca64197a5a9 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 17:06:53 -0500 Subject: [PATCH 0054/1277] fix rebase error --- src/CMakeLists.txt | 8 -------- 1 file changed, 8 deletions(-) diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index 4f4fe9546..b68077235 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -103,14 +103,6 @@ target_link_libraries(zmc ${ZM_BIN_LIBS} ${CMAKE_DL_LIBS}) -target_link_libraries(zm_rtsp_server - PRIVATE - zm-core-interface - zm - ${ZM_EXTRA_LIBS} - ${ZM_BIN_LIBS} - ${CMAKE_DL_LIBS}) - target_link_libraries(zms PRIVATE zm-core-interface From 1bc5abb9e9477fb2d0088a1cc7ee064c0c2a683c Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 1 Mar 2021 17:07:09 -0500 Subject: [PATCH 0055/1277] Only include video packets in analysis_image_count --- src/zm_monitor.cpp | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 5170a3de7..b6d5b834b 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -2114,7 +2114,8 @@ bool Monitor::Analyse() { } } else if ( state == ALERT ) { alert_to_alarm_frame_count--; - Info("%s: %03d - Alarmed frame while in alert state. Consecutive alarmed frames left to return to alarm state: %03d", name, analysis_image_count,alert_to_alarm_frame_count); + Info("%s: %03d - Alarmed frame while in alert state. Consecutive alarmed frames left to return to alarm state: %03d", + name, analysis_image_count, alert_to_alarm_frame_count); if (alert_to_alarm_frame_count == 0) { Info("%s: %03d - Gone back into alarm state", name, analysis_image_count); shared_data->state = state = ALARM; @@ -2148,7 +2149,7 @@ bool Monitor::Analyse() { // Back to IDLE shared_data->state = state = ((function != MOCORD) ? IDLE : TAPE); } else { - Debug(1, "State %d beacuse image_count(%d)-last_alarm_count(%d) > post_event_count(%d) and timestamp.tv_sec(%d) - recording.tv_src(%d) >= min_section_length(%d)", + Debug(1, "State %d because image_count(%d)-last_alarm_count(%d) > post_event_count(%d) and timestamp.tv_sec(%d) - recording.tv_src(%d) >= min_section_length(%d)", state, analysis_image_count, last_alarm_count, post_event_count, timestamp->tv_sec, video_store_data->recording.tv_sec, min_section_length); } @@ -2262,10 +2263,13 @@ bool Monitor::Analyse() { // popPacket will have placed a second lock on snap, so release it here. snap->unlock(); - shared_data->last_read_index = snap->image_index; + if ( snap->image_index > 0 ) { + // Only do these if it's a video packet. + shared_data->last_read_index = snap->image_index; + analysis_image_count++; + UpdateAnalysisFPS(); + } shared_data->last_read_time = time(nullptr); - analysis_image_count++; - UpdateAnalysisFPS(); packetqueue.clearPackets(snap); return true; From 37817f6ba42ab0fa05f3869346de36df61d2bf4f Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Mon, 1 Mar 2021 23:14:56 +0100 Subject: [PATCH 0056/1277] dep/jwt-cpp: Update to version 0.5 Fixes some compile warnings. Unit tests pass with this version as well. --- dep/jwt-cpp/BaseTest.cpp | 30 - dep/jwt-cpp/CMakeLists.txt | 2 +- dep/jwt-cpp/ClaimTest.cpp | 33 - dep/jwt-cpp/Doxyfile | 10 +- dep/jwt-cpp/HelperTest.cpp | 55 - dep/jwt-cpp/README.md | 208 +- dep/jwt-cpp/TestMain.cpp | 7 - dep/jwt-cpp/TokenFormatTest.cpp | 15 - dep/jwt-cpp/TokenTest.cpp | 420 - dep/jwt-cpp/include/jwt-cpp/base.h | 158 +- dep/jwt-cpp/include/jwt-cpp/jwt.h | 3037 ++ dep/jwt-cpp/include/jwt-cpp/jwt_cpp.h | 1593 - dep/jwt-cpp/include/nlohmann/json.hpp | 25447 ++++++++++++++++ .../include/{jwt-cpp => picojson}/picojson.h | 52 +- dep/jwt-cpp/jwt-cpp.sln | 28 - dep/jwt-cpp/jwt-cpp.vcxproj | 160 - dep/jwt-cpp/jwt-cpp.vcxproj.filters | 36 - dep/jwt-cpp/vcpkg/CONTROL | 3 - dep/jwt-cpp/vcpkg/fix-picojson.patch | 12 - dep/jwt-cpp/vcpkg/fix-warning.patch | 31 - dep/jwt-cpp/vcpkg/portfile.cmake | 23 - src/zm_crypt.cpp | 2 +- 22 files changed, 28791 insertions(+), 2571 deletions(-) delete mode 100644 dep/jwt-cpp/BaseTest.cpp delete mode 100644 dep/jwt-cpp/ClaimTest.cpp delete mode 100644 dep/jwt-cpp/HelperTest.cpp delete mode 100644 dep/jwt-cpp/TestMain.cpp delete mode 100644 dep/jwt-cpp/TokenFormatTest.cpp delete mode 100644 dep/jwt-cpp/TokenTest.cpp create mode 100644 dep/jwt-cpp/include/jwt-cpp/jwt.h delete mode 100644 dep/jwt-cpp/include/jwt-cpp/jwt_cpp.h create mode 100644 dep/jwt-cpp/include/nlohmann/json.hpp rename dep/jwt-cpp/include/{jwt-cpp => picojson}/picojson.h (96%) delete mode 100644 dep/jwt-cpp/jwt-cpp.sln delete mode 100644 dep/jwt-cpp/jwt-cpp.vcxproj delete mode 100644 dep/jwt-cpp/jwt-cpp.vcxproj.filters delete mode 100644 dep/jwt-cpp/vcpkg/CONTROL delete mode 100644 dep/jwt-cpp/vcpkg/fix-picojson.patch delete mode 100644 dep/jwt-cpp/vcpkg/fix-warning.patch delete mode 100644 dep/jwt-cpp/vcpkg/portfile.cmake diff --git a/dep/jwt-cpp/BaseTest.cpp b/dep/jwt-cpp/BaseTest.cpp deleted file mode 100644 index aceeb38d4..000000000 --- a/dep/jwt-cpp/BaseTest.cpp +++ /dev/null @@ -1,30 +0,0 @@ -#include -#include "include/jwt-cpp/base.h" - -TEST(BaseTest, Base64Decode) { - ASSERT_EQ("1", jwt::base::decode("MQ==")); - ASSERT_EQ("12", jwt::base::decode("MTI=")); - ASSERT_EQ("123", jwt::base::decode("MTIz")); - ASSERT_EQ("1234", jwt::base::decode("MTIzNA==")); -} - -TEST(BaseTest, Base64DecodeURL) { - ASSERT_EQ("1", jwt::base::decode("MQ%3d%3d")); - ASSERT_EQ("12", jwt::base::decode("MTI%3d")); - ASSERT_EQ("123", jwt::base::decode("MTIz")); - ASSERT_EQ("1234", jwt::base::decode("MTIzNA%3d%3d")); -} - -TEST(BaseTest, Base64Encode) { - ASSERT_EQ("MQ==", jwt::base::encode("1")); - ASSERT_EQ("MTI=", jwt::base::encode("12")); - ASSERT_EQ("MTIz", jwt::base::encode("123")); - ASSERT_EQ("MTIzNA==", jwt::base::encode("1234")); -} - -TEST(BaseTest, Base64EncodeURL) { - ASSERT_EQ("MQ%3d%3d", jwt::base::encode("1")); - ASSERT_EQ("MTI%3d", jwt::base::encode("12")); - ASSERT_EQ("MTIz", jwt::base::encode("123")); - ASSERT_EQ("MTIzNA%3d%3d", jwt::base::encode("1234")); -} \ No newline at end of file diff --git a/dep/jwt-cpp/CMakeLists.txt b/dep/jwt-cpp/CMakeLists.txt index c2643f583..81ddc84a1 100644 --- a/dep/jwt-cpp/CMakeLists.txt +++ b/dep/jwt-cpp/CMakeLists.txt @@ -3,4 +3,4 @@ add_library(jwt-cpp::jwt-cpp ALIAS jwt-cpp) target_include_directories(jwt-cpp INTERFACE - ${CMAKE_CURRENT_SOURCE_DIR}/include/jwt-cpp) + ${CMAKE_CURRENT_SOURCE_DIR}/include) diff --git a/dep/jwt-cpp/ClaimTest.cpp b/dep/jwt-cpp/ClaimTest.cpp deleted file mode 100644 index 749edf2ef..000000000 --- a/dep/jwt-cpp/ClaimTest.cpp +++ /dev/null @@ -1,33 +0,0 @@ -#include -#include "include/jwt-cpp/jwt.h" - -TEST(ClaimTest, AudienceAsString) { - std::string token = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJ0ZXN0In0.WZnM3SIiSRHsbO3O7Z2bmIzTJ4EC32HRBKfLznHhrh4"; - auto decoded = jwt::decode(token); - - ASSERT_TRUE(decoded.has_algorithm()); - ASSERT_TRUE(decoded.has_type()); - ASSERT_FALSE(decoded.has_content_type()); - ASSERT_FALSE(decoded.has_key_id()); - ASSERT_FALSE(decoded.has_issuer()); - ASSERT_FALSE(decoded.has_subject()); - ASSERT_TRUE(decoded.has_audience()); - ASSERT_FALSE(decoded.has_expires_at()); - ASSERT_FALSE(decoded.has_not_before()); - ASSERT_FALSE(decoded.has_issued_at()); - ASSERT_FALSE(decoded.has_id()); - - ASSERT_EQ("HS256", decoded.get_algorithm()); - ASSERT_EQ("JWT", decoded.get_type()); - auto aud = decoded.get_audience(); - ASSERT_EQ(1, aud.size()); - ASSERT_EQ("test", *aud.begin()); -} - -TEST(ClaimTest, SetAudienceAsString) { - auto token = jwt::create() - .set_type("JWT") - .set_audience("test") - .sign(jwt::algorithm::hs256("test")); - ASSERT_EQ("eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJ0ZXN0In0.ny5Fa0vzAg7tNL95KWg_ecBNd3XP3tdAzq0SFA6diY4", token); -} diff --git a/dep/jwt-cpp/Doxyfile b/dep/jwt-cpp/Doxyfile index e3303d2b9..0e912fd79 100644 --- a/dep/jwt-cpp/Doxyfile +++ b/dep/jwt-cpp/Doxyfile @@ -38,7 +38,7 @@ PROJECT_NAME = "JWT-C++" # could be handy for archiving the generated documentation or if some version # control system is used. -PROJECT_NUMBER = +PROJECT_NUMBER = 0.5.0 # Using the PROJECT_BRIEF tag one can provide an optional one line description # for a project that appears at the top of each page and should give viewer a @@ -889,7 +889,7 @@ EXCLUDE_SYMLINKS = NO # Note that the wildcards are matched against the file with absolute path, so to # exclude all test directories for example use the pattern */test/* -EXCLUDE_PATTERNS = +EXCLUDE_PATTERNS = *nlohmann*, *picojson* # The EXCLUDE_SYMBOLS tag can be used to specify one or more symbol names # (namespaces, classes, functions, etc.) that should be excluded from the @@ -900,13 +900,13 @@ EXCLUDE_PATTERNS = # Note that the wildcards are matched against the file with absolute path, so to # exclude all test directories use the pattern */test/* -EXCLUDE_SYMBOLS = +EXCLUDE_SYMBOLS = jwt::details # The EXAMPLE_PATH tag can be used to specify one or more files or directories # that contain example code fragments that are included (see the \include # command). -EXAMPLE_PATH = +EXAMPLE_PATH = example # If the value of the EXAMPLE_PATH tag contains directories, you can use the # EXAMPLE_PATTERNS tag to specify one or more wildcard pattern (like *.cpp and @@ -1666,7 +1666,7 @@ EXTRA_SEARCH_MAPPINGS = # If the GENERATE_LATEX tag is set to YES, doxygen will generate LaTeX output. # The default value is: YES. -GENERATE_LATEX = YES +GENERATE_LATEX = NO # The LATEX_OUTPUT tag is used to specify where the LaTeX docs will be put. If a # relative path is entered the value of OUTPUT_DIRECTORY will be put in front of diff --git a/dep/jwt-cpp/HelperTest.cpp b/dep/jwt-cpp/HelperTest.cpp deleted file mode 100644 index f998e1289..000000000 --- a/dep/jwt-cpp/HelperTest.cpp +++ /dev/null @@ -1,55 +0,0 @@ -#include -#include "include/jwt-cpp/jwt.h" - -namespace { - extern std::string google_cert; - extern std::string google_cert_key; -} - -TEST(HelperTest, Cert2Pubkey) { - auto key = jwt::helper::extract_pubkey_from_cert(google_cert); - ASSERT_EQ(google_cert_key, key); -} - -namespace { - std::string google_cert = R"(-----BEGIN CERTIFICATE----- -MIIF8DCCBVmgAwIBAgIKYFOB9QABAACIvTANBgkqhkiG9w0BAQUFADBGMQswCQYD -VQQGEwJVUzETMBEGA1UEChMKR29vZ2xlIEluYzEiMCAGA1UEAxMZR29vZ2xlIElu -dGVybmV0IEF1dGhvcml0eTAeFw0xMzA1MjIxNTQ5MDRaFw0xMzEwMzEyMzU5NTla -MGYxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQHEw1N -b3VudGFpbiBWaWV3MRMwEQYDVQQKEwpHb29nbGUgSW5jMRUwEwYDVQQDFAwqLmdv -b2dsZS5jb20wWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAARmSpIUbCqhUBq1UwnR -Ai7/TNSk6W8JmasR+I0r/NLDYv5yApbAz8HXXN8hDdurMRP6Jy1Q0UIKmyls8HPH -exoCo4IECjCCBAYwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMAsGA1Ud -DwQEAwIHgDAdBgNVHQ4EFgQUU3jT0NVNRgU5ZinRHGrlyoGEnoYwHwYDVR0jBBgw -FoAUv8Aw6/VDET5nup6R+/xq2uNrEiQwWwYDVR0fBFQwUjBQoE6gTIZKaHR0cDov -L3d3dy5nc3RhdGljLmNvbS9Hb29nbGVJbnRlcm5ldEF1dGhvcml0eS9Hb29nbGVJ -bnRlcm5ldEF1dGhvcml0eS5jcmwwZgYIKwYBBQUHAQEEWjBYMFYGCCsGAQUFBzAC -hkpodHRwOi8vd3d3LmdzdGF0aWMuY29tL0dvb2dsZUludGVybmV0QXV0aG9yaXR5 -L0dvb2dsZUludGVybmV0QXV0aG9yaXR5LmNydDAMBgNVHRMBAf8EAjAAMIICwwYD -VR0RBIICujCCAraCDCouZ29vZ2xlLmNvbYINKi5hbmRyb2lkLmNvbYIWKi5hcHBl -bmdpbmUuZ29vZ2xlLmNvbYISKi5jbG91ZC5nb29nbGUuY29tghYqLmdvb2dsZS1h -bmFseXRpY3MuY29tggsqLmdvb2dsZS5jYYILKi5nb29nbGUuY2yCDiouZ29vZ2xl -LmNvLmlugg4qLmdvb2dsZS5jby5qcIIOKi5nb29nbGUuY28udWuCDyouZ29vZ2xl -LmNvbS5hcoIPKi5nb29nbGUuY29tLmF1gg8qLmdvb2dsZS5jb20uYnKCDyouZ29v -Z2xlLmNvbS5jb4IPKi5nb29nbGUuY29tLm14gg8qLmdvb2dsZS5jb20udHKCDyou -Z29vZ2xlLmNvbS52boILKi5nb29nbGUuZGWCCyouZ29vZ2xlLmVzggsqLmdvb2ds -ZS5mcoILKi5nb29nbGUuaHWCCyouZ29vZ2xlLml0ggsqLmdvb2dsZS5ubIILKi5n -b29nbGUucGyCCyouZ29vZ2xlLnB0gg8qLmdvb2dsZWFwaXMuY26CFCouZ29vZ2xl -Y29tbWVyY2UuY29tgg0qLmdzdGF0aWMuY29tggwqLnVyY2hpbi5jb22CECoudXJs -Lmdvb2dsZS5jb22CFioueW91dHViZS1ub2Nvb2tpZS5jb22CDSoueW91dHViZS5j -b22CFioueW91dHViZWVkdWNhdGlvbi5jb22CCyoueXRpbWcuY29tggthbmRyb2lk -LmNvbYIEZy5jb4IGZ29vLmdsghRnb29nbGUtYW5hbHl0aWNzLmNvbYIKZ29vZ2xl -LmNvbYISZ29vZ2xlY29tbWVyY2UuY29tggp1cmNoaW4uY29tggh5b3V0dS5iZYIL -eW91dHViZS5jb22CFHlvdXR1YmVlZHVjYXRpb24uY29tMA0GCSqGSIb3DQEBBQUA -A4GBAAMn0K3j3yhC+X+uyh6eABa2Eq7xiY5/mUB886Ir19vxluSMNKD6n/iY8vHj -trn0BhuW8/vmJyudFkIcEDUYE4ivQMlsfIL7SOGw6OevVLmm02aiRHWj5T20Ds+S -OpueYUG3NBcHP/5IzhUYIQJbGzlQaUaZBMaQeC8ZslMNLWI2 ------END CERTIFICATE-----)"; - - std::string google_cert_key = R"(-----BEGIN PUBLIC KEY----- -MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEZkqSFGwqoVAatVMJ0QIu/0zUpOlv -CZmrEfiNK/zSw2L+cgKWwM/B11zfIQ3bqzET+ictUNFCCpspbPBzx3saAg== ------END PUBLIC KEY----- -)"; -} \ No newline at end of file diff --git a/dep/jwt-cpp/README.md b/dep/jwt-cpp/README.md index 7636f9f67..5e3903262 100644 --- a/dep/jwt-cpp/README.md +++ b/dep/jwt-cpp/README.md @@ -1,98 +1,208 @@ -# jwt-cpp +# ![logo](https://raw.githubusercontent.com/Thalhammer/jwt-cpp/master/.github/logo.svg) +[![License Badge](https://img.shields.io/github/license/Thalhammer/jwt-cpp)](https://github.com/Thalhammer/jwt-cpp/blob/master/LICENSE) [![Codacy Badge](https://api.codacy.com/project/badge/Grade/5f7055e294744901991fd0a1620b231d)](https://app.codacy.com/app/Thalhammer/jwt-cpp?utm_source=github.com&utm_medium=referral&utm_content=Thalhammer/jwt-cpp&utm_campaign=Badge_Grade_Settings) +[![Linux Badge][Linux]][Cross-Platform] +[![MacOS Badge][MacOS]][Cross-Platform] +[![Windows Badge][Windows]][Cross-Platform] +[![Coverage Status](https://coveralls.io/repos/github/Thalhammer/jwt-cpp/badge.svg?branch=master)](https://coveralls.io/github/Thalhammer/jwt-cpp?branch=master) +[![Documentation Badge](https://img.shields.io/badge/Documentation-master-blue)](https://thalhammer.github.io/jwt-cpp/) +[![GitHub release (latest SemVer including pre-releases)](https://img.shields.io/github/v/release/Thalhammer/jwt-cpp?include_prereleases)](https://github.com/Thalhammer/jwt-cpp/releases) +[![Stars Badge](https://img.shields.io/github/stars/Thalhammer/jwt-cpp)](https://github.com/Thalhammer/jwt-cpp/stargazers) -A header only library for creating and validating json web tokens in c++. +[Linux]: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/Thalhammer/jwt-cpp/badges/cross-platform/ubuntu-latest/shields.json +[MacOS]: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/Thalhammer/jwt-cpp/badges/cross-platform/macos-latest/shields.json +[Windows]: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/Thalhammer/jwt-cpp/badges/cross-platform/windows-latest/shields.json +[Cross-Platform]: https://github.com/Thalhammer/jwt-cpp/actions?query=workflow%3A%22Cross-Platform+CI%22 + +A header only library for creating and validating [JSON Web Tokens](https://tools.ietf.org/html/rfc7519) in C++11. For a great introduction, [read this](https://jwt.io/introduction/). ## Signature algorithms -As of version 0.2.0 jwt-cpp supports all algorithms defined by the spec. The modular design of jwt-cpp allows one to add additional algorithms without any problems. If you need any feel free to open a pull request. -For the sake of completeness, here is a list of all supported algorithms: -* HS256 -* HS384 -* HS512 -* RS256 -* RS384 -* RS512 -* ES256 -* ES384 -* ES512 -* PS256 -* PS384 -* PS512 -## Examples -Simple example of decoding a token and printing all claims: -```c++ +jwt-cpp supports all the algorithms defined by the specifications. The modular design allows to easily add additional algorithms without any problems. If you need any feel free to create a pull request or [open an issue](https://github.com/Thalhammer/jwt-cpp/issues/new). + +For completeness, here is a list of all supported algorithms: + +| HMSC | RSA | ECDSA | PSS | EdDSA | +| ----- | ----- | ----- | ----- | ------- | +| HS256 | RS256 | ES256 | PS256 | Ed25519 | +| HS384 | RS384 | ES384 | PS384 | Ed448 | +| HS512 | RS512 | ES512 | PS512 | | + +## SSL Compatibility + +In the name of flexibility and extensibility, jwt-cpp supports both [OpenSSL](https://github.com/openssl/openssl) and [LibreSSL](https://github.com/libressl-portable/portable). These are the version which are, or have been, tested: + +| OpenSSL | LibreSSL | +| -------------- | --------------- | +| [1.0.2][1.0.2] | ![3.1.5][3.1] | +| 1.1.0 | ![3.2.3][3.2] | +| [1.1.1][1.1.1] | ![3.3.1][3.3] | + +[1.0.2]: https://travis-ci.com/github/Thalhammer/jwt-cpp +[1.1.1]: https://github.com/Thalhammer/jwt-cpp/actions?query=workflow%3A%22Coverage+CI%22 +[3.1]: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/Thalhammer/jwt-cpp/badges/libressl/3.1.5/shields.json +[3.2]: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/Thalhammer/jwt-cpp/badges/libressl/3.2.3/shields.json +[3.3]: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/Thalhammer/jwt-cpp/badges/libressl/3.3.1/shields.json + +## Overview + +There is no hard dependency on a JSON library. Instead, there's a generic `jwt::basic_claim` which is templated around type traits, which described the semantic [JSON types](https://json-schema.org/understanding-json-schema/reference/type.html) for a value, object, array, string, number, integer and boolean, as well as methods to translate between them. + +```cpp +jwt::basic_claim claim(json::object({{"json", true},{"example", 0}})); +``` + +This allows for complete freedom when picking which libraries you want to use. For more information, [see below](#providing-your-own-json-traits-your-traits). + +In order to maintain compatibility, [picojson](https://github.com/kazuho/picojson) is still used to provide a specialized `jwt::claim` along with all helpers. Defining `JWT_DISABLE_PICOJSON` will remove this optional dependency. + +As for the base64 requirements of JWTs, this libary provides `base.h` with all the required implentation; However base64 implementations are very common, with varying degrees of performance. When providing your own base64 implementation, you can define `JWT_DISABLE_BASE64` to remove the jwt-cpp implementation. + +### Getting Started + +Simple example of decoding a token and printing all [claims](https://tools.ietf.org/html/rfc7519#section-4) ([try it out](https://github.com/Thalhammer/jwt-cpp/tree/master/example/print-claims.cpp)): + +```cpp #include #include -int main(int argc, const char** argv) { - std::string token = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXUyJ9.eyJpc3MiOiJhdXRoMCJ9.AbIJTDMFc7yUa5MhvcP03nJPyCPzZtQcGEp-zWfOkEE"; - auto decoded = jwt::decode(token); +int main() { + std::string token = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXUyJ9.eyJpc3MiOiJhdXRoMCJ9.AbIJTDMFc7yUa5MhvcP03nJPyCPzZtQcGEp-zWfOkEE"; + auto decoded = jwt::decode(token); - for(auto& e : decoded.get_payload_claims()) - std::cout << e.first << " = " << e.second.to_json() << std::endl; + for(auto& e : decoded.get_payload_claims()) + std::cout << e.first << " = " << e.second << std::endl; } ``` In order to verify a token you first build a verifier and use it to verify a decoded token. -```c++ + +```cpp auto verifier = jwt::verify() - .allow_algorithm(jwt::algorithm::hs256{ "secret" }) - .with_issuer("auth0"); + .allow_algorithm(jwt::algorithm::hs256{ "secret" }) + .with_issuer("auth0"); verifier.verify(decoded_token); ``` + The created verifier is stateless so you can reuse it for different tokens. -Creating a token (and signing) is equally easy. -```c++ +Creating a token (and signing) is equally as easy. + +```cpp auto token = jwt::create() - .set_issuer("auth0") - .set_type("JWS") - .set_payload_claim("sample", std::string("test")) - .sign(jwt::algorithm::hs256{"secret"}); + .set_issuer("auth0") + .set_type("JWS") + .set_payload_claim("sample", jwt::claim(std::string("test"))) + .sign(jwt::algorithm::hs256{"secret"}); ``` -Here is a simple example of creating a token that will expire in 2 hours: +Here is a simple example of creating a token that will expire in one hour: -```c++ +```cpp +auto token = jwt::create() + .set_issuer("auth0") + .set_issued_at(std::chrono::system_clock::now()) + .set_expires_at(std::chrono::system_clock::now() + std::chrono::seconds{3600}) + .sign(jwt::algorithm::hs256{"secret"}); +``` - // Note to @Thalhammer: please replace with a better example if this is not a good way - auto token = jwt::create() - .set_issuer("auth0") - .set_issued_at(jwt::date(std::chrono::system_clock::now())) - .set_expires_at(jwt::date(std::chrono::system_clock::now()+ std::chrono::seconds{3600})) - .sign(jwt::algorithm::hs256{"secret"} +> To see more examples working with RSA public and private keys, visit our [examples](https://github.com/Thalhammer/jwt-cpp/tree/master/example)! +### Providing your own JSON Traits +There are several key items that need to be provided to a `jwt::basic_claim` in order for it to be interoptable with you JSON library of choice. + +* type specifications +* conversion from generic "value type" to a specific type +* serialization and parsing + +If ever you are not sure, the traits are heavily checked against static asserts to make sure you provide everything that's required. + +> :warning: Not all JSON libraries are a like, you may need to extent certain types such that it can be used by jwt-cpp. See this [example](https://github.com/Thalhammer/jwt-cpp/blob/ac3de9e69bc698a464dacb256a1b50512843f092/tests/jsoncons/JsonconsTest.cpp). + +```cpp +struct my_favorite_json_library_traits { + // Type Specifications + using value_type = json; // The generic "value type" implementation, most libraries have one + using object_type = json::object_t; // The "map type" string to value + using array_type = json::array_t; // The "list type" array of values + using string_type = std::string; // The "list of chars", must be a narrow char + using number_type = double; // The "percision type" + using integer_type = int64_t; // The "integral type" + using boolean_type = bool; // The "boolean type" + + // Translation between the implementation notion of type, to the jwt::json::type equivilant + static jwt::json::type get_type(const value_type &val) { + using jwt::json::type; + + if (val.type() == json::value_t::object) + return type::object; + if (val.type() == json::value_t::array) + return type::array; + if (val.type() == json::value_t::string) + return type::string; + if (val.type() == json::value_t::number_float) + return type::number; + if (val.type() == json::value_t::number_integer) + return type::integer; + if (val.type() == json::value_t::boolean) + return type::boolean; + + throw std::logic_error("invalid type"); + } + + // Conversion from generic value to specific type + static object_type as_object(const value_type &val); + static array_type as_array(const value_type &val); + static string_type as_string(const value_type &val); + static number_type as_number(const value_type &val); + static integer_type as_int(const value_type &val); + static boolean_type as_bool(const value_type &val); + + // serilization and parsing + static bool parse(value_type &val, string_type str); + static string_type serialize(const value_type &val); // with no extra whitespace, padding or indentation +}; ``` ## Contributing + If you have an improvement or found a bug feel free to [open an issue](https://github.com/Thalhammer/jwt-cpp/issues/new) or add the change and create a pull request. If you file a bug please make sure to include as much information about your environment (compiler version, etc.) as possible to help reproduce the issue. If you add a new feature please make sure to also include test cases for it. ## Dependencies + In order to use jwt-cpp you need the following tools. + * libcrypto (openssl or compatible) * libssl-dev (for the header files) * a compiler supporting at least c++11 * basic stl support In order to build the test cases you also need -* gtest installed in linker path + +* gtest * pthread ## Troubleshooting -#### Expired tokens -If you are generating tokens that seem to immediately expire, you are likely not using UTC. Specifically, -if you use `get_time` to get the current time, it likely uses localtime, while this library uses UTC, which may be why your token is immediately expiring. Please see example above on the right way to use current time. -#### Missing _HMAC amd _EVP_sha256 symbols on Mac +### Expired tokens + +If you are generating tokens that seem to immediately expire, you are likely not using UTC. Specifically, +if you use `get_time` to get the current time, it likely uses localtime, while this library uses UTC, +which may be why your token is immediately expiring. Please see example above on the right way to use current time. + +### Missing \_HMAC and \_EVP_sha256 symbols on Mac + There seems to exists a problem with the included openssl library of MacOS. Make sure you link to one provided by brew. See [here](https://github.com/Thalhammer/jwt-cpp/issues/6) for more details. -#### Building on windows fails with syntax errors -The header "Windows.h", which is often included in windowsprojects, defines macros for MIN and MAX which screw up std::numeric_limits. + +### Building on windows fails with syntax errors + +The header ``, which is often included in windowsprojects, defines macros for MIN and MAX which screw up std::numeric_limits. See [here](https://github.com/Thalhammer/jwt-cpp/issues/5) for more details. To fix this do one of the following things: + * define NOMINMAX, which suppresses this behaviour * include this library before you include windows.h -* place ```#undef max``` and ```#undef min``` before you include this library +* place `#undef max` and `#undef min` before you include this library diff --git a/dep/jwt-cpp/TestMain.cpp b/dep/jwt-cpp/TestMain.cpp deleted file mode 100644 index b8b7262bc..000000000 --- a/dep/jwt-cpp/TestMain.cpp +++ /dev/null @@ -1,7 +0,0 @@ -#include - -int main(int argc, char *argv[]) -{ - ::testing::InitGoogleTest(&argc, argv); - return RUN_ALL_TESTS(); -} \ No newline at end of file diff --git a/dep/jwt-cpp/TokenFormatTest.cpp b/dep/jwt-cpp/TokenFormatTest.cpp deleted file mode 100644 index e670a82c8..000000000 --- a/dep/jwt-cpp/TokenFormatTest.cpp +++ /dev/null @@ -1,15 +0,0 @@ -#include -#include "include/jwt-cpp/jwt.h" - -TEST(TokenFormatTest, MissingDot) { - ASSERT_THROW(jwt::decode("eyJhbGciOiJub25lIiwidHlwIjoiSldTIn0.eyJpc3MiOiJhdXRoMCJ9"), std::invalid_argument); - ASSERT_THROW(jwt::decode("eyJhbGciOiJub25lIiwidHlwIjoiSldTIn0eyJpc3MiOiJhdXRoMCJ9."), std::invalid_argument); -} - -TEST(TokenFormatTest, InvalidChar) { - ASSERT_THROW(jwt::decode("eyJhbGciOiJub25lIiwidHlwIjoiSldTIn0().eyJpc3MiOiJhdXRoMCJ9."), std::runtime_error); -} - -TEST(TokenFormatTest, InvalidJSON) { - ASSERT_THROW(jwt::decode("YXsiYWxnIjoibm9uZSIsInR5cCI6IkpXUyJ9YQ.eyJpc3MiOiJhdXRoMCJ9."), std::runtime_error); -} \ No newline at end of file diff --git a/dep/jwt-cpp/TokenTest.cpp b/dep/jwt-cpp/TokenTest.cpp deleted file mode 100644 index 6d2d004c3..000000000 --- a/dep/jwt-cpp/TokenTest.cpp +++ /dev/null @@ -1,420 +0,0 @@ -#include -#include "include/jwt-cpp/jwt.h" - -namespace { - extern std::string rsa_priv_key; - extern std::string rsa_pub_key; - extern std::string rsa_pub_key_invalid; - extern std::string rsa512_priv_key; - extern std::string rsa512_pub_key; - extern std::string rsa512_pub_key_invalid; - extern std::string ecdsa_priv_key; - extern std::string ecdsa_pub_key; - extern std::string ecdsa_pub_key_invalid; -} - -TEST(TokenTest, DecodeToken) { - std::string token = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXUyJ9.eyJpc3MiOiJhdXRoMCJ9.AbIJTDMFc7yUa5MhvcP03nJPyCPzZtQcGEp-zWfOkEE"; - auto decoded = jwt::decode(token); - - ASSERT_TRUE(decoded.has_algorithm()); - ASSERT_TRUE(decoded.has_type()); - ASSERT_FALSE(decoded.has_content_type()); - ASSERT_FALSE(decoded.has_key_id()); - ASSERT_TRUE(decoded.has_issuer()); - ASSERT_FALSE(decoded.has_subject()); - ASSERT_FALSE(decoded.has_audience()); - ASSERT_FALSE(decoded.has_expires_at()); - ASSERT_FALSE(decoded.has_not_before()); - ASSERT_FALSE(decoded.has_issued_at()); - ASSERT_FALSE(decoded.has_id()); - - ASSERT_EQ("HS256", decoded.get_algorithm()); - ASSERT_EQ("JWS", decoded.get_type()); - ASSERT_EQ("auth0", decoded.get_issuer()); -} - -TEST(TokenTest, CreateToken) { - auto token = jwt::create() - .set_issuer("auth0") - .set_type("JWS") - .sign(jwt::algorithm::none{}); - ASSERT_EQ("eyJhbGciOiJub25lIiwidHlwIjoiSldTIn0.eyJpc3MiOiJhdXRoMCJ9.", token); -} - -TEST(TokenTest, CreateTokenHS256) { - auto token = jwt::create() - .set_issuer("auth0") - .set_type("JWS") - .sign(jwt::algorithm::hs256{"secret"}); - ASSERT_EQ("eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXUyJ9.eyJpc3MiOiJhdXRoMCJ9.AbIJTDMFc7yUa5MhvcP03nJPyCPzZtQcGEp-zWfOkEE", token); -} - -TEST(TokenTest, CreateTokenRS256) { - auto token = jwt::create() - .set_issuer("auth0") - .set_type("JWS") - .sign(jwt::algorithm::rs256(rsa_pub_key, rsa_priv_key, "", "")); - - ASSERT_EQ( - "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXUyJ9.eyJpc3MiOiJhdXRoMCJ9.VA2i1ui1cnoD6I3wnji1WAVCf29EekysvevGrT2GXqK1dDMc8" - "HAZCTQxa1Q8NppnpYV-hlqxh-X3Bb0JOePTGzjynpNZoJh2aHZD-GKpZt7OO1Zp8AFWPZ3p8Cahq8536fD8RiBES9jRsvChZvOqA7gMcFc4" - "YD0iZhNIcI7a654u5yPYyTlf5kjR97prCf_OXWRn-bYY74zna4p_bP9oWCL4BkaoRcMxi-IR7kmVcCnvbYqyIrKloXP2qPO442RBGqU7Ov9" - "sGQxiVqtRHKXZR9RbfvjrErY1KGiCp9M5i2bsUHadZEY44FE2jiOmx-uc2z5c05CCXqVSpfCjWbh9gQ", token); -} - -TEST(TokenTest, CreateTokenRS512) { - auto token = jwt::create() - .set_issuer("auth0") - .set_type("JWS") - .sign(jwt::algorithm::rs512(rsa512_pub_key, rsa512_priv_key, "", "")); - - ASSERT_EQ( - "eyJhbGciOiJSUzUxMiIsInR5cCI6IkpXUyJ9.eyJpc3MiOiJhdXRoMCJ9.GZhnjtsvBl2_KDSxg4JW6xnmNjr2mWhYSZSSQyLKvI0" - "TK86sJKchkt_HDy2IC5l5BGRhq_Xv9pHdA1umidQZG3a7gWvHsujqybCBgBraMTd1wJrCl4QxFg2RYHhHbRqb9BnPJgFD_vryd4GB" - "hfGgejPBCBlGrQtqFGFdHHOjNHY", token); -} - -TEST(TokenTest, CreateTokenPS256) { - auto token = jwt::create() - .set_issuer("auth0") - .set_type("JWS") - .sign(jwt::algorithm::ps256(rsa_pub_key, rsa_priv_key, "", "")); - - // TODO: Find a better way to check if generated signature is valid - // Can't do simple check for equal since pss adds random salt. -} - -TEST(TokenTest, CreateTokenPS384) { - auto token = jwt::create() - .set_issuer("auth0") - .set_type("JWS") - .sign(jwt::algorithm::ps384(rsa_pub_key, rsa_priv_key, "", "")); - - // TODO: Find a better way to check if generated signature is valid - // Can't do simple check for equal since pss adds random salt. -} - -TEST(TokenTest, CreateTokenPS512) { - auto token = jwt::create() - .set_issuer("auth0") - .set_type("JWS") - .sign(jwt::algorithm::ps512(rsa_pub_key, rsa_priv_key, "", "")); - - // TODO: Find a better way to check if generated signature is valid - // Can't do simple check for equal since pss adds random salt. -} - -TEST(TokenTest, CreateTokenES256) { - - auto token = jwt::create() - .set_issuer("auth0") - .set_type("JWS") - .sign(jwt::algorithm::es256("", ecdsa_priv_key, "", "")); - - auto decoded = jwt::decode(token); - - ASSERT_THROW(jwt::verify().allow_algorithm(jwt::algorithm::es256(ecdsa_pub_key_invalid, "", "", "")).verify(decoded), jwt::signature_verification_exception); - ASSERT_NO_THROW(jwt::verify().allow_algorithm(jwt::algorithm::es256(ecdsa_pub_key, "", "", "")).verify(decoded)); -} - -TEST(TokenTest, CreateTokenES256NoPrivate) { - - ASSERT_THROW([](){ - auto token = jwt::create() - .set_issuer("auth0") - .set_type("JWS") - .sign(jwt::algorithm::es256(ecdsa_pub_key, "", "", "")); - }(), jwt::signature_generation_exception); -} - -TEST(TokenTest, VerifyTokenRS256) { - std::string token = "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXUyJ9.eyJpc3MiOiJhdXRoMCJ9.VA2i1ui1cnoD6I3wnji1WAVCf29EekysvevGrT2GXqK1dDMc8" - "HAZCTQxa1Q8NppnpYV-hlqxh-X3Bb0JOePTGzjynpNZoJh2aHZD-GKpZt7OO1Zp8AFWPZ3p8Cahq8536fD8RiBES9jRsvChZvOqA7gMcFc4" - "YD0iZhNIcI7a654u5yPYyTlf5kjR97prCf_OXWRn-bYY74zna4p_bP9oWCL4BkaoRcMxi-IR7kmVcCnvbYqyIrKloXP2qPO442RBGqU7Ov9" - "sGQxiVqtRHKXZR9RbfvjrErY1KGiCp9M5i2bsUHadZEY44FE2jiOmx-uc2z5c05CCXqVSpfCjWbh9gQ"; - - auto verify = jwt::verify() - .allow_algorithm(jwt::algorithm::rs256(rsa_pub_key, rsa_priv_key, "", "")) - .with_issuer("auth0"); - - auto decoded_token = jwt::decode(token); - - verify.verify(decoded_token); -} - -TEST(TokenTest, VerifyTokenRS256PublicOnly) { - std::string token = "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXUyJ9.eyJpc3MiOiJhdXRoMCJ9.VA2i1ui1cnoD6I3wnji1WAVCf29EekysvevGrT2GXqK1dDMc8" - "HAZCTQxa1Q8NppnpYV-hlqxh-X3Bb0JOePTGzjynpNZoJh2aHZD-GKpZt7OO1Zp8AFWPZ3p8Cahq8536fD8RiBES9jRsvChZvOqA7gMcFc4" - "YD0iZhNIcI7a654u5yPYyTlf5kjR97prCf_OXWRn-bYY74zna4p_bP9oWCL4BkaoRcMxi-IR7kmVcCnvbYqyIrKloXP2qPO442RBGqU7Ov9" - "sGQxiVqtRHKXZR9RbfvjrErY1KGiCp9M5i2bsUHadZEY44FE2jiOmx-uc2z5c05CCXqVSpfCjWbh9gQ"; - - auto verify = jwt::verify() - .allow_algorithm(jwt::algorithm::rs256(rsa_pub_key, "", "", "")) - .with_issuer("auth0"); - - auto decoded_token = jwt::decode(token); - - verify.verify(decoded_token); -} - -TEST(TokenTest, VerifyTokenRS256Fail) { - std::string token = "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXUyJ9.eyJpc3MiOiJhdXRoMCJ9.VA2i1ui1cnoD6I3wnji1WAVCf29EekysvevGrT2GXqK1dDMc8" - "HAZCTQxa1Q8NppnpYV-hlqxh-X3Bb0JOePTGzjynpNZoJh2aHZD-GKpZt7OO1Zp8AFWPZ3p8Cahq8536fD8RiBES9jRsvChZvOqA7gMcFc4" - "YD0iZhNIcI7a654u5yPYyTlf5kjR97prCf_OXWRn-bYY74zna4p_bP9oWCL4BkaoRcMxi-IR7kmVcCnvbYqyIrKloXP2qPO442RBGqU7Ov9" - "sGQxiVqtRHKXZR9RbfvjrErY1KGiCp9M5i2bsUHadZEY44FE2jiOmx-uc2z5c05CCXqVSpfCjWbh9gQ"; - - auto verify = jwt::verify() - .allow_algorithm(jwt::algorithm::rs256(rsa_pub_key_invalid, "", "", "")) - .with_issuer("auth0"); - - auto decoded_token = jwt::decode(token); - - ASSERT_THROW(verify.verify(decoded_token), jwt::signature_verification_exception); -} - -TEST(TokenTest, VerifyTokenRS512) { - std::string token = "eyJhbGciOiJSUzUxMiIsInR5cCI6IkpXUyJ9.eyJpc3MiOiJhdXRoMCJ9.GZhnjtsvBl2_KDSxg4JW6xnmNjr2mWhYSZ" - "SSQyLKvI0TK86sJKchkt_HDy2IC5l5BGRhq_Xv9pHdA1umidQZG3a7gWvHsujqybCBgBraMTd1wJrCl4QxFg2RYHhHbRqb9BnPJgFD_vryd4" - "GBhfGgejPBCBlGrQtqFGFdHHOjNHY"; - - auto verify = jwt::verify() - .allow_algorithm(jwt::algorithm::rs512(rsa512_pub_key, rsa512_priv_key, "", "")) - .with_issuer("auth0"); - - auto decoded_token = jwt::decode(token); - - verify.verify(decoded_token); -} - -TEST(TokenTest, VerifyTokenRS512PublicOnly) { - std::string token = "eyJhbGciOiJSUzUxMiIsInR5cCI6IkpXUyJ9.eyJpc3MiOiJhdXRoMCJ9.GZhnjtsvBl2_KDSxg4JW6xnmNjr2mWhYSZ" - "SSQyLKvI0TK86sJKchkt_HDy2IC5l5BGRhq_Xv9pHdA1umidQZG3a7gWvHsujqybCBgBraMTd1wJrCl4QxFg2RYHhHbRqb9BnPJgFD_vryd4" - "GBhfGgejPBCBlGrQtqFGFdHHOjNHY"; - - auto verify = jwt::verify() - .allow_algorithm(jwt::algorithm::rs512(rsa512_pub_key, "", "", "")) - .with_issuer("auth0"); - - auto decoded_token = jwt::decode(token); - - verify.verify(decoded_token); -} - -TEST(TokenTest, VerifyTokenRS512Fail) { - std::string token = "eyJhbGciOiJSUzUxMiIsInR5cCI6IkpXUyJ9.eyJpc3MiOiJhdXRoMCJ9.GZhnjtsvBl2_KDSxg4JW6xnmNjr2mWhYSZ" - "SSQyLKvI0TK86sJKchkt_HDy2IC5l5BGRhq_Xv9pHdA1umidQZG3a7gWvHsujqybCBgBraMTd1wJrCl4QxFg2RYHhHbRqb9BnPJgFD_vryd4" - "GBhfGgejPBCBlGrQtqFGFdHHOjNHY"; - - auto verify = jwt::verify() - .allow_algorithm(jwt::algorithm::rs512(rsa_pub_key_invalid, "", "", "")) - .with_issuer("auth0"); - - auto decoded_token = jwt::decode(token); - - ASSERT_THROW(verify.verify(decoded_token), jwt::signature_verification_exception); -} - -TEST(TokenTest, VerifyTokenHS256) { - std::string token = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXUyJ9.eyJpc3MiOiJhdXRoMCJ9.AbIJTDMFc7yUa5MhvcP03nJPyCPzZtQcGEp-zWfOkEE"; - - auto verify = jwt::verify() - .allow_algorithm(jwt::algorithm::hs256{ "secret" }) - .with_issuer("auth0"); - - auto decoded_token = jwt::decode(token); - verify.verify(decoded_token); -} - -TEST(TokenTest, VerifyFail) { - auto token = jwt::create() - .set_issuer("auth0") - .set_type("JWS") - .sign(jwt::algorithm::none{}); - - auto decoded_token = jwt::decode(token); - - { - auto verify = jwt::verify() - .allow_algorithm(jwt::algorithm::none{}) - .with_issuer("auth"); - ASSERT_THROW(verify.verify(decoded_token), jwt::token_verification_exception); - } - { - auto verify = jwt::verify() - .allow_algorithm(jwt::algorithm::none{}) - .with_issuer("auth0") - .with_audience({ "test" }); - ASSERT_THROW(verify.verify(decoded_token), jwt::token_verification_exception); - } - { - auto verify = jwt::verify() - .allow_algorithm(jwt::algorithm::none{}) - .with_issuer("auth0") - .with_subject("test"); - ASSERT_THROW(verify.verify(decoded_token), jwt::token_verification_exception); - } - { - auto verify = jwt::verify() - .allow_algorithm(jwt::algorithm::none{}) - .with_issuer("auth0") - .with_claim("myclaim", jwt::claim(std::string("test"))); - ASSERT_THROW(verify.verify(decoded_token), jwt::token_verification_exception); - } -} - -TEST(TokenTest, VerifyTokenES256) { - const std::string token = "eyJhbGciOiJFUzI1NiJ9.eyJpc3MiOiJhdXRoMCJ9.4iVk3-Y0v4RT4_9IaQlp-8dZ_4fsTzIylgrPTDLrEvTHBTyVS3tgPbr2_IZfLETtiKRqCg0aQ5sh9eIsTTwB1g"; - - auto verify = jwt::verify().allow_algorithm(jwt::algorithm::es256(ecdsa_pub_key, "", "", "")); - auto decoded_token = jwt::decode(token); - - verify.verify(decoded_token); -} - -TEST(TokenTest, VerifyTokenES256Fail) { - const std::string token = "eyJhbGciOiJFUzI1NiJ9.eyJpc3MiOiJhdXRoMCJ9.4iVk3-Y0v4RT4_9IaQlp-8dZ_4fsTzIylgrPTDLrEvTHBTyVS3tgPbr2_IZfLETtiKRqCg0aQ5sh9eIsTTwB1g"; - - auto verify = jwt::verify() - .allow_algorithm(jwt::algorithm::es256(ecdsa_pub_key_invalid, "", "", "")); - auto decoded_token = jwt::decode(token); - - ASSERT_THROW(verify.verify(decoded_token), jwt::signature_verification_exception); -} - -TEST(TokenTest, VerifyTokenPS256) { - std::string token = "eyJhbGciOiJQUzI1NiIsInR5cCI6IkpXUyJ9.eyJpc3MiOiJhdXRoMCJ9.CJ4XjVWdbV6vXGZkD4GdJbtYc80SN9cmPOqRhZBRzOyDRqTFE" - "4MsbdKyQuhAWcvuMOjn-24qOTjVMR_P_uTC1uG6WPLcucxZyLnbb56zbKnEklW2SX0mQnCGewr-93a_vDaFT6Cp45MsF_OwFPRCMaS5CJg-" - "N5KY67UrVSr3s9nkuK9ZTQkyODHfyEUh9F_FhRCATGrb5G7_qHqBYvTvaPUXqzhhpCjN855Tocg7A24Hl0yMwM-XdasucW5xNdKjG_YCkis" - "HX7ax--JiF5GNYCO61eLFteO4THUg-3Z0r4OlGqlppyWo5X5tjcxOZCvBh7WDWfkxA48KFZPRv0nlKA"; - - auto verify = jwt::verify() - .allow_algorithm(jwt::algorithm::ps256(rsa_pub_key, rsa_priv_key, "", "")) - .with_issuer("auth0"); - - auto decoded_token = jwt::decode(token); - - verify.verify(decoded_token); -} - -TEST(TokenTest, VerifyTokenPS256PublicOnly) { - std::string token = "eyJhbGciOiJQUzI1NiIsInR5cCI6IkpXUyJ9.eyJpc3MiOiJhdXRoMCJ9.CJ4XjVWdbV6vXGZkD4GdJbtYc80SN9cmPOqRhZBRzOyDRqTFE" - "4MsbdKyQuhAWcvuMOjn-24qOTjVMR_P_uTC1uG6WPLcucxZyLnbb56zbKnEklW2SX0mQnCGewr-93a_vDaFT6Cp45MsF_OwFPRCMaS5CJg-" - "N5KY67UrVSr3s9nkuK9ZTQkyODHfyEUh9F_FhRCATGrb5G7_qHqBYvTvaPUXqzhhpCjN855Tocg7A24Hl0yMwM-XdasucW5xNdKjG_YCkis" - "HX7ax--JiF5GNYCO61eLFteO4THUg-3Z0r4OlGqlppyWo5X5tjcxOZCvBh7WDWfkxA48KFZPRv0nlKA"; - - auto verify = jwt::verify() - .allow_algorithm(jwt::algorithm::ps256(rsa_pub_key, "", "", "")) - .with_issuer("auth0"); - - auto decoded_token = jwt::decode(token); - - verify.verify(decoded_token); -} - -TEST(TokenTest, VerifyTokenPS256Fail) { - std::string token = "eyJhbGciOiJQUzI1NiIsInR5cCI6IkpXUyJ9.eyJpc3MiOiJhdXRoMCJ9.CJ4XjVWdbV6vXGZkD4GdJbtYc80SN9cmPOqRhZBRzOyDRqTFE" - "4MsbdKyQuhAWcvuMOjn-24qOTjVMR_P_uTC1uG6WPLcucxZyLnbb56zbKnEklW2SX0mQnCGewr-93a_vDaFT6Cp45MsF_OwFPRCMaS5CJg-" - "N5KY67UrVSr3s9nkuK9ZTQkyODHfyEUh9F_FhRCATGrb5G7_qHqBYvTvaPUXqzhhpCjN855Tocg7A24Hl0yMwM-XdasucW5xNdKjG_YCkis" - "HX7ax--JiF5GNYCO61eLFteO4THUg-3Z0r4OlGqlppyWo5X5tjcxOZCvBh7WDWfkxA48KFZPRv0nlKA"; - - auto verify = jwt::verify() - .allow_algorithm(jwt::algorithm::ps256(rsa_pub_key_invalid, "", "", "")) - .with_issuer("auth0"); - - auto decoded_token = jwt::decode(token); - - ASSERT_THROW(verify.verify(decoded_token), jwt::signature_verification_exception); -} - -namespace { - std::string rsa_priv_key = R"(-----BEGIN PRIVATE KEY----- -MIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQC4ZtdaIrd1BPIJ -tfnF0TjIK5inQAXZ3XlCrUlJdP+XHwIRxdv1FsN12XyMYO/6ymLmo9ryoQeIrsXB -XYqlET3zfAY+diwCb0HEsVvhisthwMU4gZQu6TYW2s9LnXZB5rVtcBK69hcSlA2k -ZudMZWxZcj0L7KMfO2rIvaHw/qaVOE9j0T257Z8Kp2CLF9MUgX0ObhIsdumFRLaL -DvDUmBPr2zuh/34j2XmWwn1yjN/WvGtdfhXW79Ki1S40HcWnygHgLV8sESFKUxxQ -mKvPUTwDOIwLFL5WtE8Mz7N++kgmDcmWMCHc8kcOIu73Ta/3D4imW7VbKgHZo9+K -3ESFE3RjAgMBAAECggEBAJTEIyjMqUT24G2FKiS1TiHvShBkTlQdoR5xvpZMlYbN -tVWxUmrAGqCQ/TIjYnfpnzCDMLhdwT48Ab6mQJw69MfiXwc1PvwX1e9hRscGul36 -ryGPKIVQEBsQG/zc4/L2tZe8ut+qeaK7XuYrPp8bk/X1e9qK5m7j+JpKosNSLgJj -NIbYsBkG2Mlq671irKYj2hVZeaBQmWmZxK4fw0Istz2WfN5nUKUeJhTwpR+JLUg4 -ELYYoB7EO0Cej9UBG30hbgu4RyXA+VbptJ+H042K5QJROUbtnLWuuWosZ5ATldwO -u03dIXL0SH0ao5NcWBzxU4F2sBXZRGP2x/jiSLHcqoECgYEA4qD7mXQpu1b8XO8U -6abpKloJCatSAHzjgdR2eRDRx5PMvloipfwqA77pnbjTUFajqWQgOXsDTCjcdQui -wf5XAaWu+TeAVTytLQbSiTsBhrnoqVrr3RoyDQmdnwHT8aCMouOgcC5thP9vQ8Us -rVdjvRRbnJpg3BeSNimH+u9AHgsCgYEA0EzcbOltCWPHRAY7B3Ge/AKBjBQr86Kv -TdpTlxePBDVIlH+BM6oct2gaSZZoHbqPjbq5v7yf0fKVcXE4bSVgqfDJ/sZQu9Lp -PTeV7wkk0OsAMKk7QukEpPno5q6tOTNnFecpUhVLLlqbfqkB2baYYwLJR3IRzboJ -FQbLY93E8gkCgYB+zlC5VlQbbNqcLXJoImqItgQkkuW5PCgYdwcrSov2ve5r/Acz -FNt1aRdSlx4176R3nXyibQA1Vw+ztiUFowiP9WLoM3PtPZwwe4bGHmwGNHPIfwVG -m+exf9XgKKespYbLhc45tuC08DATnXoYK7O1EnUINSFJRS8cezSI5eHcbQKBgQDC -PgqHXZ2aVftqCc1eAaxaIRQhRmY+CgUjumaczRFGwVFveP9I6Gdi+Kca3DE3F9Pq -PKgejo0SwP5vDT+rOGHN14bmGJUMsX9i4MTmZUZ5s8s3lXh3ysfT+GAhTd6nKrIE -kM3Nh6HWFhROptfc6BNusRh1kX/cspDplK5x8EpJ0QKBgQDWFg6S2je0KtbV5PYe -RultUEe2C0jYMDQx+JYxbPmtcopvZQrFEur3WKVuLy5UAy7EBvwMnZwIG7OOohJb -vkSpADK6VPn9lbqq7O8cTedEHttm6otmLt8ZyEl3hZMaL3hbuRj6ysjmoFKx6CrX -rK0/Ikt5ybqUzKCMJZg2VKGTxg== ------END PRIVATE KEY-----)"; - std::string rsa_pub_key = R"(-----BEGIN PUBLIC KEY----- -MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAuGbXWiK3dQTyCbX5xdE4 -yCuYp0AF2d15Qq1JSXT/lx8CEcXb9RbDddl8jGDv+spi5qPa8qEHiK7FwV2KpRE9 -83wGPnYsAm9BxLFb4YrLYcDFOIGULuk2FtrPS512Qea1bXASuvYXEpQNpGbnTGVs -WXI9C+yjHztqyL2h8P6mlThPY9E9ue2fCqdgixfTFIF9Dm4SLHbphUS2iw7w1JgT -69s7of9+I9l5lsJ9cozf1rxrXX4V1u/SotUuNB3Fp8oB4C1fLBEhSlMcUJirz1E8 -AziMCxS+VrRPDM+zfvpIJg3JljAh3PJHDiLu902v9w+Iplu1WyoB2aPfitxEhRN0 -YwIDAQAB ------END PUBLIC KEY-----)"; - std::string rsa_pub_key_invalid = R"(-----BEGIN PUBLIC KEY----- -MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxzYuc22QSst/dS7geYYK -5l5kLxU0tayNdixkEQ17ix+CUcUbKIsnyftZxaCYT46rQtXgCaYRdJcbB3hmyrOa -vkhTpX79xJZnQmfuamMbZBqitvscxW9zRR9tBUL6vdi/0rpoUwPMEh8+Bw7CgYR0 -FK0DhWYBNDfe9HKcyZEv3max8Cdq18htxjEsdYO0iwzhtKRXomBWTdhD5ykd/fAC -VTr4+KEY+IeLvubHVmLUhbE5NgWXxrRpGasDqzKhCTmsa2Ysf712rl57SlH0Wz/M -r3F7aM9YpErzeYLrl0GhQr9BVJxOvXcVd4kmY+XkiCcrkyS1cnghnllh+LCwQu1s -YwIDAQAB ------END PUBLIC KEY-----)"; - std::string rsa512_priv_key = R"(-----BEGIN RSA PRIVATE KEY----- -MIICWwIBAAKBgQDdlatRjRjogo3WojgGHFHYLugdUWAY9iR3fy4arWNA1KoS8kVw -33cJibXr8bvwUAUparCwlvdbH6dvEOfou0/gCFQsHUfQrSDv+MuSUMAe8jzKE4qW -+jK+xQU9a03GUnKHkkle+Q0pX/g6jXZ7r1/xAK5Do2kQ+X5xK9cipRgEKwIDAQAB -AoGAD+onAtVye4ic7VR7V50DF9bOnwRwNXrARcDhq9LWNRrRGElESYYTQ6EbatXS -3MCyjjX2eMhu/aF5YhXBwkppwxg+EOmXeh+MzL7Zh284OuPbkglAaGhV9bb6/5Cp -uGb1esyPbYW+Ty2PC0GSZfIXkXs76jXAu9TOBvD0ybc2YlkCQQDywg2R/7t3Q2OE -2+yo382CLJdrlSLVROWKwb4tb2PjhY4XAwV8d1vy0RenxTB+K5Mu57uVSTHtrMK0 -GAtFr833AkEA6avx20OHo61Yela/4k5kQDtjEf1N0LfI+BcWZtxsS3jDM3i1Hp0K -Su5rsCPb8acJo5RO26gGVrfAsDcIXKC+bQJAZZ2XIpsitLyPpuiMOvBbzPavd4gY -6Z8KWrfYzJoI/Q9FuBo6rKwl4BFoToD7WIUS+hpkagwWiz+6zLoX1dbOZwJACmH5 -fSSjAkLRi54PKJ8TFUeOP15h9sQzydI8zJU+upvDEKZsZc/UhT/SySDOxQ4G/523 -Y0sz/OZtSWcol/UMgQJALesy++GdvoIDLfJX5GBQpuFgFenRiRDabxrE9MNUZ2aP -FaFp+DyAe+b4nDwuJaW2LURbr8AEZga7oQj0uYxcYw== ------END RSA PRIVATE KEY-----)"; - std::string rsa512_pub_key = R"(-----BEGIN PUBLIC KEY----- -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDdlatRjRjogo3WojgGHFHYLugd -UWAY9iR3fy4arWNA1KoS8kVw33cJibXr8bvwUAUparCwlvdbH6dvEOfou0/gCFQs -HUfQrSDv+MuSUMAe8jzKE4qW+jK+xQU9a03GUnKHkkle+Q0pX/g6jXZ7r1/xAK5D -o2kQ+X5xK9cipRgEKwIDAQAB ------END PUBLIC KEY-----)"; - std::string rsa512_pub_key_invalid = R"(-----BEGIN PUBLIC KEY----- -MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxzYuc22QSst/dS7geYYK -5l5kLxU0tayNdixkEQ17ix+CUcUbKIsnyftZxaCYT46rQtXgCaYRdJcbB3hmyrOa -vkhTpX79xJZnQmfuamMbZBqitvscxW9zRR9tBUL6vdi/0rpoUwPMEh8+Bw7CgYR0 -FK0DhWYBNDfe9HKcyZEv3max8Cdq18htxjEsdYO0iwzhtKRXomBWTdhD5ykd/fAC -VTr4+KEY+IeLvubHVmLUhbE5NgWXxrRpGasDqzKhCTmsa2Ysf712rl57SlH0Wz/M -r3F7aM9YpErzeYLrl0GhQr9BVJxOvXcVd4kmY+XkiCcrkyS1cnghnllh+LCwQu1s -YwIDAQAB ------END PUBLIC KEY-----)"; - std::string ecdsa_priv_key = R"(-----BEGIN PRIVATE KEY----- -MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgPGJGAm4X1fvBuC1z -SpO/4Izx6PXfNMaiKaS5RUkFqEGhRANCAARCBvmeksd3QGTrVs2eMrrfa7CYF+sX -sjyGg+Bo5mPKGH4Gs8M7oIvoP9pb/I85tdebtKlmiCZHAZE5w4DfJSV6 ------END PRIVATE KEY-----)"; - std::string ecdsa_pub_key = R"(-----BEGIN PUBLIC KEY----- -MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEQgb5npLHd0Bk61bNnjK632uwmBfr -F7I8hoPgaOZjyhh+BrPDO6CL6D/aW/yPObXXm7SpZogmRwGROcOA3yUleg== ------END PUBLIC KEY-----)"; - std::string ecdsa_pub_key_invalid = R"(-----BEGIN PUBLIC KEY----- -MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEoBUyo8CQAFPeYPvv78ylh5MwFZjT -CLQeb042TjiMJxG+9DLFmRSMlBQ9T/RsLLc+PmpB1+7yPAR+oR5gZn3kJQ== ------END PUBLIC KEY-----)"; -} diff --git a/dep/jwt-cpp/include/jwt-cpp/base.h b/dep/jwt-cpp/include/jwt-cpp/base.h index dfca7fc08..c447113c9 100644 --- a/dep/jwt-cpp/include/jwt-cpp/base.h +++ b/dep/jwt-cpp/include/jwt-cpp/base.h @@ -1,39 +1,64 @@ -#pragma once -#include +#ifndef JWT_CPP_BASE_H +#define JWT_CPP_BASE_H + #include +#include +#include + +#ifdef __has_cpp_attribute +#if __has_cpp_attribute(fallthrough) +#define JWT_FALLTHROUGH [[fallthrough]] +#endif +#endif + +#ifndef JWT_FALLTHROUGH +#define JWT_FALLTHROUGH +#endif namespace jwt { + /** + * \brief character maps when encoding and decoding + */ namespace alphabet { + /** + * \brief valid list of characted when working with [Base64](https://tools.ietf.org/html/rfc3548) + */ struct base64 { static const std::array& data() { - static std::array data = { - {'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', - 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z', 'a', 'b', 'c', 'd', 'e', 'f', - 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', - 'w', 'x', 'y', 'z', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '+', '/'}}; - return data; - }; + static constexpr std::array data{ + {'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', + 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z', 'a', 'b', 'c', 'd', 'e', 'f', + 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', + 'w', 'x', 'y', 'z', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '+', '/'}}; + return data; + } static const std::string& fill() { - static std::string fill = "="; + static std::string fill{"="}; return fill; } }; + /** + * \brief valid list of characted when working with [Base64URL](https://tools.ietf.org/html/rfc4648) + */ struct base64url { static const std::array& data() { - static std::array data = { - {'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', - 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z', 'a', 'b', 'c', 'd', 'e', 'f', - 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', - 'w', 'x', 'y', 'z', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '-', '_'}}; - return data; - }; + static constexpr std::array data{ + {'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', + 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z', 'a', 'b', 'c', 'd', 'e', 'f', + 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', + 'w', 'x', 'y', 'z', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '-', '_'}}; + return data; + } static const std::string& fill() { - static std::string fill = "%3d"; + static std::string fill{"%3d"}; return fill; } }; - } + } // namespace alphabet + /** + * \brief Alphabet generic methods for working with encoding/decoding the base64 family + */ class base { public: template @@ -44,18 +69,27 @@ namespace jwt { static std::string decode(const std::string& base) { return decode(base, T::data(), T::fill()); } + template + static std::string pad(const std::string& base) { + return pad(base, T::fill()); + } + template + static std::string trim(const std::string& base) { + return trim(base, T::fill()); + } private: - static std::string encode(const std::string& bin, const std::array& alphabet, const std::string& fill) { + static std::string encode(const std::string& bin, const std::array& alphabet, + const std::string& fill) { size_t size = bin.size(); std::string res; // clear incomplete bytes size_t fast_size = size - size % 3; for (size_t i = 0; i < fast_size;) { - uint32_t octet_a = (unsigned char)bin[i++]; - uint32_t octet_b = (unsigned char)bin[i++]; - uint32_t octet_c = (unsigned char)bin[i++]; + uint32_t octet_a = static_cast(bin[i++]); + uint32_t octet_b = static_cast(bin[i++]); + uint32_t octet_c = static_cast(bin[i++]); uint32_t triple = (octet_a << 0x10) + (octet_b << 0x08) + octet_c; @@ -65,14 +99,13 @@ namespace jwt { res += alphabet[(triple >> 0 * 6) & 0x3F]; } - if (fast_size == size) - return res; + if (fast_size == size) return res; size_t mod = size % 3; - uint32_t octet_a = fast_size < size ? (unsigned char)bin[fast_size++] : 0; - uint32_t octet_b = fast_size < size ? (unsigned char)bin[fast_size++] : 0; - uint32_t octet_c = fast_size < size ? (unsigned char)bin[fast_size++] : 0; + uint32_t octet_a = fast_size < size ? static_cast(bin[fast_size++]) : 0; + uint32_t octet_b = fast_size < size ? static_cast(bin[fast_size++]) : 0; + uint32_t octet_c = fast_size < size ? static_cast(bin[fast_size++]) : 0; uint32_t triple = (octet_a << 0x10) + (octet_b << 0x08) + octet_c; @@ -89,14 +122,14 @@ namespace jwt { res += alphabet[(triple >> 1 * 6) & 0x3F]; res += fill; break; - default: - break; + default: break; } return res; } - static std::string decode(const std::string& base, const std::array& alphabet, const std::string& fill) { + static std::string decode(const std::string& base, const std::array& alphabet, + const std::string& fill) { size_t size = base.size(); size_t fill_cnt = 0; @@ -104,14 +137,12 @@ namespace jwt { if (base.substr(size - fill.size(), fill.size()) == fill) { fill_cnt++; size -= fill.size(); - if(fill_cnt > 2) - throw std::runtime_error("Invalid input"); - } - else break; + if (fill_cnt > 2) throw std::runtime_error("Invalid input"); + } else + break; } - if ((size + fill_cnt) % 4 != 0) - throw std::runtime_error("Invalid input"); + if ((size + fill_cnt) % 4 != 0) throw std::runtime_error("Invalid input"); size_t out_size = size / 4 * 3; std::string res; @@ -119,13 +150,11 @@ namespace jwt { auto get_sextet = [&](size_t offset) { for (size_t i = 0; i < alphabet.size(); i++) { - if (alphabet[i] == base[offset]) - return i; + if (alphabet[i] == base[offset]) return static_cast(i); } throw std::runtime_error("Invalid input"); }; - size_t fast_size = size - size % 4; for (size_t i = 0; i < fast_size;) { uint32_t sextet_a = get_sextet(i++); @@ -133,36 +162,47 @@ namespace jwt { uint32_t sextet_c = get_sextet(i++); uint32_t sextet_d = get_sextet(i++); - uint32_t triple = (sextet_a << 3 * 6) - + (sextet_b << 2 * 6) - + (sextet_c << 1 * 6) - + (sextet_d << 0 * 6); + uint32_t triple = (sextet_a << 3 * 6) + (sextet_b << 2 * 6) + (sextet_c << 1 * 6) + (sextet_d << 0 * 6); - res += (triple >> 2 * 8) & 0xFF; - res += (triple >> 1 * 8) & 0xFF; - res += (triple >> 0 * 8) & 0xFF; + res += static_cast((triple >> 2 * 8) & 0xFFU); + res += static_cast((triple >> 1 * 8) & 0xFFU); + res += static_cast((triple >> 0 * 8) & 0xFFU); } - if (fill_cnt == 0) - return res; + if (fill_cnt == 0) return res; - uint32_t triple = (get_sextet(fast_size) << 3 * 6) - + (get_sextet(fast_size + 1) << 2 * 6); + uint32_t triple = (get_sextet(fast_size) << 3 * 6) + (get_sextet(fast_size + 1) << 2 * 6); switch (fill_cnt) { case 1: triple |= (get_sextet(fast_size + 2) << 1 * 6); - res += (triple >> 2 * 8) & 0xFF; - res += (triple >> 1 * 8) & 0xFF; - break; - case 2: - res += (triple >> 2 * 8) & 0xFF; - break; - default: + res += static_cast((triple >> 2 * 8) & 0xFFU); + res += static_cast((triple >> 1 * 8) & 0xFFU); break; + case 2: res += static_cast((triple >> 2 * 8) & 0xFFU); break; + default: break; } return res; } + + static std::string pad(const std::string& base, const std::string& fill) { + std::string padding; + switch (base.size() % 4) { + case 1: padding += fill; JWT_FALLTHROUGH; + case 2: padding += fill; JWT_FALLTHROUGH; + case 3: padding += fill; JWT_FALLTHROUGH; + default: break; + } + + return base + padding; + } + + static std::string trim(const std::string& base, const std::string& fill) { + auto pos = base.find(fill); + return base.substr(0, pos); + } }; -} +} // namespace jwt + +#endif diff --git a/dep/jwt-cpp/include/jwt-cpp/jwt.h b/dep/jwt-cpp/include/jwt-cpp/jwt.h new file mode 100644 index 000000000..a5c85f9a6 --- /dev/null +++ b/dep/jwt-cpp/include/jwt-cpp/jwt.h @@ -0,0 +1,3037 @@ +#ifndef JWT_CPP_JWT_H +#define JWT_CPP_JWT_H + +#ifndef JWT_DISABLE_PICOJSON +#ifndef PICOJSON_USE_INT64 +#define PICOJSON_USE_INT64 +#endif +#include "picojson/picojson.h" +#endif + +#ifndef JWT_DISABLE_BASE64 +#include "base.h" +#endif + +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#if __cplusplus >= 201402L +#ifdef __has_include +#if __has_include() +#include +#endif +#endif +#endif + +// If openssl version less than 1.1 +#if OPENSSL_VERSION_NUMBER < 0x10100000L +#define OPENSSL10 +#endif + +// If openssl version less than 1.1.1 +#if OPENSSL_VERSION_NUMBER < 0x10101000L +#define OPENSSL110 +#endif + +#if defined(LIBRESSL_VERSION_NUMBER) +#define OPENSSL10 +#define OPENSSL110 +#endif + +#ifndef JWT_CLAIM_EXPLICIT +#define JWT_CLAIM_EXPLICIT explicit +#endif + +/** + * \brief JSON Web Token + * + * A namespace to contain everything related to handling JSON Web Tokens, JWT for short, + * as a part of [RFC7519](https://tools.ietf.org/html/rfc7519), or alternatively for + * JWS (JSON Web Signature) from [RFC7515](https://tools.ietf.org/html/rfc7515) + */ +namespace jwt { + using date = std::chrono::system_clock::time_point; + + /** + * \brief Everything related to error codes issued by the library + */ + namespace error { + struct signature_verification_exception : public std::system_error { + using system_error::system_error; + }; + struct signature_generation_exception : public std::system_error { + using system_error::system_error; + }; + struct rsa_exception : public std::system_error { + using system_error::system_error; + }; + struct ecdsa_exception : public std::system_error { + using system_error::system_error; + }; + struct token_verification_exception : public std::system_error { + using system_error::system_error; + }; + /** + * \brief Errors related to processing of RSA signatures + */ + enum class rsa_error { + ok = 0, + cert_load_failed = 10, + get_key_failed, + write_key_failed, + write_cert_failed, + convert_to_pem_failed, + load_key_bio_write, + load_key_bio_read, + create_mem_bio_failed, + no_key_provided + }; + /** + * \brief Error category for RSA errors + */ + inline std::error_category& rsa_error_category() { + class rsa_error_cat : public std::error_category { + public: + const char* name() const noexcept override { return "rsa_error"; }; + std::string message(int ev) const override { + switch (static_cast(ev)) { + case rsa_error::ok: return "no error"; + case rsa_error::cert_load_failed: return "error loading cert into memory"; + case rsa_error::get_key_failed: return "error getting key from certificate"; + case rsa_error::write_key_failed: return "error writing key data in PEM format"; + case rsa_error::write_cert_failed: return "error writing cert data in PEM format"; + case rsa_error::convert_to_pem_failed: return "failed to convert key to pem"; + case rsa_error::load_key_bio_write: return "failed to load key: bio write failed"; + case rsa_error::load_key_bio_read: return "failed to load key: bio read failed"; + case rsa_error::create_mem_bio_failed: return "failed to create memory bio"; + case rsa_error::no_key_provided: return "at least one of public or private key need to be present"; + default: return "unknown RSA error"; + } + } + }; + static rsa_error_cat cat; + return cat; + } + + inline std::error_code make_error_code(rsa_error e) { return {static_cast(e), rsa_error_category()}; } + /** + * \brief Errors related to processing of RSA signatures + */ + enum class ecdsa_error { + ok = 0, + load_key_bio_write = 10, + load_key_bio_read, + create_mem_bio_failed, + no_key_provided, + invalid_key_size, + invalid_key + }; + /** + * \brief Error category for ECDSA errors + */ + inline std::error_category& ecdsa_error_category() { + class ecdsa_error_cat : public std::error_category { + public: + const char* name() const noexcept override { return "ecdsa_error"; }; + std::string message(int ev) const override { + switch (static_cast(ev)) { + case ecdsa_error::ok: return "no error"; + case ecdsa_error::load_key_bio_write: return "failed to load key: bio write failed"; + case ecdsa_error::load_key_bio_read: return "failed to load key: bio read failed"; + case ecdsa_error::create_mem_bio_failed: return "failed to create memory bio"; + case ecdsa_error::no_key_provided: + return "at least one of public or private key need to be present"; + case ecdsa_error::invalid_key_size: return "invalid key size"; + case ecdsa_error::invalid_key: return "invalid key"; + default: return "unknown ECDSA error"; + } + } + }; + static ecdsa_error_cat cat; + return cat; + } + + inline std::error_code make_error_code(ecdsa_error e) { return {static_cast(e), ecdsa_error_category()}; } + + /** + * \brief Errors related to verification of signatures + */ + enum class signature_verification_error { + ok = 0, + invalid_signature = 10, + create_context_failed, + verifyinit_failed, + verifyupdate_failed, + verifyfinal_failed, + get_key_failed + }; + /** + * \brief Error category for verification errors + */ + inline std::error_category& signature_verification_error_category() { + class verification_error_cat : public std::error_category { + public: + const char* name() const noexcept override { return "signature_verification_error"; }; + std::string message(int ev) const override { + switch (static_cast(ev)) { + case signature_verification_error::ok: return "no error"; + case signature_verification_error::invalid_signature: return "invalid signature"; + case signature_verification_error::create_context_failed: + return "failed to verify signature: could not create context"; + case signature_verification_error::verifyinit_failed: + return "failed to verify signature: VerifyInit failed"; + case signature_verification_error::verifyupdate_failed: + return "failed to verify signature: VerifyUpdate failed"; + case signature_verification_error::verifyfinal_failed: + return "failed to verify signature: VerifyFinal failed"; + case signature_verification_error::get_key_failed: + return "failed to verify signature: Could not get key"; + default: return "unknown signature verification error"; + } + } + }; + static verification_error_cat cat; + return cat; + } + + inline std::error_code make_error_code(signature_verification_error e) { + return {static_cast(e), signature_verification_error_category()}; + } + + /** + * \brief Errors related to signature generation errors + */ + enum class signature_generation_error { + ok = 0, + hmac_failed = 10, + create_context_failed, + signinit_failed, + signupdate_failed, + signfinal_failed, + ecdsa_do_sign_failed, + digestinit_failed, + digestupdate_failed, + digestfinal_failed, + rsa_padding_failed, + rsa_private_encrypt_failed, + get_key_failed + }; + /** + * \brief Error category for signature generation errors + */ + inline std::error_category& signature_generation_error_category() { + class signature_generation_error_cat : public std::error_category { + public: + const char* name() const noexcept override { return "signature_generation_error"; }; + std::string message(int ev) const override { + switch (static_cast(ev)) { + case signature_generation_error::ok: return "no error"; + case signature_generation_error::hmac_failed: return "hmac failed"; + case signature_generation_error::create_context_failed: + return "failed to create signature: could not create context"; + case signature_generation_error::signinit_failed: + return "failed to create signature: SignInit failed"; + case signature_generation_error::signupdate_failed: + return "failed to create signature: SignUpdate failed"; + case signature_generation_error::signfinal_failed: + return "failed to create signature: SignFinal failed"; + case signature_generation_error::ecdsa_do_sign_failed: return "failed to generate ecdsa signature"; + case signature_generation_error::digestinit_failed: + return "failed to create signature: DigestInit failed"; + case signature_generation_error::digestupdate_failed: + return "failed to create signature: DigestUpdate failed"; + case signature_generation_error::digestfinal_failed: + return "failed to create signature: DigestFinal failed"; + case signature_generation_error::rsa_padding_failed: + return "failed to create signature: RSA_padding_add_PKCS1_PSS_mgf1 failed"; + case signature_generation_error::rsa_private_encrypt_failed: + return "failed to create signature: RSA_private_encrypt failed"; + case signature_generation_error::get_key_failed: + return "failed to generate signature: Could not get key"; + default: return "unknown signature generation error"; + } + } + }; + static signature_generation_error_cat cat = {}; + return cat; + } + + inline std::error_code make_error_code(signature_generation_error e) { + return {static_cast(e), signature_generation_error_category()}; + } + + /** + * \brief Errors related to token verification errors + */ + enum class token_verification_error { + ok = 0, + wrong_algorithm = 10, + missing_claim, + claim_type_missmatch, + claim_value_missmatch, + token_expired, + audience_missmatch + }; + /** + * \brief Error category for token verification errors + */ + inline std::error_category& token_verification_error_category() { + class token_verification_error_cat : public std::error_category { + public: + const char* name() const noexcept override { return "token_verification_error"; }; + std::string message(int ev) const override { + switch (static_cast(ev)) { + case token_verification_error::ok: return "no error"; + case token_verification_error::wrong_algorithm: return "wrong algorithm"; + case token_verification_error::missing_claim: return "decoded JWT is missing required claim(s)"; + case token_verification_error::claim_type_missmatch: + return "claim type does not match expected type"; + case token_verification_error::claim_value_missmatch: + return "claim value does not match expected value"; + case token_verification_error::token_expired: return "token expired"; + case token_verification_error::audience_missmatch: + return "token doesn't contain the required audience"; + default: return "unknown token verification error"; + } + } + }; + static token_verification_error_cat cat = {}; + return cat; + } + + inline std::error_code make_error_code(token_verification_error e) { + return {static_cast(e), token_verification_error_category()}; + } + + inline void throw_if_error(std::error_code ec) { + if (ec) { + if (ec.category() == rsa_error_category()) throw rsa_exception(ec); + if (ec.category() == ecdsa_error_category()) throw ecdsa_exception(ec); + if (ec.category() == signature_verification_error_category()) + throw signature_verification_exception(ec); + if (ec.category() == signature_generation_error_category()) throw signature_generation_exception(ec); + if (ec.category() == token_verification_error_category()) throw token_verification_exception(ec); + } + } + } // namespace error + + // FIXME: Remove + // Keep backward compat at least for a couple of revisions + using error::ecdsa_exception; + using error::rsa_exception; + using error::signature_generation_exception; + using error::signature_verification_exception; + using error::token_verification_exception; +} // namespace jwt +namespace std { + template<> + struct is_error_code_enum : true_type {}; + template<> + struct is_error_code_enum : true_type {}; + template<> + struct is_error_code_enum : true_type {}; + template<> + struct is_error_code_enum : true_type {}; + template<> + struct is_error_code_enum : true_type {}; +} // namespace std +namespace jwt { + /** + * \brief A collection for working with certificates + * + * These _helpers_ are usefully when working with certificates OpenSSL APIs. + * For example, when dealing with JWKS (JSON Web Key Set)[https://tools.ietf.org/html/rfc7517] + * you maybe need to extract the modulus and exponent of an RSA Public Key. + */ + namespace helper { + /** + * \brief Extract the public key of a pem certificate + * + * \param certstr String containing the certificate encoded as pem + * \param pw Password used to decrypt certificate (leave empty if not encrypted) + * \param ec error_code for error_detection (gets cleared if no error occures) + */ + inline std::string extract_pubkey_from_cert(const std::string& certstr, const std::string& pw, + std::error_code& ec) { + ec.clear(); +#if OPENSSL_VERSION_NUMBER <= 0x10100003L + std::unique_ptr certbio( + BIO_new_mem_buf(const_cast(certstr.data()), static_cast(certstr.size())), BIO_free_all); +#else + std::unique_ptr certbio( + BIO_new_mem_buf(certstr.data(), static_cast(certstr.size())), BIO_free_all); +#endif + std::unique_ptr keybio(BIO_new(BIO_s_mem()), BIO_free_all); + if (!certbio || !keybio) { + ec = error::rsa_error::create_mem_bio_failed; + return {}; + } + + std::unique_ptr cert( + PEM_read_bio_X509(certbio.get(), nullptr, nullptr, const_cast(pw.c_str())), X509_free); + if (!cert) { + ec = error::rsa_error::cert_load_failed; + return {}; + } + std::unique_ptr key(X509_get_pubkey(cert.get()), EVP_PKEY_free); + if (!key) { + ec = error::rsa_error::get_key_failed; + return {}; + } + if (PEM_write_bio_PUBKEY(keybio.get(), key.get()) == 0) { + ec = error::rsa_error::write_key_failed; + return {}; + } + char* ptr = nullptr; + auto len = BIO_get_mem_data(keybio.get(), &ptr); + if (len <= 0 || ptr == nullptr) { + ec = error::rsa_error::convert_to_pem_failed; + return {}; + } + return {ptr, static_cast(len)}; + } + + /** + * \brief Extract the public key of a pem certificate + * + * \param certstr String containing the certificate encoded as pem + * \param pw Password used to decrypt certificate (leave empty if not encrypted) + * \throw rsa_exception if an error occurred + */ + inline std::string extract_pubkey_from_cert(const std::string& certstr, const std::string& pw = "") { + std::error_code ec; + auto res = extract_pubkey_from_cert(certstr, pw, ec); + error::throw_if_error(ec); + return res; + } + + /** + * \brief Convert the certificate provided as base64 DER to PEM. + * + * This is useful when using with JWKs as x5c claim is encoded as base64 DER. More info + * (here)[https://tools.ietf.org/html/rfc7517#section-4.7] + * + * \tparam Decode is callabled, taking a string_type and returns a string_type. + * It should ensure the padding of the input and then base64 decode and return + * the results. + * + * \param cert_base64_der_str String containing the certificate encoded as base64 DER + * \param decode The function to decode the cert + * \param ec error_code for error_detection (gets cleared if no error occures) + */ + template + std::string convert_base64_der_to_pem(const std::string& cert_base64_der_str, Decode decode, + std::error_code& ec) { + ec.clear(); + const auto decodedStr = decode(cert_base64_der_str); + auto c_str = reinterpret_cast(decodedStr.c_str()); + + std::unique_ptr cert(d2i_X509(NULL, &c_str, decodedStr.size()), X509_free); + std::unique_ptr certbio(BIO_new(BIO_s_mem()), BIO_free_all); + if (!cert || !certbio) { + ec = error::rsa_error::create_mem_bio_failed; + return {}; + } + + if (!PEM_write_bio_X509(certbio.get(), cert.get())) { + ec = error::rsa_error::write_cert_failed; + return {}; + } + + char* ptr = nullptr; + const auto len = BIO_get_mem_data(certbio.get(), &ptr); + if (len <= 0 || ptr == nullptr) { + ec = error::rsa_error::convert_to_pem_failed; + return {}; + } + + return {ptr, static_cast(len)}; + } + + /** + * \brief Convert the certificate provided as base64 DER to PEM. + * + * This is useful when using with JWKs as x5c claim is encoded as base64 DER. More info + * (here)[https://tools.ietf.org/html/rfc7517#section-4.7] + * + * \tparam Decode is callabled, taking a string_type and returns a string_type. + * It should ensure the padding of the input and then base64 decode and return + * the results. + * + * \param cert_base64_der_str String containing the certificate encoded as base64 DER + * \param decode The function to decode the cert + * \throw rsa_exception if an error occurred + */ + template + std::string convert_base64_der_to_pem(const std::string& cert_base64_der_str, Decode decode) { + std::error_code ec; + auto res = convert_base64_der_to_pem(cert_base64_der_str, std::move(decode), ec); + error::throw_if_error(ec); + return res; + } +#ifndef JWT_DISABLE_BASE64 + /** + * \brief Convert the certificate provided as base64 DER to PEM. + * + * This is useful when using with JWKs as x5c claim is encoded as base64 DER. More info + * (here)[https://tools.ietf.org/html/rfc7517#section-4.7] + * + * \param cert_base64_der_str String containing the certificate encoded as base64 DER + * \param ec error_code for error_detection (gets cleared if no error occures) + */ + inline std::string convert_base64_der_to_pem(const std::string& cert_base64_der_str, std::error_code& ec) { + auto decode = [](const std::string& token) { + return base::decode(base::pad(token)); + }; + return convert_base64_der_to_pem(cert_base64_der_str, std::move(decode), ec); + } + + /** + * \brief Convert the certificate provided as base64 DER to PEM. + * + * This is useful when using with JWKs as x5c claim is encoded as base64 DER. More info + * (here)[https://tools.ietf.org/html/rfc7517#section-4.7] + * + * \param cert_base64_der_str String containing the certificate encoded as base64 DER + * \throw rsa_exception if an error occurred + */ + inline std::string convert_base64_der_to_pem(const std::string& cert_base64_der_str) { + std::error_code ec; + auto res = convert_base64_der_to_pem(cert_base64_der_str, ec); + error::throw_if_error(ec); + return res; + } +#endif + /** + * \brief Load a public key from a string. + * + * The string should contain a pem encoded certificate or public key + * + * \param certstr String containing the certificate encoded as pem + * \param pw Password used to decrypt certificate (leave empty if not encrypted) + * \param ec error_code for error_detection (gets cleared if no error occures) + */ + inline std::shared_ptr load_public_key_from_string(const std::string& key, + const std::string& password, std::error_code& ec) { + ec.clear(); + std::unique_ptr pubkey_bio(BIO_new(BIO_s_mem()), BIO_free_all); + if (!pubkey_bio) { + ec = error::rsa_error::create_mem_bio_failed; + return nullptr; + } + if (key.substr(0, 27) == "-----BEGIN CERTIFICATE-----") { + auto epkey = helper::extract_pubkey_from_cert(key, password, ec); + if (ec) return nullptr; + const int len = static_cast(epkey.size()); + if (BIO_write(pubkey_bio.get(), epkey.data(), len) != len) { + ec = error::rsa_error::load_key_bio_write; + return nullptr; + } + } else { + const int len = static_cast(key.size()); + if (BIO_write(pubkey_bio.get(), key.data(), len) != len) { + ec = error::rsa_error::load_key_bio_write; + return nullptr; + } + } + + std::shared_ptr pkey( + PEM_read_bio_PUBKEY(pubkey_bio.get(), nullptr, nullptr, + (void*)password.data()), // NOLINT(google-readability-casting) requires `const_cast` + EVP_PKEY_free); + if (!pkey) { + ec = error::rsa_error::load_key_bio_read; + return nullptr; + } + return pkey; + } + + /** + * \brief Load a public key from a string. + * + * The string should contain a pem encoded certificate or public key + * + * \param certstr String containing the certificate or key encoded as pem + * \param pw Password used to decrypt certificate or key (leave empty if not encrypted) + * \throw rsa_exception if an error occurred + */ + inline std::shared_ptr load_public_key_from_string(const std::string& key, + const std::string& password = "") { + std::error_code ec; + auto res = load_public_key_from_string(key, password, ec); + error::throw_if_error(ec); + return res; + } + + /** + * \brief Load a private key from a string. + * + * \param key String containing a private key as pem + * \param pw Password used to decrypt key (leave empty if not encrypted) + * \param ec error_code for error_detection (gets cleared if no error occures) + */ + inline std::shared_ptr + load_private_key_from_string(const std::string& key, const std::string& password, std::error_code& ec) { + std::unique_ptr privkey_bio(BIO_new(BIO_s_mem()), BIO_free_all); + if (!privkey_bio) { + ec = error::rsa_error::create_mem_bio_failed; + return nullptr; + } + const int len = static_cast(key.size()); + if (BIO_write(privkey_bio.get(), key.data(), len) != len) { + ec = error::rsa_error::load_key_bio_write; + return nullptr; + } + std::shared_ptr pkey( + PEM_read_bio_PrivateKey(privkey_bio.get(), nullptr, nullptr, const_cast(password.c_str())), + EVP_PKEY_free); + if (!pkey) { + ec = error::rsa_error::load_key_bio_read; + return nullptr; + } + return pkey; + } + + /** + * \brief Load a private key from a string. + * + * \param key String containing a private key as pem + * \param pw Password used to decrypt key (leave empty if not encrypted) + * \throw rsa_exception if an error occurred + */ + inline std::shared_ptr load_private_key_from_string(const std::string& key, + const std::string& password = "") { + std::error_code ec; + auto res = load_private_key_from_string(key, password, ec); + error::throw_if_error(ec); + return res; + } + + /** + * Convert a OpenSSL BIGNUM to a std::string + * \param bn BIGNUM to convert + * \return bignum as string + */ + inline +#ifdef OPENSSL10 + static std::string + bn2raw(BIGNUM* bn) +#else + static std::string + bn2raw(const BIGNUM* bn) +#endif + { + std::string res(BN_num_bytes(bn), '\0'); + BN_bn2bin(bn, (unsigned char*)res.data()); // NOLINT(google-readability-casting) requires `const_cast` + return res; + } + /** + * Convert an std::string to a OpenSSL BIGNUM + * \param raw String to convert + * \return BIGNUM representation + */ + inline static std::unique_ptr raw2bn(const std::string& raw) { + return std::unique_ptr( + BN_bin2bn(reinterpret_cast(raw.data()), static_cast(raw.size()), nullptr), + BN_free); + } + } // namespace helper + + /** + * \brief Various cryptographic algorithms when working with JWT + * + * JWT (JSON Web Tokens) signatures are typically used as the payload for a JWS (JSON Web Signature) or + * JWE (JSON Web Encryption). Both of these use various cryptographic as specified by + * [RFC7518](https://tools.ietf.org/html/rfc7518) and are exposed through the a [JOSE + * Header](https://tools.ietf.org/html/rfc7515#section-4) which points to one of the JWA (JSON Web + * Algorithms)(https://tools.ietf.org/html/rfc7518#section-3.1) + */ + namespace algorithm { + /** + * \brief "none" algorithm. + * + * Returns and empty signature and checks if the given signature is empty. + */ + struct none { + /** + * \brief Return an empty string + */ + std::string sign(const std::string& /*unused*/, std::error_code& ec) const { + ec.clear(); + return {}; + } + /** + * \brief Check if the given signature is empty. + * + * JWT's with "none" algorithm should not contain a signature. + * \param signature Signature data to verify + * \param ec error_code filled with details about the error + */ + void verify(const std::string& /*unused*/, const std::string& signature, std::error_code& ec) const { + ec.clear(); + if (!signature.empty()) { ec = error::signature_verification_error::invalid_signature; } + } + /// Get algorithm name + std::string name() const { return "none"; } + }; + /** + * \brief Base class for HMAC family of algorithms + */ + struct hmacsha { + /** + * Construct new hmac algorithm + * \param key Key to use for HMAC + * \param md Pointer to hash function + * \param name Name of the algorithm + */ + hmacsha(std::string key, const EVP_MD* (*md)(), std::string name) + : secret(std::move(key)), md(md), alg_name(std::move(name)) {} + /** + * Sign jwt data + * \param data The data to sign + * \param ec error_code filled with details on error + * \return HMAC signature for the given data + */ + std::string sign(const std::string& data, std::error_code& ec) const { + ec.clear(); + std::string res(static_cast(EVP_MAX_MD_SIZE), '\0'); + auto len = static_cast(res.size()); + if (HMAC(md(), secret.data(), static_cast(secret.size()), + reinterpret_cast(data.data()), static_cast(data.size()), + (unsigned char*)res.data(), // NOLINT(google-readability-casting) requires `const_cast` + &len) == nullptr) { + ec = error::signature_generation_error::hmac_failed; + return {}; + } + res.resize(len); + return res; + } + /** + * Check if signature is valid + * \param data The data to check signature against + * \param signature Signature provided by the jwt + * \param ec Filled with details about failure. + */ + void verify(const std::string& data, const std::string& signature, std::error_code& ec) const { + ec.clear(); + auto res = sign(data, ec); + if (ec) return; + + bool matched = true; + for (size_t i = 0; i < std::min(res.size(), signature.size()); i++) + if (res[i] != signature[i]) matched = false; + if (res.size() != signature.size()) matched = false; + if (!matched) { + ec = error::signature_verification_error::invalid_signature; + return; + } + } + /** + * Returns the algorithm name provided to the constructor + * \return algorithm's name + */ + std::string name() const { return alg_name; } + + private: + /// HMAC secrect + const std::string secret; + /// HMAC hash generator + const EVP_MD* (*md)(); + /// algorithm's name + const std::string alg_name; + }; + /** + * \brief Base class for RSA family of algorithms + */ + struct rsa { + /** + * Construct new rsa algorithm + * \param public_key RSA public key in PEM format + * \param private_key RSA private key or empty string if not available. If empty, signing will always fail. + * \param public_key_password Password to decrypt public key pem. + * \param private_key_password Password to decrypt private key pem. + * \param md Pointer to hash function + * \param name Name of the algorithm + */ + rsa(const std::string& public_key, const std::string& private_key, const std::string& public_key_password, + const std::string& private_key_password, const EVP_MD* (*md)(), std::string name) + : md(md), alg_name(std::move(name)) { + if (!private_key.empty()) { + pkey = helper::load_private_key_from_string(private_key, private_key_password); + } else if (!public_key.empty()) { + pkey = helper::load_public_key_from_string(public_key, public_key_password); + } else + throw rsa_exception(error::rsa_error::no_key_provided); + } + /** + * Sign jwt data + * \param data The data to sign + * \param ec error_code filled with details on error + * \return RSA signature for the given data + */ + std::string sign(const std::string& data, std::error_code& ec) const { + ec.clear(); +#ifdef OPENSSL10 + std::unique_ptr ctx(EVP_MD_CTX_create(), EVP_MD_CTX_destroy); +#else + std::unique_ptr ctx(EVP_MD_CTX_create(), EVP_MD_CTX_free); +#endif + if (!ctx) { + ec = error::signature_generation_error::create_context_failed; + return {}; + } + if (!EVP_SignInit(ctx.get(), md())) { + ec = error::signature_generation_error::signinit_failed; + return {}; + } + + std::string res(EVP_PKEY_size(pkey.get()), '\0'); + unsigned int len = 0; + + if (!EVP_SignUpdate(ctx.get(), data.data(), data.size())) { + ec = error::signature_generation_error::signupdate_failed; + return {}; + } + if (EVP_SignFinal(ctx.get(), (unsigned char*)res.data(), &len, pkey.get()) == 0) { + ec = error::signature_generation_error::signfinal_failed; + return {}; + } + + res.resize(len); + return res; + } + /** + * Check if signature is valid + * \param data The data to check signature against + * \param signature Signature provided by the jwt + * \param ec Filled with details on failure + */ + void verify(const std::string& data, const std::string& signature, std::error_code& ec) const { + ec.clear(); +#ifdef OPENSSL10 + std::unique_ptr ctx(EVP_MD_CTX_create(), EVP_MD_CTX_destroy); +#else + std::unique_ptr ctx(EVP_MD_CTX_create(), EVP_MD_CTX_free); +#endif + if (!ctx) { + ec = error::signature_verification_error::create_context_failed; + return; + } + if (!EVP_VerifyInit(ctx.get(), md())) { + ec = error::signature_verification_error::verifyinit_failed; + return; + } + if (!EVP_VerifyUpdate(ctx.get(), data.data(), data.size())) { + ec = error::signature_verification_error::verifyupdate_failed; + return; + } + auto res = EVP_VerifyFinal(ctx.get(), reinterpret_cast(signature.data()), + static_cast(signature.size()), pkey.get()); + if (res != 1) { + ec = error::signature_verification_error::verifyfinal_failed; + return; + } + } + /** + * Returns the algorithm name provided to the constructor + * \return algorithm's name + */ + std::string name() const { return alg_name; } + + private: + /// OpenSSL structure containing converted keys + std::shared_ptr pkey; + /// Hash generator + const EVP_MD* (*md)(); + /// algorithm's name + const std::string alg_name; + }; + /** + * \brief Base class for ECDSA family of algorithms + */ + struct ecdsa { + /** + * Construct new ecdsa algorithm + * \param public_key ECDSA public key in PEM format + * \param private_key ECDSA private key or empty string if not available. If empty, signing will always + * fail. \param public_key_password Password to decrypt public key pem. \param private_key_password Password + * to decrypt private key pem. \param md Pointer to hash function \param name Name of the algorithm + */ + ecdsa(const std::string& public_key, const std::string& private_key, const std::string& public_key_password, + const std::string& private_key_password, const EVP_MD* (*md)(), std::string name, size_t siglen) + : md(md), alg_name(std::move(name)), signature_length(siglen) { + if (!public_key.empty()) { + std::unique_ptr pubkey_bio(BIO_new(BIO_s_mem()), BIO_free_all); + if (!pubkey_bio) throw ecdsa_exception(error::ecdsa_error::create_mem_bio_failed); + if (public_key.substr(0, 27) == "-----BEGIN CERTIFICATE-----") { + auto epkey = helper::extract_pubkey_from_cert(public_key, public_key_password); + const int len = static_cast(epkey.size()); + if (BIO_write(pubkey_bio.get(), epkey.data(), len) != len) + throw ecdsa_exception(error::ecdsa_error::load_key_bio_write); + } else { + const int len = static_cast(public_key.size()); + if (BIO_write(pubkey_bio.get(), public_key.data(), len) != len) + throw ecdsa_exception(error::ecdsa_error::load_key_bio_write); + } + + pkey.reset(PEM_read_bio_EC_PUBKEY( + pubkey_bio.get(), nullptr, nullptr, + (void*)public_key_password + .c_str()), // NOLINT(google-readability-casting) requires `const_cast` + EC_KEY_free); + if (!pkey) throw ecdsa_exception(error::ecdsa_error::load_key_bio_read); + size_t keysize = EC_GROUP_get_degree(EC_KEY_get0_group(pkey.get())); + if (keysize != signature_length * 4 && (signature_length != 132 || keysize != 521)) + throw ecdsa_exception(error::ecdsa_error::invalid_key_size); + } + + if (!private_key.empty()) { + std::unique_ptr privkey_bio(BIO_new(BIO_s_mem()), BIO_free_all); + if (!privkey_bio) throw ecdsa_exception(error::ecdsa_error::create_mem_bio_failed); + const int len = static_cast(private_key.size()); + if (BIO_write(privkey_bio.get(), private_key.data(), len) != len) + throw ecdsa_exception(error::ecdsa_error::load_key_bio_write); + pkey.reset(PEM_read_bio_ECPrivateKey(privkey_bio.get(), nullptr, nullptr, + const_cast(private_key_password.c_str())), + EC_KEY_free); + if (!pkey) throw ecdsa_exception(error::ecdsa_error::load_key_bio_read); + size_t keysize = EC_GROUP_get_degree(EC_KEY_get0_group(pkey.get())); + if (keysize != signature_length * 4 && (signature_length != 132 || keysize != 521)) + throw ecdsa_exception(error::ecdsa_error::invalid_key_size); + } + if (!pkey) throw ecdsa_exception(error::ecdsa_error::no_key_provided); + + if (EC_KEY_check_key(pkey.get()) == 0) throw ecdsa_exception(error::ecdsa_error::invalid_key); + } + /** + * Sign jwt data + * \param data The data to sign + * \param ec error_code filled with details on error + * \return ECDSA signature for the given data + */ + std::string sign(const std::string& data, std::error_code& ec) const { + ec.clear(); + const std::string hash = generate_hash(data, ec); + if (ec) return {}; + + std::unique_ptr sig( + ECDSA_do_sign(reinterpret_cast(hash.data()), static_cast(hash.size()), + pkey.get()), + ECDSA_SIG_free); + if (!sig) { + ec = error::signature_generation_error::ecdsa_do_sign_failed; + return {}; + } +#ifdef OPENSSL10 + + auto rr = helper::bn2raw(sig->r); + auto rs = helper::bn2raw(sig->s); +#else + const BIGNUM* r; + const BIGNUM* s; + ECDSA_SIG_get0(sig.get(), &r, &s); + auto rr = helper::bn2raw(r); + auto rs = helper::bn2raw(s); +#endif + if (rr.size() > signature_length / 2 || rs.size() > signature_length / 2) + throw std::logic_error("bignum size exceeded expected length"); + rr.insert(0, signature_length / 2 - rr.size(), '\0'); + rs.insert(0, signature_length / 2 - rs.size(), '\0'); + return rr + rs; + } + + /** + * Check if signature is valid + * \param data The data to check signature against + * \param signature Signature provided by the jwt + * \param ec Filled with details on error + */ + void verify(const std::string& data, const std::string& signature, std::error_code& ec) const { + ec.clear(); + const std::string hash = generate_hash(data, ec); + if (ec) return; + auto r = helper::raw2bn(signature.substr(0, signature.size() / 2)); + auto s = helper::raw2bn(signature.substr(signature.size() / 2)); + +#ifdef OPENSSL10 + ECDSA_SIG sig; + sig.r = r.get(); + sig.s = s.get(); + + if (ECDSA_do_verify((const unsigned char*)hash.data(), static_cast(hash.size()), &sig, + pkey.get()) != 1) { + ec = error::signature_verification_error::invalid_signature; + return; + } +#else + std::unique_ptr sig(ECDSA_SIG_new(), ECDSA_SIG_free); + if (!sig) { + ec = error::signature_verification_error::create_context_failed; + return; + } + + ECDSA_SIG_set0(sig.get(), r.release(), s.release()); + + if (ECDSA_do_verify(reinterpret_cast(hash.data()), static_cast(hash.size()), + sig.get(), pkey.get()) != 1) { + ec = error::signature_verification_error::invalid_signature; + return; + } +#endif + } + /** + * Returns the algorithm name provided to the constructor + * \return algorithm's name + */ + std::string name() const { return alg_name; } + + private: + /** + * Hash the provided data using the hash function specified in constructor + * \param data Data to hash + * \return Hash of data + */ + std::string generate_hash(const std::string& data, std::error_code& ec) const { +#ifdef OPENSSL10 + std::unique_ptr ctx(EVP_MD_CTX_create(), + &EVP_MD_CTX_destroy); +#else + std::unique_ptr ctx(EVP_MD_CTX_new(), EVP_MD_CTX_free); +#endif + if (!ctx) { + ec = error::signature_generation_error::create_context_failed; + return {}; + } + if (EVP_DigestInit(ctx.get(), md()) == 0) { + ec = error::signature_generation_error::digestinit_failed; + return {}; + } + if (EVP_DigestUpdate(ctx.get(), data.data(), data.size()) == 0) { + ec = error::signature_generation_error::digestupdate_failed; + return {}; + } + unsigned int len = 0; + std::string res(EVP_MD_CTX_size(ctx.get()), '\0'); + if (EVP_DigestFinal( + ctx.get(), + (unsigned char*)res.data(), // NOLINT(google-readability-casting) requires `const_cast` + &len) == 0) { + ec = error::signature_generation_error::digestfinal_failed; + return {}; + } + res.resize(len); + return res; + } + + /// OpenSSL struct containing keys + std::shared_ptr pkey; + /// Hash generator function + const EVP_MD* (*md)(); + /// algorithm's name + const std::string alg_name; + /// Length of the resulting signature + const size_t signature_length; + }; + +#ifndef OPENSSL110 + /** + * \brief Base class for EdDSA family of algorithms + * + * The EdDSA algorithms were introduced in [OpenSSL + * v1.1.1](https://www.openssl.org/news/openssl-1.1.1-notes.html), so these algorithms are only available when + * building against this version or higher. + */ + struct eddsa { + /** + * Construct new eddsa algorithm + * \param public_key EdDSA public key in PEM format + * \param private_key EdDSA private key or empty string if not available. If empty, signing will always + * fail. + * \param public_key_password Password to decrypt public key pem. + * \param private_key_password Password + * to decrypt private key pem. + * \param md Pointer to hash function + * \param name Name of the algorithm + */ + eddsa(const std::string& public_key, const std::string& private_key, const std::string& public_key_password, + const std::string& private_key_password, const EVP_MD* (*md)(), std::string name) + : md(md), alg_name(std::move(name)) { + if (!private_key.empty()) { + pkey = helper::load_private_key_from_string(private_key, private_key_password); + } else if (!public_key.empty()) { + pkey = helper::load_public_key_from_string(public_key, public_key_password); + } else + throw ecdsa_exception(error::ecdsa_error::load_key_bio_read); + } + /** + * Sign jwt data + * \param data The data to sign + * \param ec error_code filled with details on error + * \return EdDSA signature for the given data + */ + std::string sign(const std::string& data, std::error_code& ec) const { + ec.clear(); + std::unique_ptr ctx(EVP_MD_CTX_create(), EVP_MD_CTX_free); + if (!ctx) { + ec = error::signature_generation_error::create_context_failed; + return {}; + } + if (!EVP_DigestSignInit(ctx.get(), nullptr, nullptr, nullptr, pkey.get())) { + ec = error::signature_generation_error::signinit_failed; + return {}; + } + + size_t len = EVP_PKEY_size(pkey.get()); + std::string res(len, '\0'); + +// LibreSSL is the special kid in the block, as it does not support EVP_DigestSign. +// OpenSSL on the otherhand does not support using EVP_DigestSignUpdate for eddsa, which is why we end up with this +// mess. +#ifdef LIBRESSL_VERSION_NUMBER + ERR_clear_error(); + if (EVP_DigestSignUpdate(ctx.get(), reinterpret_cast(data.data()), data.size()) != + 1) { + std::cout << ERR_error_string(ERR_get_error(), NULL) << std::endl; + ec = error::signature_generation_error::signupdate_failed; + return {}; + } + if (EVP_DigestSignFinal(ctx.get(), reinterpret_cast(&res[0]), &len) != 1) { + ec = error::signature_generation_error::signfinal_failed; + return {}; + } +#else + if (EVP_DigestSign(ctx.get(), reinterpret_cast(&res[0]), &len, + reinterpret_cast(data.data()), data.size()) != 1) { + ec = error::signature_generation_error::signfinal_failed; + return {}; + } +#endif + + res.resize(len); + return res; + } + + /** + * Check if signature is valid + * \param data The data to check signature against + * \param signature Signature provided by the jwt + * \param ec Filled with details on error + */ + void verify(const std::string& data, const std::string& signature, std::error_code& ec) const { + ec.clear(); + std::unique_ptr ctx(EVP_MD_CTX_create(), EVP_MD_CTX_free); + if (!ctx) { + ec = error::signature_verification_error::create_context_failed; + return; + } + if (!EVP_DigestVerifyInit(ctx.get(), nullptr, nullptr, nullptr, pkey.get())) { + ec = error::signature_verification_error::verifyinit_failed; + return; + } +// LibreSSL is the special kid in the block, as it does not support EVP_DigestVerify. +// OpenSSL on the otherhand does not support using EVP_DigestVerifyUpdate for eddsa, which is why we end up with this +// mess. +#ifdef LIBRESSL_VERSION_NUMBER + if (EVP_DigestVerifyUpdate(ctx.get(), reinterpret_cast(data.data()), + data.size()) != 1) { + ec = error::signature_verification_error::verifyupdate_failed; + return; + } + if (EVP_DigestVerifyFinal(ctx.get(), reinterpret_cast(signature.data()), + signature.size()) != 1) { + ec = error::signature_verification_error::verifyfinal_failed; + return; + } +#else + auto res = EVP_DigestVerify(ctx.get(), reinterpret_cast(signature.data()), + signature.size(), reinterpret_cast(data.data()), + data.size()); + if (res != 1) { + ec = error::signature_verification_error::verifyfinal_failed; + return; + } +#endif + } + /** + * Returns the algorithm name provided to the constructor + * \return algorithm's name + */ + std::string name() const { return alg_name; } + + private: + /// OpenSSL struct containing keys + std::shared_ptr pkey; + /// Hash generator + const EVP_MD* (*md)(); + /// algorithm's name + const std::string alg_name; + }; +#endif + /** + * \brief Base class for PSS-RSA family of algorithms + */ + struct pss { + /** + * Construct new pss algorithm + * \param public_key RSA public key in PEM format + * \param private_key RSA private key or empty string if not available. If empty, signing will always fail. + * \param public_key_password Password to decrypt public key pem. + * \param private_key_password Password to decrypt private key pem. + * \param md Pointer to hash function + * \param name Name of the algorithm + */ + pss(const std::string& public_key, const std::string& private_key, const std::string& public_key_password, + const std::string& private_key_password, const EVP_MD* (*md)(), std::string name) + : md(md), alg_name(std::move(name)) { + if (!private_key.empty()) { + pkey = helper::load_private_key_from_string(private_key, private_key_password); + } else if (!public_key.empty()) { + pkey = helper::load_public_key_from_string(public_key, public_key_password); + } else + throw rsa_exception(error::rsa_error::no_key_provided); + } + + /** + * Sign jwt data + * \param data The data to sign + * \param ec error_code filled with details on error + * \return ECDSA signature for the given data + */ + std::string sign(const std::string& data, std::error_code& ec) const { + ec.clear(); + auto hash = this->generate_hash(data, ec); + if (ec) return {}; + + std::unique_ptr key(EVP_PKEY_get1_RSA(pkey.get()), RSA_free); + if (!key) { + ec = error::signature_generation_error::get_key_failed; + return {}; + } + const int size = RSA_size(key.get()); + + std::string padded(size, 0x00); + if (RSA_padding_add_PKCS1_PSS_mgf1( + key.get(), (unsigned char*)padded.data(), reinterpret_cast(hash.data()), + md(), md(), -1) == 0) { // NOLINT(google-readability-casting) requires `const_cast` + ec = error::signature_generation_error::rsa_padding_failed; + return {}; + } + + std::string res(size, 0x00); + if (RSA_private_encrypt(size, reinterpret_cast(padded.data()), + (unsigned char*)res.data(), key.get(), RSA_NO_PADDING) < + 0) { // NOLINT(google-readability-casting) requires `const_cast` + ec = error::signature_generation_error::rsa_private_encrypt_failed; + return {}; + } + return res; + } + + /** + * Check if signature is valid + * \param data The data to check signature against + * \param signature Signature provided by the jwt + * \param ec Filled with error details + */ + void verify(const std::string& data, const std::string& signature, std::error_code& ec) const { + ec.clear(); + auto hash = this->generate_hash(data, ec); + if (ec) return; + + std::unique_ptr key(EVP_PKEY_get1_RSA(pkey.get()), RSA_free); + if (!key) { + ec = error::signature_verification_error::get_key_failed; + return; + } + const int size = RSA_size(key.get()); + + std::string sig(size, 0x00); + if (RSA_public_decrypt( + static_cast(signature.size()), reinterpret_cast(signature.data()), + (unsigned char*)sig.data(), // NOLINT(google-readability-casting) requires `const_cast` + key.get(), RSA_NO_PADDING) == 0) { + ec = error::signature_verification_error::invalid_signature; + return; + } + + if (RSA_verify_PKCS1_PSS_mgf1(key.get(), reinterpret_cast(hash.data()), md(), + md(), reinterpret_cast(sig.data()), -1) == 0) { + ec = error::signature_verification_error::invalid_signature; + return; + } + } + /** + * Returns the algorithm name provided to the constructor + * \return algorithm's name + */ + std::string name() const { return alg_name; } + + private: + /** + * Hash the provided data using the hash function specified in constructor + * \param data Data to hash + * \return Hash of data + */ + std::string generate_hash(const std::string& data, std::error_code& ec) const { +#ifdef OPENSSL10 + std::unique_ptr ctx(EVP_MD_CTX_create(), + &EVP_MD_CTX_destroy); +#else + std::unique_ptr ctx(EVP_MD_CTX_new(), EVP_MD_CTX_free); +#endif + if (!ctx) { + ec = error::signature_generation_error::create_context_failed; + return {}; + } + if (EVP_DigestInit(ctx.get(), md()) == 0) { + ec = error::signature_generation_error::digestinit_failed; + return {}; + } + if (EVP_DigestUpdate(ctx.get(), data.data(), data.size()) == 0) { + ec = error::signature_generation_error::digestupdate_failed; + return {}; + } + unsigned int len = 0; + std::string res(EVP_MD_CTX_size(ctx.get()), '\0'); + if (EVP_DigestFinal(ctx.get(), (unsigned char*)res.data(), &len) == + 0) { // NOLINT(google-readability-casting) requires `const_cast` + ec = error::signature_generation_error::digestfinal_failed; + return {}; + } + res.resize(len); + return res; + } + + /// OpenSSL structure containing keys + std::shared_ptr pkey; + /// Hash generator function + const EVP_MD* (*md)(); + /// algorithm's name + const std::string alg_name; + }; + + /** + * HS256 algorithm + */ + struct hs256 : public hmacsha { + /** + * Construct new instance of algorithm + * \param key HMAC signing key + */ + explicit hs256(std::string key) : hmacsha(std::move(key), EVP_sha256, "HS256") {} + }; + /** + * HS384 algorithm + */ + struct hs384 : public hmacsha { + /** + * Construct new instance of algorithm + * \param key HMAC signing key + */ + explicit hs384(std::string key) : hmacsha(std::move(key), EVP_sha384, "HS384") {} + }; + /** + * HS512 algorithm + */ + struct hs512 : public hmacsha { + /** + * Construct new instance of algorithm + * \param key HMAC signing key + */ + explicit hs512(std::string key) : hmacsha(std::move(key), EVP_sha512, "HS512") {} + }; + /** + * RS256 algorithm + */ + struct rs256 : public rsa { + /** + * Construct new instance of algorithm + * \param public_key RSA public key in PEM format + * \param private_key RSA private key or empty string if not available. If empty, signing will always fail. + * \param public_key_password Password to decrypt public key pem. + * \param private_key_password Password to decrypt private key pem. + */ + explicit rs256(const std::string& public_key, const std::string& private_key = "", + const std::string& public_key_password = "", const std::string& private_key_password = "") + : rsa(public_key, private_key, public_key_password, private_key_password, EVP_sha256, "RS256") {} + }; + /** + * RS384 algorithm + */ + struct rs384 : public rsa { + /** + * Construct new instance of algorithm + * \param public_key RSA public key in PEM format + * \param private_key RSA private key or empty string if not available. If empty, signing will always fail. + * \param public_key_password Password to decrypt public key pem. + * \param private_key_password Password to decrypt private key pem. + */ + explicit rs384(const std::string& public_key, const std::string& private_key = "", + const std::string& public_key_password = "", const std::string& private_key_password = "") + : rsa(public_key, private_key, public_key_password, private_key_password, EVP_sha384, "RS384") {} + }; + /** + * RS512 algorithm + */ + struct rs512 : public rsa { + /** + * Construct new instance of algorithm + * \param public_key RSA public key in PEM format + * \param private_key RSA private key or empty string if not available. If empty, signing will always fail. + * \param public_key_password Password to decrypt public key pem. + * \param private_key_password Password to decrypt private key pem. + */ + explicit rs512(const std::string& public_key, const std::string& private_key = "", + const std::string& public_key_password = "", const std::string& private_key_password = "") + : rsa(public_key, private_key, public_key_password, private_key_password, EVP_sha512, "RS512") {} + }; + /** + * ES256 algorithm + */ + struct es256 : public ecdsa { + /** + * Construct new instance of algorithm + * \param public_key ECDSA public key in PEM format + * \param private_key ECDSA private key or empty string if not available. If empty, signing will always + * fail. + * \param public_key_password Password to decrypt public key pem. + * \param private_key_password Password + * to decrypt private key pem. + */ + explicit es256(const std::string& public_key, const std::string& private_key = "", + const std::string& public_key_password = "", const std::string& private_key_password = "") + : ecdsa(public_key, private_key, public_key_password, private_key_password, EVP_sha256, "ES256", 64) {} + }; + /** + * ES384 algorithm + */ + struct es384 : public ecdsa { + /** + * Construct new instance of algorithm + * \param public_key ECDSA public key in PEM format + * \param private_key ECDSA private key or empty string if not available. If empty, signing will always + * fail. + * \param public_key_password Password to decrypt public key pem. + * \param private_key_password Password + * to decrypt private key pem. + */ + explicit es384(const std::string& public_key, const std::string& private_key = "", + const std::string& public_key_password = "", const std::string& private_key_password = "") + : ecdsa(public_key, private_key, public_key_password, private_key_password, EVP_sha384, "ES384", 96) {} + }; + /** + * ES512 algorithm + */ + struct es512 : public ecdsa { + /** + * Construct new instance of algorithm + * \param public_key ECDSA public key in PEM format + * \param private_key ECDSA private key or empty string if not available. If empty, signing will always + * fail. + * \param public_key_password Password to decrypt public key pem. + * \param private_key_password Password + * to decrypt private key pem. + */ + explicit es512(const std::string& public_key, const std::string& private_key = "", + const std::string& public_key_password = "", const std::string& private_key_password = "") + : ecdsa(public_key, private_key, public_key_password, private_key_password, EVP_sha512, "ES512", 132) {} + }; + +#ifndef OPENSSL110 + /** + * Ed25519 algorithm + * + * Requires at least OpenSSL 1.1.1. + */ + struct ed25519 : public eddsa { + /** + * Construct new instance of algorithm + * \param public_key Ed25519 public key in PEM format + * \param private_key Ed25519 private key or empty string if not available. If empty, signing will always + * fail. + * \param public_key_password Password to decrypt public key pem. + * \param private_key_password Password + * to decrypt private key pem. + */ + explicit ed25519(const std::string& public_key, const std::string& private_key = "", + const std::string& public_key_password = "", const std::string& private_key_password = "") + : eddsa(public_key, private_key, public_key_password, private_key_password, EVP_sha512, "EdDSA") {} + }; + + /** + * Ed448 algorithm + * + * Requires at least OpenSSL 1.1.1. + */ + struct ed448 : public eddsa { + /** + * Construct new instance of algorithm + * \param public_key Ed448 public key in PEM format + * \param private_key Ed448 private key or empty string if not available. If empty, signing will always + * fail. + * \param public_key_password Password to decrypt public key pem. + * \param private_key_password Password + * to decrypt private key pem. + */ + explicit ed448(const std::string& public_key, const std::string& private_key = "", + const std::string& public_key_password = "", const std::string& private_key_password = "") + : eddsa(public_key, private_key, public_key_password, private_key_password, EVP_sha256, "EdDSA") {} + }; +#endif + + /** + * PS256 algorithm + */ + struct ps256 : public pss { + /** + * Construct new instance of algorithm + * \param public_key RSA public key in PEM format + * \param private_key RSA private key or empty string if not available. If empty, signing will always fail. + * \param public_key_password Password to decrypt public key pem. + * \param private_key_password Password to decrypt private key pem. + */ + explicit ps256(const std::string& public_key, const std::string& private_key = "", + const std::string& public_key_password = "", const std::string& private_key_password = "") + : pss(public_key, private_key, public_key_password, private_key_password, EVP_sha256, "PS256") {} + }; + /** + * PS384 algorithm + */ + struct ps384 : public pss { + /** + * Construct new instance of algorithm + * \param public_key RSA public key in PEM format + * \param private_key RSA private key or empty string if not available. If empty, signing will always fail. + * \param public_key_password Password to decrypt public key pem. + * \param private_key_password Password to decrypt private key pem. + */ + explicit ps384(const std::string& public_key, const std::string& private_key = "", + const std::string& public_key_password = "", const std::string& private_key_password = "") + : pss(public_key, private_key, public_key_password, private_key_password, EVP_sha384, "PS384") {} + }; + /** + * PS512 algorithm + */ + struct ps512 : public pss { + /** + * Construct new instance of algorithm + * \param public_key RSA public key in PEM format + * \param private_key RSA private key or empty string if not available. If empty, signing will always fail. + * \param public_key_password Password to decrypt public key pem. + * \param private_key_password Password to decrypt private key pem. + */ + explicit ps512(const std::string& public_key, const std::string& private_key = "", + const std::string& public_key_password = "", const std::string& private_key_password = "") + : pss(public_key, private_key, public_key_password, private_key_password, EVP_sha512, "PS512") {} + }; + } // namespace algorithm + + /** + * \brief JSON Abstractions for working with any library + */ + namespace json { + /** + * \brief Generic JSON types used in JWTs + * + * This enum is to abstract the third party underlying types + */ + enum class type { boolean, integer, number, string, array, object }; + } // namespace json + + namespace details { +#ifdef __cpp_lib_void_t + template + using void_t = std::void_t; +#else + // https://en.cppreference.com/w/cpp/types/void_t + template + struct make_void { + using type = void; + }; + + template + using void_t = typename make_void::type; +#endif + +#ifdef __cpp_lib_experimental_detect + template class _Op, typename... _Args> + using is_detected = std::experimental::is_detected<_Op, _Args...>; + + template class _Op, typename... _Args> + using is_detected_t = std::experimental::detected_t<_Op, _Args...>; +#else + struct nonesuch { + nonesuch() = delete; + ~nonesuch() = delete; + nonesuch(nonesuch const&) = delete; + nonesuch(nonesuch const&&) = delete; + void operator=(nonesuch const&) = delete; + void operator=(nonesuch&&) = delete; + }; + + // https://en.cppreference.com/w/cpp/experimental/is_detected + template class Op, class... Args> + struct detector { + using value = std::false_type; + using type = Default; + }; + + template class Op, class... Args> + struct detector>, Op, Args...> { + using value = std::true_type; + using type = Op; + }; + + template class Op, class... Args> + using is_detected = typename detector::value; + + template class Op, class... Args> + using is_detected_t = typename detector::type; +#endif + + template + using get_type_function = decltype(traits_type::get_type); + + template + using is_get_type_signature = + typename std::is_same, json::type(const value_type&)>; + + template + struct supports_get_type { + static constexpr auto value = is_detected::value && + std::is_function>::value && + is_get_type_signature::value; + }; + + template + using as_object_function = decltype(traits_type::as_object); + + template + using is_as_object_signature = + typename std::is_same, object_type(const value_type&)>; + + template + struct supports_as_object { + static constexpr auto value = std::is_constructible::value && + is_detected::value && + std::is_function>::value && + is_as_object_signature::value; + }; + + template + using as_array_function = decltype(traits_type::as_array); + + template + using is_as_array_signature = + typename std::is_same, array_type(const value_type&)>; + + template + struct supports_as_array { + static constexpr auto value = std::is_constructible::value && + is_detected::value && + std::is_function>::value && + is_as_array_signature::value; + }; + + template + using as_string_function = decltype(traits_type::as_string); + + template + using is_as_string_signature = + typename std::is_same, string_type(const value_type&)>; + + template + struct supports_as_string { + static constexpr auto value = std::is_constructible::value && + is_detected::value && + std::is_function>::value && + is_as_string_signature::value; + }; + + template + using as_number_function = decltype(traits_type::as_number); + + template + using is_as_number_signature = + typename std::is_same, number_type(const value_type&)>; + + template + struct supports_as_number { + static constexpr auto value = std::is_floating_point::value && + std::is_constructible::value && + is_detected::value && + std::is_function>::value && + is_as_number_signature::value; + }; + + template + using as_integer_function = decltype(traits_type::as_int); + + template + using is_as_integer_signature = + typename std::is_same, integer_type(const value_type&)>; + + template + struct supports_as_integer { + static constexpr auto value = std::is_signed::value && + !std::is_floating_point::value && + std::is_constructible::value && + is_detected::value && + std::is_function>::value && + is_as_integer_signature::value; + }; + + template + using as_boolean_function = decltype(traits_type::as_bool); + + template + using is_as_boolean_signature = + typename std::is_same, boolean_type(const value_type&)>; + + template + struct supports_as_boolean { + static constexpr auto value = std::is_convertible::value && + std::is_constructible::value && + is_detected::value && + std::is_function>::value && + is_as_boolean_signature::value; + }; + + template + struct is_valid_traits { + // Internal assertions for better feedback + static_assert(supports_get_type::value, + "traits must provide `jwt::json::type get_type(const value_type&)`"); + static_assert(supports_as_object::value, + "traits must provide `object_type as_object(const value_type&)`"); + static_assert(supports_as_array::value, + "traits must provide `array_type as_array(const value_type&)`"); + static_assert(supports_as_string::value, + "traits must provide `string_type as_string(const value_type&)`"); + static_assert(supports_as_number::value, + "traits must provide `number_type as_number(const value_type&)`"); + static_assert( + supports_as_integer::value, + "traits must provide `integer_type as_int(const value_type&)`"); + static_assert( + supports_as_boolean::value, + "traits must provide `boolean_type as_bool(const value_type&)`"); + + static constexpr auto value = + supports_get_type::value && + supports_as_object::value && + supports_as_array::value && + supports_as_string::value && + supports_as_number::value && + supports_as_integer::value && + supports_as_boolean::value; + }; + + template + struct is_valid_json_value { + static constexpr auto value = + std::is_default_constructible::value && + std::is_constructible::value && // a more generic is_copy_constructible + std::is_move_constructible::value && std::is_assignable::value && + std::is_copy_assignable::value && std::is_move_assignable::value; + // TODO(cmcarthur): Stream operators + }; + + template + using has_mapped_type = typename traits_type::mapped_type; + + template + using has_key_type = typename traits_type::key_type; + + template + using has_value_type = typename traits_type::value_type; + + template + using has_iterator = typename object_type::iterator; + + template + using has_const_iterator = typename object_type::const_iterator; + + template + using is_begin_signature = + typename std::is_same().begin()), has_iterator>; + + template + using is_begin_const_signature = + typename std::is_same().begin()), has_const_iterator>; + + template + struct supports_begin { + static constexpr auto value = + is_detected::value && is_detected::value && + is_begin_signature::value && is_begin_const_signature::value; + }; + + template + using is_end_signature = + typename std::is_same().end()), has_iterator>; + + template + using is_end_const_signature = + typename std::is_same().end()), has_const_iterator>; + + template + struct supports_end { + static constexpr auto value = + is_detected::value && is_detected::value && + is_end_signature::value && is_end_const_signature::value; + }; + + template + using is_count_signature = typename std::is_integral().count(std::declval()))>; + + template + using is_subcription_operator_signature = + typename std::is_same()[std::declval()]), + value_type&>; + + template + using is_at_const_signature = + typename std::is_same().at(std::declval())), + const value_type&>; + + template + struct is_valid_json_object { + static constexpr auto value = + is_detected::value && + std::is_same::value && + is_detected::value && + std::is_same::value && + supports_begin::value && supports_end::value && + is_count_signature::value && + is_subcription_operator_signature::value && + is_at_const_signature::value; + + static constexpr auto supports_claims_transform = + value && is_detected::value && + std::is_same>::value; + }; + + template + struct is_valid_json_array { + static constexpr auto value = std::is_same::value; + }; + + template + struct is_valid_json_types { + // Internal assertions for better feedback + static_assert(is_valid_json_value::value, + "value type must meet basic requirements, default constructor, copyable, moveable"); + static_assert(is_valid_json_object::value, + "object_type must be a string_type to value_type container"); + static_assert(is_valid_json_array::value, + "array_type must be a container of value_type"); + + static constexpr auto value = is_valid_json_object::value && + is_valid_json_value::value && + is_valid_json_array::value; + }; + } // namespace details + + /** + * \brief a class to store a generic JSON value as claim + * + * The default template parameters use [picojson](https://github.com/kazuho/picojson) + * + * \tparam json_traits : JSON implementation traits + * + * \see [RFC 7519: JSON Web Token (JWT)](https://tools.ietf.org/html/rfc7519) + */ + template + class basic_claim { + /** + * The reason behind this is to provide an expressive abstraction without + * over complexifying the API. For more information take the time to read + * https://github.com/nlohmann/json/issues/774. It maybe be expanded to + * support custom string types. + */ + static_assert(std::is_same::value, + "string_type must be a std::string."); + + static_assert( + details::is_valid_json_types::value, + "must staisfy json container requirements"); + static_assert(details::is_valid_traits::value, "traits must satisfy requirements"); + + typename json_traits::value_type val; + + public: + using set_t = std::set; + + basic_claim() = default; + basic_claim(const basic_claim&) = default; + basic_claim(basic_claim&&) = default; + basic_claim& operator=(const basic_claim&) = default; + basic_claim& operator=(basic_claim&&) = default; + ~basic_claim() = default; + + JWT_CLAIM_EXPLICIT basic_claim(typename json_traits::string_type s) : val(std::move(s)) {} + JWT_CLAIM_EXPLICIT basic_claim(const date& d) + : val(typename json_traits::integer_type(std::chrono::system_clock::to_time_t(d))) {} + JWT_CLAIM_EXPLICIT basic_claim(typename json_traits::array_type a) : val(std::move(a)) {} + JWT_CLAIM_EXPLICIT basic_claim(typename json_traits::value_type v) : val(std::move(v)) {} + JWT_CLAIM_EXPLICIT basic_claim(const set_t& s) : val(typename json_traits::array_type(s.begin(), s.end())) {} + template + basic_claim(Iterator begin, Iterator end) : val(typename json_traits::array_type(begin, end)) {} + + /** + * Get wrapped JSON value + * \return Wrapped JSON value + */ + typename json_traits::value_type to_json() const { return val; } + + /** + * Parse input stream into underlying JSON value + * \return input stream + */ + std::istream& operator>>(std::istream& is) { return is >> val; } + + /** + * Serialize claim to output stream from wrapped JSON value + * \return ouput stream + */ + std::ostream& operator<<(std::ostream& os) { return os << val; } + + /** + * Get type of contained JSON value + * \return Type + * \throw std::logic_error An internal error occured + */ + json::type get_type() const { return json_traits::get_type(val); } + + /** + * Get the contained JSON value as a string + * \return content as string + * \throw std::bad_cast Content was not a string + */ + typename json_traits::string_type as_string() const { return json_traits::as_string(val); } + + /** + * Get the contained JSON value as a date + * \return content as date + * \throw std::bad_cast Content was not a date + */ + date as_date() const { return std::chrono::system_clock::from_time_t(as_int()); } + + /** + * Get the contained JSON value as an array + * \return content as array + * \throw std::bad_cast Content was not an array + */ + typename json_traits::array_type as_array() const { return json_traits::as_array(val); } + + /** + * Get the contained JSON value as a set of strings + * \return content as set of strings + * \throw std::bad_cast Content was not an array of string + */ + set_t as_set() const { + set_t res; + for (const auto& e : json_traits::as_array(val)) { + res.insert(json_traits::as_string(e)); + } + return res; + } + + /** + * Get the contained JSON value as an integer + * \return content as int + * \throw std::bad_cast Content was not an int + */ + typename json_traits::integer_type as_int() const { return json_traits::as_int(val); } + + /** + * Get the contained JSON value as a bool + * \return content as bool + * \throw std::bad_cast Content was not a bool + */ + typename json_traits::boolean_type as_bool() const { return json_traits::as_bool(val); } + + /** + * Get the contained JSON value as a number + * \return content as double + * \throw std::bad_cast Content was not a number + */ + typename json_traits::number_type as_number() const { return json_traits::as_number(val); } + }; + + namespace error { + struct invalid_json_exception : public std::runtime_error { + invalid_json_exception() : runtime_error("invalid json") {} + }; + struct claim_not_present_exception : public std::out_of_range { + claim_not_present_exception() : out_of_range("claim not found") {} + }; + } // namespace error + + namespace details { + template + class map_of_claims { + typename json_traits::object_type claims; + + public: + using basic_claim_t = basic_claim; + using iterator = typename json_traits::object_type::iterator; + using const_iterator = typename json_traits::object_type::const_iterator; + + map_of_claims() = default; + map_of_claims(const map_of_claims&) = default; + map_of_claims(map_of_claims&&) = default; + map_of_claims& operator=(const map_of_claims&) = default; + map_of_claims& operator=(map_of_claims&&) = default; + + map_of_claims(typename json_traits::object_type json) : claims(std::move(json)) {} + + iterator begin() { return claims.begin(); } + iterator end() { return claims.end(); } + const_iterator cbegin() const { return claims.begin(); } + const_iterator cend() const { return claims.end(); } + const_iterator begin() const { return claims.begin(); } + const_iterator end() const { return claims.end(); } + + /** + * \brief Parse a JSON string into a map of claims + * + * The implication is that a "map of claims" is identic to a JSON object + * + * \param str JSON data to be parse as an object + * \return content as JSON object + */ + static typename json_traits::object_type parse_claims(const typename json_traits::string_type& str) { + typename json_traits::value_type val; + if (!json_traits::parse(val, str)) throw error::invalid_json_exception(); + + return json_traits::as_object(val); + }; + + /** + * Check if a claim is present in the map + * \return true if claim was present, false otherwise + */ + bool has_claim(const typename json_traits::string_type& name) const noexcept { + return claims.count(name) != 0; + } + + /** + * Get a claim by name + * + * \param name the name of the desired claim + * \return Requested claim + * \throw jwt::error::claim_not_present_exception if the claim was not present + */ + basic_claim_t get_claim(const typename json_traits::string_type& name) const { + if (!has_claim(name)) throw error::claim_not_present_exception(); + return basic_claim_t{claims.at(name)}; + } + + std::unordered_map get_claims() const { + static_assert( + details::is_valid_json_object::supports_claims_transform, + "currently there is a limitation on the internal implemantation of the `object_type` to have an " + "`std::pair` like `value_type`"); + + std::unordered_map res; + std::transform(claims.begin(), claims.end(), std::inserter(res, res.end()), + [](const typename json_traits::object_type::value_type& val) { + return std::make_pair(val.first, basic_claim_t{val.second}); + }); + return res; + } + }; + } // namespace details + + /** + * Base class that represents a token payload. + * Contains Convenience accessors for common claims. + */ + template + class payload { + protected: + details::map_of_claims payload_claims; + + public: + using basic_claim_t = basic_claim; + + /** + * Check if issuer is present ("iss") + * \return true if present, false otherwise + */ + bool has_issuer() const noexcept { return has_payload_claim("iss"); } + /** + * Check if subject is present ("sub") + * \return true if present, false otherwise + */ + bool has_subject() const noexcept { return has_payload_claim("sub"); } + /** + * Check if audience is present ("aud") + * \return true if present, false otherwise + */ + bool has_audience() const noexcept { return has_payload_claim("aud"); } + /** + * Check if expires is present ("exp") + * \return true if present, false otherwise + */ + bool has_expires_at() const noexcept { return has_payload_claim("exp"); } + /** + * Check if not before is present ("nbf") + * \return true if present, false otherwise + */ + bool has_not_before() const noexcept { return has_payload_claim("nbf"); } + /** + * Check if issued at is present ("iat") + * \return true if present, false otherwise + */ + bool has_issued_at() const noexcept { return has_payload_claim("iat"); } + /** + * Check if token id is present ("jti") + * \return true if present, false otherwise + */ + bool has_id() const noexcept { return has_payload_claim("jti"); } + /** + * Get issuer claim + * \return issuer as string + * \throw std::runtime_error If claim was not present + * \throw std::bad_cast Claim was present but not a string (Should not happen in a valid token) + */ + typename json_traits::string_type get_issuer() const { return get_payload_claim("iss").as_string(); } + /** + * Get subject claim + * \return subject as string + * \throw std::runtime_error If claim was not present + * \throw std::bad_cast Claim was present but not a string (Should not happen in a valid token) + */ + typename json_traits::string_type get_subject() const { return get_payload_claim("sub").as_string(); } + /** + * Get audience claim + * \return audience as a set of strings + * \throw std::runtime_error If claim was not present + * \throw std::bad_cast Claim was present but not a set (Should not happen in a valid token) + */ + typename basic_claim_t::set_t get_audience() const { + auto aud = get_payload_claim("aud"); + if (aud.get_type() == json::type::string) return {aud.as_string()}; + + return aud.as_set(); + } + /** + * Get expires claim + * \return expires as a date in utc + * \throw std::runtime_error If claim was not present + * \throw std::bad_cast Claim was present but not a date (Should not happen in a valid token) + */ + date get_expires_at() const { return get_payload_claim("exp").as_date(); } + /** + * Get not valid before claim + * \return nbf date in utc + * \throw std::runtime_error If claim was not present + * \throw std::bad_cast Claim was present but not a date (Should not happen in a valid token) + */ + date get_not_before() const { return get_payload_claim("nbf").as_date(); } + /** + * Get issued at claim + * \return issued at as date in utc + * \throw std::runtime_error If claim was not present + * \throw std::bad_cast Claim was present but not a date (Should not happen in a valid token) + */ + date get_issued_at() const { return get_payload_claim("iat").as_date(); } + /** + * Get id claim + * \return id as string + * \throw std::runtime_error If claim was not present + * \throw std::bad_cast Claim was present but not a string (Should not happen in a valid token) + */ + typename json_traits::string_type get_id() const { return get_payload_claim("jti").as_string(); } + /** + * Check if a payload claim is present + * \return true if claim was present, false otherwise + */ + bool has_payload_claim(const typename json_traits::string_type& name) const noexcept { + return payload_claims.has_claim(name); + } + /** + * Get payload claim + * \return Requested claim + * \throw std::runtime_error If claim was not present + */ + basic_claim_t get_payload_claim(const typename json_traits::string_type& name) const { + return payload_claims.get_claim(name); + } + }; + + /** + * Base class that represents a token header. + * Contains Convenience accessors for common claims. + */ + template + class header { + protected: + details::map_of_claims header_claims; + + public: + using basic_claim_t = basic_claim; + /** + * Check if algortihm is present ("alg") + * \return true if present, false otherwise + */ + bool has_algorithm() const noexcept { return has_header_claim("alg"); } + /** + * Check if type is present ("typ") + * \return true if present, false otherwise + */ + bool has_type() const noexcept { return has_header_claim("typ"); } + /** + * Check if content type is present ("cty") + * \return true if present, false otherwise + */ + bool has_content_type() const noexcept { return has_header_claim("cty"); } + /** + * Check if key id is present ("kid") + * \return true if present, false otherwise + */ + bool has_key_id() const noexcept { return has_header_claim("kid"); } + /** + * Get algorithm claim + * \return algorithm as string + * \throw std::runtime_error If claim was not present + * \throw std::bad_cast Claim was present but not a string (Should not happen in a valid token) + */ + typename json_traits::string_type get_algorithm() const { return get_header_claim("alg").as_string(); } + /** + * Get type claim + * \return type as a string + * \throw std::runtime_error If claim was not present + * \throw std::bad_cast Claim was present but not a string (Should not happen in a valid token) + */ + typename json_traits::string_type get_type() const { return get_header_claim("typ").as_string(); } + /** + * Get content type claim + * \return content type as string + * \throw std::runtime_error If claim was not present + * \throw std::bad_cast Claim was present but not a string (Should not happen in a valid token) + */ + typename json_traits::string_type get_content_type() const { return get_header_claim("cty").as_string(); } + /** + * Get key id claim + * \return key id as string + * \throw std::runtime_error If claim was not present + * \throw std::bad_cast Claim was present but not a string (Should not happen in a valid token) + */ + typename json_traits::string_type get_key_id() const { return get_header_claim("kid").as_string(); } + /** + * Check if a header claim is present + * \return true if claim was present, false otherwise + */ + bool has_header_claim(const typename json_traits::string_type& name) const noexcept { + return header_claims.has_claim(name); + } + /** + * Get header claim + * \return Requested claim + * \throw std::runtime_error If claim was not present + */ + basic_claim_t get_header_claim(const typename json_traits::string_type& name) const { + return header_claims.get_claim(name); + } + }; + + /** + * Class containing all information about a decoded token + */ + template + class decoded_jwt : public header, public payload { + protected: + /// Unmodifed token, as passed to constructor + const typename json_traits::string_type token; + /// Header part decoded from base64 + typename json_traits::string_type header; + /// Unmodified header part in base64 + typename json_traits::string_type header_base64; + /// Payload part decoded from base64 + typename json_traits::string_type payload; + /// Unmodified payload part in base64 + typename json_traits::string_type payload_base64; + /// Signature part decoded from base64 + typename json_traits::string_type signature; + /// Unmodified signature part in base64 + typename json_traits::string_type signature_base64; + + public: + using basic_claim_t = basic_claim; +#ifndef JWT_DISABLE_BASE64 + /** + * \brief Parses a given token + * + * \note Decodes using the jwt::base64url which supports an std::string + * + * \param token The token to parse + * \throw std::invalid_argument Token is not in correct format + * \throw std::runtime_error Base64 decoding failed or invalid json + */ + JWT_CLAIM_EXPLICIT decoded_jwt(const typename json_traits::string_type& token) + : decoded_jwt(token, [](const typename json_traits::string_type& token) { + return base::decode(base::pad(token)); + }) {} +#endif + /** + * \brief Parses a given token + * + * \tparam Decode is callabled, taking a string_type and returns a string_type. + * It should ensure the padding of the input and then base64url decode and + * return the results. + * \param token The token to parse + * \param decode The function to decode the token + * \throw std::invalid_argument Token is not in correct format + * \throw std::runtime_error Base64 decoding failed or invalid json + */ + template + decoded_jwt(const typename json_traits::string_type& token, Decode decode) : token(token) { + auto hdr_end = token.find('.'); + if (hdr_end == json_traits::string_type::npos) throw std::invalid_argument("invalid token supplied"); + auto payload_end = token.find('.', hdr_end + 1); + if (payload_end == json_traits::string_type::npos) throw std::invalid_argument("invalid token supplied"); + header_base64 = token.substr(0, hdr_end); + payload_base64 = token.substr(hdr_end + 1, payload_end - hdr_end - 1); + signature_base64 = token.substr(payload_end + 1); + + header = decode(header_base64); + payload = decode(payload_base64); + signature = decode(signature_base64); + + this->header_claims = details::map_of_claims::parse_claims(header); + this->payload_claims = details::map_of_claims::parse_claims(payload); + } + + /** + * Get token string, as passed to constructor + * \return token as passed to constructor + */ + const typename json_traits::string_type& get_token() const noexcept { return token; } + /** + * Get header part as json string + * \return header part after base64 decoding + */ + const typename json_traits::string_type& get_header() const noexcept { return header; } + /** + * Get payload part as json string + * \return payload part after base64 decoding + */ + const typename json_traits::string_type& get_payload() const noexcept { return payload; } + /** + * Get signature part as json string + * \return signature part after base64 decoding + */ + const typename json_traits::string_type& get_signature() const noexcept { return signature; } + /** + * Get header part as base64 string + * \return header part before base64 decoding + */ + const typename json_traits::string_type& get_header_base64() const noexcept { return header_base64; } + /** + * Get payload part as base64 string + * \return payload part before base64 decoding + */ + const typename json_traits::string_type& get_payload_base64() const noexcept { return payload_base64; } + /** + * Get signature part as base64 string + * \return signature part before base64 decoding + */ + const typename json_traits::string_type& get_signature_base64() const noexcept { return signature_base64; } + /** + * Get all payload claims + * \return map of claims + */ + std::unordered_map get_payload_claims() const { + return this->payload_claims.get_claims(); + } + /** + * Get all header claims + * \return map of claims + */ + std::unordered_map get_header_claims() const { + return this->header_claims.get_claims(); + } + }; + + /** + * Builder class to build and sign a new token + * Use jwt::create() to get an instance of this class. + */ + template + class builder { + typename json_traits::object_type header_claims; + typename json_traits::object_type payload_claims; + + public: + builder() = default; + /** + * Set a header claim. + * \param id Name of the claim + * \param c Claim to add + * \return *this to allow for method chaining + */ + builder& set_header_claim(const typename json_traits::string_type& id, typename json_traits::value_type c) { + header_claims[id] = std::move(c); + return *this; + } + + /** + * Set a header claim. + * \param id Name of the claim + * \param c Claim to add + * \return *this to allow for method chaining + */ + builder& set_header_claim(const typename json_traits::string_type& id, basic_claim c) { + header_claims[id] = c.to_json(); + return *this; + } + /** + * Set a payload claim. + * \param id Name of the claim + * \param c Claim to add + * \return *this to allow for method chaining + */ + builder& set_payload_claim(const typename json_traits::string_type& id, typename json_traits::value_type c) { + payload_claims[id] = std::move(c); + return *this; + } + /** + * Set a payload claim. + * \param id Name of the claim + * \param c Claim to add + * \return *this to allow for method chaining + */ + builder& set_payload_claim(const typename json_traits::string_type& id, basic_claim c) { + payload_claims[id] = c.to_json(); + return *this; + } + /** + * Set algorithm claim + * You normally don't need to do this, as the algorithm is automatically set if you don't change it. + * \param str Name of algorithm + * \return *this to allow for method chaining + */ + builder& set_algorithm(typename json_traits::string_type str) { + return set_header_claim("alg", typename json_traits::value_type(str)); + } + /** + * Set type claim + * \param str Type to set + * \return *this to allow for method chaining + */ + builder& set_type(typename json_traits::string_type str) { + return set_header_claim("typ", typename json_traits::value_type(str)); + } + /** + * Set content type claim + * \param str Type to set + * \return *this to allow for method chaining + */ + builder& set_content_type(typename json_traits::string_type str) { + return set_header_claim("cty", typename json_traits::value_type(str)); + } + /** + * Set key id claim + * \param str Key id to set + * \return *this to allow for method chaining + */ + builder& set_key_id(typename json_traits::string_type str) { + return set_header_claim("kid", typename json_traits::value_type(str)); + } + /** + * Set issuer claim + * \param str Issuer to set + * \return *this to allow for method chaining + */ + builder& set_issuer(typename json_traits::string_type str) { + return set_payload_claim("iss", typename json_traits::value_type(str)); + } + /** + * Set subject claim + * \param str Subject to set + * \return *this to allow for method chaining + */ + builder& set_subject(typename json_traits::string_type str) { + return set_payload_claim("sub", typename json_traits::value_type(str)); + } + /** + * Set audience claim + * \param a Audience set + * \return *this to allow for method chaining + */ + builder& set_audience(typename json_traits::array_type a) { + return set_payload_claim("aud", typename json_traits::value_type(a)); + } + /** + * Set audience claim + * \param aud Single audience + * \return *this to allow for method chaining + */ + builder& set_audience(typename json_traits::string_type aud) { + return set_payload_claim("aud", typename json_traits::value_type(aud)); + } + /** + * Set expires at claim + * \param d Expires time + * \return *this to allow for method chaining + */ + builder& set_expires_at(const date& d) { return set_payload_claim("exp", basic_claim(d)); } + /** + * Set not before claim + * \param d First valid time + * \return *this to allow for method chaining + */ + builder& set_not_before(const date& d) { return set_payload_claim("nbf", basic_claim(d)); } + /** + * Set issued at claim + * \param d Issued at time, should be current time + * \return *this to allow for method chaining + */ + builder& set_issued_at(const date& d) { return set_payload_claim("iat", basic_claim(d)); } + /** + * Set id claim + * \param str ID to set + * \return *this to allow for method chaining + */ + builder& set_id(const typename json_traits::string_type& str) { + return set_payload_claim("jti", typename json_traits::value_type(str)); + } + + /** + * Sign token and return result + * \tparam Algo Callable method which takes a string_type and return the signed input as a string_type + * \tparam Encode Callable method which takes a string_type and base64url safe encodes it, + * MUST return the result with no padding; trim the result. + * \param algo Instance of an algorithm to sign the token with + * \param encode Callable to transform the serialized json to base64 with no padding + * \return Final token as a string + * + * \note If the 'alg' header in not set in the token it will be set to `algo.name()` + */ + template + typename json_traits::string_type sign(const Algo& algo, Encode encode) const { + std::error_code ec; + auto res = sign(algo, encode, ec); + error::throw_if_error(ec); + return res; + } +#ifndef JWT_DISABLE_BASE64 + /** + * Sign token and return result + * + * using the `jwt::base` functions provided + * + * \param algo Instance of an algorithm to sign the token with + * \return Final token as a string + */ + template + typename json_traits::string_type sign(const Algo& algo) const { + std::error_code ec; + auto res = sign(algo, ec); + error::throw_if_error(ec); + return res; + } +#endif + + /** + * Sign token and return result + * \tparam Algo Callable method which takes a string_type and return the signed input as a string_type + * \tparam Encode Callable method which takes a string_type and base64url safe encodes it, + * MUST return the result with no padding; trim the result. + * \param algo Instance of an algorithm to sign the token with + * \param encode Callable to transform the serialized json to base64 with no padding + * \param ec error_code filled with details on error + * \return Final token as a string + * + * \note If the 'alg' header in not set in the token it will be set to `algo.name()` + */ + template + typename json_traits::string_type sign(const Algo& algo, Encode encode, std::error_code& ec) const { + // make a copy such that a builder can be re-used + typename json_traits::object_type obj_header = header_claims; + if (header_claims.count("alg") == 0) obj_header["alg"] = typename json_traits::value_type(algo.name()); + + const auto header = encode(json_traits::serialize(typename json_traits::value_type(obj_header))); + const auto payload = encode(json_traits::serialize(typename json_traits::value_type(payload_claims))); + const auto token = header + "." + payload; + + auto signature = algo.sign(token, ec); + if (ec) return {}; + + return token + "." + encode(signature); + } +#ifndef JWT_DISABLE_BASE64 + /** + * Sign token and return result + * + * using the `jwt::base` functions provided + * + * \param algo Instance of an algorithm to sign the token with + * \param ec error_code filled with details on error + * \return Final token as a string + */ + template + typename json_traits::string_type sign(const Algo& algo, std::error_code& ec) const { + return sign( + algo, + [](const typename json_traits::string_type& data) { + return base::trim(base::encode(data)); + }, + ec); + } +#endif + }; + + /** + * Verifier class used to check if a decoded token contains all claims required by your application and has a valid + * signature. + */ + template + class verifier { + struct algo_base { + virtual ~algo_base() = default; + virtual void verify(const std::string& data, const std::string& sig, std::error_code& ec) = 0; + }; + template + struct algo : public algo_base { + T alg; + explicit algo(T a) : alg(a) {} + void verify(const std::string& data, const std::string& sig, std::error_code& ec) override { + alg.verify(data, sig, ec); + } + }; + + using basic_claim_t = basic_claim; + /// Required claims + std::unordered_map claims; + /// Leeway time for exp, nbf and iat + size_t default_leeway = 0; + /// Instance of clock type + Clock clock; + /// Supported algorithms + std::unordered_map> algs; + + public: + /** + * Constructor for building a new verifier instance + * \param c Clock instance + */ + explicit verifier(Clock c) : clock(c) {} + + /** + * Set default leeway to use. + * \param leeway Default leeway to use if not specified otherwise + * \return *this to allow chaining + */ + verifier& leeway(size_t leeway) { + default_leeway = leeway; + return *this; + } + /** + * Set leeway for expires at. + * If not specified the default leeway will be used. + * \param leeway Set leeway to use for expires at. + * \return *this to allow chaining + */ + verifier& expires_at_leeway(size_t leeway) { + return with_claim("exp", basic_claim_t(std::chrono::system_clock::from_time_t(leeway))); + } + /** + * Set leeway for not before. + * If not specified the default leeway will be used. + * \param leeway Set leeway to use for not before. + * \return *this to allow chaining + */ + verifier& not_before_leeway(size_t leeway) { + return with_claim("nbf", basic_claim_t(std::chrono::system_clock::from_time_t(leeway))); + } + /** + * Set leeway for issued at. + * If not specified the default leeway will be used. + * \param leeway Set leeway to use for issued at. + * \return *this to allow chaining + */ + verifier& issued_at_leeway(size_t leeway) { + return with_claim("iat", basic_claim_t(std::chrono::system_clock::from_time_t(leeway))); + } + /** + * Set an issuer to check for. + * Check is casesensitive. + * \param iss Issuer to check for. + * \return *this to allow chaining + */ + verifier& with_issuer(const typename json_traits::string_type& iss) { + return with_claim("iss", basic_claim_t(iss)); + } + /** + * Set a subject to check for. + * Check is casesensitive. + * \param sub Subject to check for. + * \return *this to allow chaining + */ + verifier& with_subject(const typename json_traits::string_type& sub) { + return with_claim("sub", basic_claim_t(sub)); + } + /** + * Set an audience to check for. + * If any of the specified audiences is not present in the token the check fails. + * \param aud Audience to check for. + * \return *this to allow chaining + */ + verifier& with_audience(const typename basic_claim_t::set_t& aud) { + return with_claim("aud", basic_claim_t(aud)); + } + /** + * Set an audience to check for. + * If the specified audiences is not present in the token the check fails. + * \param aud Audience to check for. + * \return *this to allow chaining + */ + verifier& with_audience(const typename json_traits::string_type& aud) { + return with_claim("aud", basic_claim_t(aud)); + } + /** + * Set an id to check for. + * Check is casesensitive. + * \param id ID to check for. + * \return *this to allow chaining + */ + verifier& with_id(const typename json_traits::string_type& id) { return with_claim("jti", basic_claim_t(id)); } + /** + * Specify a claim to check for. + * \param name Name of the claim to check for + * \param c Claim to check for + * \return *this to allow chaining + */ + verifier& with_claim(const typename json_traits::string_type& name, basic_claim_t c) { + claims[name] = c; + return *this; + } + + /** + * Add an algorithm available for checking. + * \param alg Algorithm to allow + * \return *this to allow chaining + */ + template + verifier& allow_algorithm(Algorithm alg) { + algs[alg.name()] = std::make_shared>(alg); + return *this; + } + + /** + * Verify the given token. + * \param jwt Token to check + * \throw token_verification_exception Verification failed + */ + void verify(const decoded_jwt& jwt) const { + std::error_code ec; + verify(jwt, ec); + error::throw_if_error(ec); + } + /** + * Verify the given token. + * \param jwt Token to check + * \param ec error_code filled with details on error + */ + void verify(const decoded_jwt& jwt, std::error_code& ec) const { + ec.clear(); + const typename json_traits::string_type data = jwt.get_header_base64() + "." + jwt.get_payload_base64(); + const typename json_traits::string_type sig = jwt.get_signature(); + const std::string algo = jwt.get_algorithm(); + if (algs.count(algo) == 0) { + ec = error::token_verification_error::wrong_algorithm; + return; + } + algs.at(algo)->verify(data, sig, ec); + if (ec) return; + + auto assert_claim_eq = [](const decoded_jwt& jwt, const typename json_traits::string_type& key, + const basic_claim_t& c, std::error_code& ec) { + if (!jwt.has_payload_claim(key)) { + ec = error::token_verification_error::missing_claim; + return; + } + auto jc = jwt.get_payload_claim(key); + if (jc.get_type() != c.get_type()) { + ec = error::token_verification_error::claim_type_missmatch; + return; + } + if (c.get_type() == json::type::integer) { + if (c.as_date() != jc.as_date()) { + ec = error::token_verification_error::claim_value_missmatch; + return; + } + } else if (c.get_type() == json::type::array) { + auto s1 = c.as_set(); + auto s2 = jc.as_set(); + if (s1.size() != s2.size()) { + ec = error::token_verification_error::claim_value_missmatch; + return; + } + auto it1 = s1.cbegin(); + auto it2 = s2.cbegin(); + while (it1 != s1.cend() && it2 != s2.cend()) { + if (*it1++ != *it2++) { + ec = error::token_verification_error::claim_value_missmatch; + return; + } + } + } else if (c.get_type() == json::type::object) { + if (json_traits::serialize(c.to_json()) != json_traits::serialize(jc.to_json())) { + ec = error::token_verification_error::claim_value_missmatch; + return; + } + } else if (c.get_type() == json::type::string) { + if (c.as_string() != jc.as_string()) { + ec = error::token_verification_error::claim_value_missmatch; + return; + } + } else + throw std::logic_error("internal error, should be unreachable"); + }; + + auto time = clock.now(); + + if (jwt.has_expires_at()) { + auto leeway = claims.count("exp") == 1 + ? std::chrono::system_clock::to_time_t(claims.at("exp").as_date()) + : default_leeway; + auto exp = jwt.get_expires_at(); + if (time > exp + std::chrono::seconds(leeway)) { + ec = error::token_verification_error::token_expired; + return; + } + } + if (jwt.has_issued_at()) { + auto leeway = claims.count("iat") == 1 + ? std::chrono::system_clock::to_time_t(claims.at("iat").as_date()) + : default_leeway; + auto iat = jwt.get_issued_at(); + if (time < iat - std::chrono::seconds(leeway)) { + ec = error::token_verification_error::token_expired; + return; + } + } + if (jwt.has_not_before()) { + auto leeway = claims.count("nbf") == 1 + ? std::chrono::system_clock::to_time_t(claims.at("nbf").as_date()) + : default_leeway; + auto nbf = jwt.get_not_before(); + if (time < nbf - std::chrono::seconds(leeway)) { + ec = error::token_verification_error::token_expired; + return; + } + } + for (auto& c : claims) { + if (c.first == "exp" || c.first == "iat" || c.first == "nbf") { + // Nothing to do here, already checked + } else if (c.first == "aud") { + if (!jwt.has_audience()) { + ec = error::token_verification_error::audience_missmatch; + return; + } + auto aud = jwt.get_audience(); + typename basic_claim_t::set_t expected = {}; + if (c.second.get_type() == json::type::string) + expected = {c.second.as_string()}; + else + expected = c.second.as_set(); + for (auto& e : expected) { + if (aud.count(e) == 0) { + ec = error::token_verification_error::audience_missmatch; + return; + } + } + } else { + assert_claim_eq(jwt, c.first, c.second, ec); + if (ec) return; + } + } + } + }; + + /** + * Create a verifier using the given clock + * \param c Clock instance to use + * \return verifier instance + */ + template + verifier verify(Clock c) { + return verifier(c); + } + + /** + * Default clock class using std::chrono::system_clock as a backend. + */ + struct default_clock { + date now() const { return date::clock::now(); } + }; + + /** + * Return a builder instance to create a new token + */ + template + builder create() { + return builder(); + } + + /** + * Decode a token + * \param token Token to decode + * \param decode function that will pad and base64url decode the token + * \return Decoded token + * \throw std::invalid_argument Token is not in correct format + * \throw std::runtime_error Base64 decoding failed or invalid json + */ + template + decoded_jwt decode(const typename json_traits::string_type& token, Decode decode) { + return decoded_jwt(token, decode); + } + + /** + * Decode a token + * \param token Token to decode + * \return Decoded token + * \throw std::invalid_argument Token is not in correct format + * \throw std::runtime_error Base64 decoding failed or invalid json + */ + template + decoded_jwt decode(const typename json_traits::string_type& token) { + return decoded_jwt(token); + } + +#ifndef JWT_DISABLE_PICOJSON + struct picojson_traits { + using value_type = picojson::value; + using object_type = picojson::object; + using array_type = picojson::array; + using string_type = std::string; + using number_type = double; + using integer_type = int64_t; + using boolean_type = bool; + + static json::type get_type(const picojson::value& val) { + using json::type; + if (val.is()) return type::boolean; + if (val.is()) return type::integer; + if (val.is()) return type::number; + if (val.is()) return type::string; + if (val.is()) return type::array; + if (val.is()) return type::object; + + throw std::logic_error("invalid type"); + } + + static picojson::object as_object(const picojson::value& val) { + if (!val.is()) throw std::bad_cast(); + return val.get(); + } + + static std::string as_string(const picojson::value& val) { + if (!val.is()) throw std::bad_cast(); + return val.get(); + } + + static picojson::array as_array(const picojson::value& val) { + if (!val.is()) throw std::bad_cast(); + return val.get(); + } + + static int64_t as_int(const picojson::value& val) { + if (!val.is()) throw std::bad_cast(); + return val.get(); + } + + static bool as_bool(const picojson::value& val) { + if (!val.is()) throw std::bad_cast(); + return val.get(); + } + + static double as_number(const picojson::value& val) { + if (!val.is()) throw std::bad_cast(); + return val.get(); + } + + static bool parse(picojson::value& val, const std::string& str) { return picojson::parse(val, str).empty(); } + + static std::string serialize(const picojson::value& val) { return val.serialize(); } + }; + + /** + * Default JSON claim + * + * This type is the default specialization of the \ref basic_claim class which + * uses the standard template types. + */ + using claim = basic_claim; + + /** + * Create a verifier using the default clock + * \return verifier instance + */ + inline verifier verify() { + return verify(default_clock{}); + } + /** + * Return a picojson builder instance to create a new token + */ + inline builder create() { return builder(); } +#ifndef JWT_DISABLE_BASE64 + /** + * Decode a token + * \param token Token to decode + * \return Decoded token + * \throw std::invalid_argument Token is not in correct format + * \throw std::runtime_error Base64 decoding failed or invalid json + */ + inline decoded_jwt decode(const std::string& token) { return decoded_jwt(token); } +#endif + /** + * Decode a token + * \tparam Decode is callabled, taking a string_type and returns a string_type. + * It should ensure the padding of the input and then base64url decode and + * return the results. + * \param token Token to decode + * \param decode The token to parse + * \return Decoded token + * \throw std::invalid_argument Token is not in correct format + * \throw std::runtime_error Base64 decoding failed or invalid json + */ + template + decoded_jwt decode(const std::string& token, Decode decode) { + return decoded_jwt(token, decode); + } +#endif +} // namespace jwt + +template +std::istream& operator>>(std::istream& is, jwt::basic_claim& c) { + return c.operator>>(is); +} + +template +std::ostream& operator<<(std::ostream& os, const jwt::basic_claim& c) { + return os << c.to_json(); +} + +#endif diff --git a/dep/jwt-cpp/include/jwt-cpp/jwt_cpp.h b/dep/jwt-cpp/include/jwt-cpp/jwt_cpp.h deleted file mode 100644 index c8c3c8719..000000000 --- a/dep/jwt-cpp/include/jwt-cpp/jwt_cpp.h +++ /dev/null @@ -1,1593 +0,0 @@ -#pragma once -#define PICOJSON_USE_INT64 -#include "picojson.h" -#include "base.h" -#include -#include -#include -#include -#include -#include -#include -#include -#include - -//If openssl version less than 1.1 -#if OPENSSL_VERSION_NUMBER < 269484032 -#define OPENSSL10 -#endif - -#ifndef JWT_CLAIM_EXPLICIT -#define JWT_CLAIM_EXPLICIT 0 -#endif - -namespace jwt { - using date = std::chrono::system_clock::time_point; - - struct signature_verification_exception : public std::runtime_error { - signature_verification_exception() - : std::runtime_error("signature verification failed") - {} - explicit signature_verification_exception(const std::string& msg) - : std::runtime_error(msg) - {} - explicit signature_verification_exception(const char* msg) - : std::runtime_error(msg) - {} - }; - struct signature_generation_exception : public std::runtime_error { - signature_generation_exception() - : std::runtime_error("signature generation failed") - {} - explicit signature_generation_exception(const std::string& msg) - : std::runtime_error(msg) - {} - explicit signature_generation_exception(const char* msg) - : std::runtime_error(msg) - {} - }; - struct rsa_exception : public std::runtime_error { - explicit rsa_exception(const std::string& msg) - : std::runtime_error(msg) - {} - explicit rsa_exception(const char* msg) - : std::runtime_error(msg) - {} - }; - struct ecdsa_exception : public std::runtime_error { - explicit ecdsa_exception(const std::string& msg) - : std::runtime_error(msg) - {} - explicit ecdsa_exception(const char* msg) - : std::runtime_error(msg) - {} - }; - struct token_verification_exception : public std::runtime_error { - token_verification_exception() - : std::runtime_error("token verification failed") - {} - explicit token_verification_exception(const std::string& msg) - : std::runtime_error("token verification failed: " + msg) - {} - }; - - namespace helper { - inline - std::string extract_pubkey_from_cert(const std::string& certstr, const std::string& pw = "") { - // TODO: Cannot find the exact version this change happended -#if OPENSSL_VERSION_NUMBER <= 0x1000114fL - std::unique_ptr certbio(BIO_new_mem_buf(const_cast(certstr.data()), certstr.size()), BIO_free_all); -#else - std::unique_ptr certbio(BIO_new_mem_buf(certstr.data(), certstr.size()), BIO_free_all); -#endif - std::unique_ptr keybio(BIO_new(BIO_s_mem()), BIO_free_all); - - std::unique_ptr cert(PEM_read_bio_X509(certbio.get(), nullptr, nullptr, const_cast(pw.c_str())), X509_free); - if (!cert) throw rsa_exception("Error loading cert into memory"); - std::unique_ptr key(X509_get_pubkey(cert.get()), EVP_PKEY_free); - if(!key) throw rsa_exception("Error getting public key from certificate"); - if(!PEM_write_bio_PUBKEY(keybio.get(), key.get())) throw rsa_exception("Error writing public key data in PEM format"); - char* ptr = nullptr; - auto len = BIO_get_mem_data(keybio.get(), &ptr); - if(len <= 0 || ptr == nullptr) throw rsa_exception("Failed to convert pubkey to pem"); - std::string res(ptr, len); - return res; - } - } - - namespace algorithm { - /** - * "none" algorithm. - * - * Returns and empty signature and checks if the given signature is empty. - */ - struct none { - /// Return an empty string - std::string sign(const std::string&) const { - return ""; - } - /// Check if the given signature is empty. JWT's with "none" algorithm should not contain a signature. - void verify(const std::string&, const std::string& signature) const { - if (!signature.empty()) - throw signature_verification_exception(); - } - /// Get algorithm name - std::string name() const { - return "none"; - } - }; - /** - * Base class for HMAC family of algorithms - */ - struct hmacsha { - /** - * Construct new hmac algorithm - * \param key Key to use for HMAC - * \param md Pointer to hash function - * \param name Name of the algorithm - */ - hmacsha(std::string key, const EVP_MD*(*md)(), const std::string& name) - : secret(std::move(key)), md(md), alg_name(name) - {} - /** - * Sign jwt data - * \param data The data to sign - * \return HMAC signature for the given data - * \throws signature_generation_exception - */ - std::string sign(const std::string& data) const { - std::string res; - res.resize(EVP_MAX_MD_SIZE); - unsigned int len = res.size(); - if (HMAC(md(), secret.data(), secret.size(), (const unsigned char*)data.data(), data.size(), (unsigned char*)res.data(), &len) == nullptr) - throw signature_generation_exception(); - res.resize(len); - return res; - } - /** - * Check if signature is valid - * \param data The data to check signature against - * \param signature Signature provided by the jwt - * \throws signature_verification_exception If the provided signature does not match - */ - void verify(const std::string& data, const std::string& signature) const { - try { - auto res = sign(data); - bool matched = true; - for (size_t i = 0; i < std::min(res.size(), signature.size()); i++) - if (res[i] != signature[i]) - matched = false; - if (res.size() != signature.size()) - matched = false; - if (!matched) - throw signature_verification_exception(); - } - catch (const signature_generation_exception&) { - throw signature_verification_exception(); - } - } - /** - * Returns the algorithm name provided to the constructor - * \return Algorithmname - */ - std::string name() const { - return alg_name; - } - private: - /// HMAC secrect - const std::string secret; - /// HMAC hash generator - const EVP_MD*(*md)(); - /// Algorithmname - const std::string alg_name; - }; - /** - * Base class for RSA family of algorithms - */ - struct rsa { - /** - * Construct new rsa algorithm - * \param public_key RSA public key in PEM format - * \param private_key RSA private key or empty string if not available. If empty, signing will always fail. - * \param public_key_password Password to decrypt public key pem. - * \param privat_key_password Password to decrypt private key pem. - * \param md Pointer to hash function - * \param name Name of the algorithm - */ - rsa(const std::string& public_key, const std::string& private_key, const std::string& public_key_password, const std::string& private_key_password, const EVP_MD*(*md)(), const std::string& name) - : md(md), alg_name(name) - { - - std::unique_ptr pubkey_bio(BIO_new(BIO_s_mem()), BIO_free_all); - - if(public_key.substr(0, 27) == "-----BEGIN CERTIFICATE-----") { - auto pkey = helper::extract_pubkey_from_cert(public_key, public_key_password); - if ((size_t)BIO_write(pubkey_bio.get(), pkey.data(), pkey.size()) != pkey.size()) - throw rsa_exception("failed to load public key: bio_write failed"); - } else { - if ((size_t)BIO_write(pubkey_bio.get(), public_key.data(), public_key.size()) != public_key.size()) - throw rsa_exception("failed to load public key: bio_write failed"); - } - pkey.reset(PEM_read_bio_PUBKEY(pubkey_bio.get(), nullptr, nullptr, (void*)public_key_password.c_str()), EVP_PKEY_free); - if (!pkey) - throw rsa_exception("failed to load public key: PEM_read_bio_PUBKEY failed"); - - if (!private_key.empty()) { - std::unique_ptr privkey_bio(BIO_new(BIO_s_mem()), BIO_free_all); - if ((size_t)BIO_write(privkey_bio.get(), private_key.data(), private_key.size()) != private_key.size()) - throw rsa_exception("failed to load private key: bio_write failed"); - RSA* privkey = PEM_read_bio_RSAPrivateKey(privkey_bio.get(), nullptr, nullptr, (void*)private_key_password.c_str()); - if (privkey == nullptr) - throw rsa_exception("failed to load private key: PEM_read_bio_RSAPrivateKey failed"); - if (EVP_PKEY_assign_RSA(pkey.get(), privkey) == 0) { - RSA_free(privkey); - throw rsa_exception("failed to load private key: EVP_PKEY_assign_RSA failed"); - } - } - } - /** - * Sign jwt data - * \param data The data to sign - * \return RSA signature for the given data - * \throws signature_generation_exception - */ - std::string sign(const std::string& data) const { -#ifdef OPENSSL10 - std::unique_ptr ctx(EVP_MD_CTX_create(), EVP_MD_CTX_destroy); -#else - std::unique_ptr ctx(EVP_MD_CTX_create(), EVP_MD_CTX_free); -#endif - if (!ctx) - throw signature_generation_exception("failed to create signature: could not create context"); - if (!EVP_SignInit(ctx.get(), md())) - throw signature_generation_exception("failed to create signature: SignInit failed"); - - std::string res; - res.resize(EVP_PKEY_size(pkey.get())); - unsigned int len = 0; - - if (!EVP_SignUpdate(ctx.get(), data.data(), data.size())) - throw signature_generation_exception(); - if (!EVP_SignFinal(ctx.get(), (unsigned char*)res.data(), &len, pkey.get())) - throw signature_generation_exception(); - - res.resize(len); - return res; - } - /** - * Check if signature is valid - * \param data The data to check signature against - * \param signature Signature provided by the jwt - * \throws signature_verification_exception If the provided signature does not match - */ - void verify(const std::string& data, const std::string& signature) const { -#ifdef OPENSSL10 - std::unique_ptr ctx(EVP_MD_CTX_create(), EVP_MD_CTX_destroy); -#else - std::unique_ptr ctx(EVP_MD_CTX_create(), EVP_MD_CTX_free); -#endif - if (!ctx) - throw signature_verification_exception("failed to verify signature: could not create context"); - if (!EVP_VerifyInit(ctx.get(), md())) - throw signature_verification_exception("failed to verify signature: VerifyInit failed"); - if (!EVP_VerifyUpdate(ctx.get(), data.data(), data.size())) - throw signature_verification_exception("failed to verify signature: VerifyUpdate failed"); - if (!EVP_VerifyFinal(ctx.get(), (const unsigned char*)signature.data(), signature.size(), pkey.get())) - throw signature_verification_exception(); - } - /** - * Returns the algorithm name provided to the constructor - * \return Algorithmname - */ - std::string name() const { - return alg_name; - } - private: - /// OpenSSL structure containing converted keys - std::shared_ptr pkey; - /// Hash generator - const EVP_MD*(*md)(); - /// Algorithmname - const std::string alg_name; - }; - /** - * Base class for ECDSA family of algorithms - */ - struct ecdsa { - /** - * Construct new ecdsa algorithm - * \param public_key ECDSA public key in PEM format - * \param private_key ECDSA private key or empty string if not available. If empty, signing will always fail. - * \param public_key_password Password to decrypt public key pem. - * \param privat_key_password Password to decrypt private key pem. - * \param md Pointer to hash function - * \param name Name of the algorithm - */ - ecdsa(const std::string& public_key, const std::string& private_key, const std::string& public_key_password, const std::string& private_key_password, const EVP_MD*(*md)(), const std::string& name) - : md(md), alg_name(name) - { - if (private_key.empty()) { - std::unique_ptr pubkey_bio(BIO_new(BIO_s_mem()), BIO_free_all); - if(public_key.substr(0, 27) == "-----BEGIN CERTIFICATE-----") { - auto pkey = helper::extract_pubkey_from_cert(public_key, public_key_password); - if ((size_t)BIO_write(pubkey_bio.get(), pkey.data(), pkey.size()) != pkey.size()) - throw ecdsa_exception("failed to load public key: bio_write failed"); - } else { - if ((size_t)BIO_write(pubkey_bio.get(), public_key.data(), public_key.size()) != public_key.size()) - throw ecdsa_exception("failed to load public key: bio_write failed"); - } - - pkey.reset(PEM_read_bio_EC_PUBKEY(pubkey_bio.get(), nullptr, nullptr, (void*)public_key_password.c_str()), EC_KEY_free); - if (!pkey) - throw ecdsa_exception("failed to load public key: PEM_read_bio_EC_PUBKEY failed"); - } else { - std::unique_ptr privkey_bio(BIO_new(BIO_s_mem()), BIO_free_all); - if ((size_t)BIO_write(privkey_bio.get(), private_key.data(), private_key.size()) != private_key.size()) - throw ecdsa_exception("failed to load private key: bio_write failed"); - pkey.reset(PEM_read_bio_ECPrivateKey(privkey_bio.get(), nullptr, nullptr, (void*)private_key_password.c_str()), EC_KEY_free); - if (!pkey) - throw ecdsa_exception("failed to load private key: PEM_read_bio_RSAPrivateKey failed"); - } - - if(EC_KEY_check_key(pkey.get()) == 0) - throw ecdsa_exception("failed to load key: key is invalid"); - } - /** - * Sign jwt data - * \param data The data to sign - * \return ECDSA signature for the given data - * \throws signature_generation_exception - */ - std::string sign(const std::string& data) const { - const std::string hash = generate_hash(data); - - std::unique_ptr - sig(ECDSA_do_sign((const unsigned char*)hash.data(), hash.size(), pkey.get()), ECDSA_SIG_free); - if(!sig) - throw signature_generation_exception(); -#ifdef OPENSSL10 - - return bn2raw(sig->r) + bn2raw(sig->s); -#else - const BIGNUM *r; - const BIGNUM *s; - ECDSA_SIG_get0(sig.get(), &r, &s); - return bn2raw(r) + bn2raw(s); -#endif - } - /** - * Check if signature is valid - * \param data The data to check signature against - * \param signature Signature provided by the jwt - * \throws signature_verification_exception If the provided signature does not match - */ - void verify(const std::string& data, const std::string& signature) const { - const std::string hash = generate_hash(data); - auto r = raw2bn(signature.substr(0, signature.size() / 2)); - auto s = raw2bn(signature.substr(signature.size() / 2)); - -#ifdef OPENSSL10 - ECDSA_SIG sig; - sig.r = r.get(); - sig.s = s.get(); - - if(ECDSA_do_verify((const unsigned char*)hash.data(), hash.size(), &sig, pkey.get()) != 1) - throw signature_verification_exception("Invalid signature"); -#else - ECDSA_SIG *sig = ECDSA_SIG_new(); - - ECDSA_SIG_set0(sig, r.get(), s.get()); - - if(ECDSA_do_verify((const unsigned char*)hash.data(), hash.size(), sig, pkey.get()) != 1) - throw signature_verification_exception("Invalid signature"); -#endif - } - /** - * Returns the algorithm name provided to the constructor - * \return Algorithmname - */ - std::string name() const { - return alg_name; - } - private: - /** - * Convert a OpenSSL BIGNUM to a std::string - * \param bn BIGNUM to convert - * \return bignum as string - */ -#ifdef OPENSSL10 - static std::string bn2raw(BIGNUM* bn) -#else - static std::string bn2raw(const BIGNUM* bn) -#endif - { - std::string res; - res.resize(BN_num_bytes(bn)); - BN_bn2bin(bn, (unsigned char*)res.data()); - if(res.size()%2 == 1 && res[0] == 0x00) - return res.substr(1); - return res; - } - /** - * Convert an std::string to a OpenSSL BIGNUM - * \param raw String to convert - * \return BIGNUM representation - */ - static std::unique_ptr raw2bn(const std::string& raw) { - if(static_cast(raw[0]) >= 0x80) { - std::string str(1, 0x00); - str += raw; - return std::unique_ptr(BN_bin2bn((const unsigned char*)str.data(), str.size(), nullptr), BN_free); - } - return std::unique_ptr(BN_bin2bn((const unsigned char*)raw.data(), raw.size(), nullptr), BN_free); - } - - /** - * Hash the provided data using the hash function specified in constructor - * \param data Data to hash - * \return Hash of data - */ - std::string generate_hash(const std::string& data) const { -#ifdef OPENSSL10 - std::unique_ptr ctx(EVP_MD_CTX_create(), &EVP_MD_CTX_destroy); -#else - std::unique_ptr ctx(EVP_MD_CTX_new(), EVP_MD_CTX_free); -#endif - if(EVP_DigestInit(ctx.get(), md()) == 0) - throw signature_generation_exception("EVP_DigestInit failed"); - if(EVP_DigestUpdate(ctx.get(), data.data(), data.size()) == 0) - throw signature_generation_exception("EVP_DigestUpdate failed"); - unsigned int len = 0; - std::string res; - res.resize(EVP_MD_CTX_size(ctx.get())); - if(EVP_DigestFinal(ctx.get(), (unsigned char*)res.data(), &len) == 0) - throw signature_generation_exception("EVP_DigestFinal failed"); - res.resize(len); - return res; - } - - /// OpenSSL struct containing keys - std::shared_ptr pkey; - /// Hash generator function - const EVP_MD*(*md)(); - /// Algorithmname - const std::string alg_name; - }; - - /** - * Base class for PSS-RSA family of algorithms - */ - struct pss { - /** - * Construct new pss algorithm - * \param public_key RSA public key in PEM format - * \param private_key RSA private key or empty string if not available. If empty, signing will always fail. - * \param public_key_password Password to decrypt public key pem. - * \param privat_key_password Password to decrypt private key pem. - * \param md Pointer to hash function - * \param name Name of the algorithm - */ - pss(const std::string& public_key, const std::string& private_key, const std::string& public_key_password, const std::string& private_key_password, const EVP_MD*(*md)(), const std::string& name) - : md(md), alg_name(name) - { - std::unique_ptr pubkey_bio(BIO_new(BIO_s_mem()), BIO_free_all); - if(public_key.substr(0, 27) == "-----BEGIN CERTIFICATE-----") { - auto pkey = helper::extract_pubkey_from_cert(public_key, public_key_password); - if ((size_t)BIO_write(pubkey_bio.get(), pkey.data(), pkey.size()) != pkey.size()) - throw rsa_exception("failed to load public key: bio_write failed"); - } else { - if ((size_t)BIO_write(pubkey_bio.get(), public_key.data(), public_key.size()) != public_key.size()) - throw rsa_exception("failed to load public key: bio_write failed"); - } - pkey.reset(PEM_read_bio_PUBKEY(pubkey_bio.get(), nullptr, nullptr, (void*)public_key_password.c_str()), EVP_PKEY_free); - if (!pkey) - throw rsa_exception("failed to load public key: PEM_read_bio_PUBKEY failed"); - - if (!private_key.empty()) { - std::unique_ptr privkey_bio(BIO_new(BIO_s_mem()), BIO_free_all); - if ((size_t)BIO_write(privkey_bio.get(), private_key.data(), private_key.size()) != private_key.size()) - throw rsa_exception("failed to load private key: bio_write failed"); - RSA* privkey = PEM_read_bio_RSAPrivateKey(privkey_bio.get(), nullptr, nullptr, (void*)private_key_password.c_str()); - if (privkey == nullptr) - throw rsa_exception("failed to load private key: PEM_read_bio_RSAPrivateKey failed"); - if (EVP_PKEY_assign_RSA(pkey.get(), privkey) == 0) { - RSA_free(privkey); - throw rsa_exception("failed to load private key: EVP_PKEY_assign_RSA failed"); - } - } - } - /** - * Sign jwt data - * \param data The data to sign - * \return ECDSA signature for the given data - * \throws signature_generation_exception - */ - std::string sign(const std::string& data) const { - auto hash = this->generate_hash(data); - - std::unique_ptr key(EVP_PKEY_get1_RSA(pkey.get()), RSA_free); - const int size = RSA_size(key.get()); - - std::string padded(size, 0x00); - if (!RSA_padding_add_PKCS1_PSS_mgf1(key.get(), (unsigned char*)padded.data(), (const unsigned char*)hash.data(), md(), md(), -1)) - throw signature_generation_exception("failed to create signature: RSA_padding_add_PKCS1_PSS_mgf1 failed"); - - std::string res(size, 0x00); - if (RSA_private_encrypt(size, (const unsigned char*)padded.data(), (unsigned char*)res.data(), key.get(), RSA_NO_PADDING) < 0) - throw signature_generation_exception("failed to create signature: RSA_private_encrypt failed"); - return res; - } - /** - * Check if signature is valid - * \param data The data to check signature against - * \param signature Signature provided by the jwt - * \throws signature_verification_exception If the provided signature does not match - */ - void verify(const std::string& data, const std::string& signature) const { - auto hash = this->generate_hash(data); - - std::unique_ptr key(EVP_PKEY_get1_RSA(pkey.get()), RSA_free); - const int size = RSA_size(key.get()); - - std::string sig(size, 0x00); - if(!RSA_public_decrypt(signature.size(), (const unsigned char*)signature.data(), (unsigned char*)sig.data(), key.get(), RSA_NO_PADDING)) - throw signature_verification_exception("Invalid signature"); - - if(!RSA_verify_PKCS1_PSS_mgf1(key.get(), (const unsigned char*)hash.data(), md(), md(), (const unsigned char*)sig.data(), -1)) - throw signature_verification_exception("Invalid signature"); - } - /** - * Returns the algorithm name provided to the constructor - * \return Algorithmname - */ - std::string name() const { - return alg_name; - } - private: - /** - * Hash the provided data using the hash function specified in constructor - * \param data Data to hash - * \return Hash of data - */ - std::string generate_hash(const std::string& data) const { -#ifdef OPENSSL10 - std::unique_ptr ctx(EVP_MD_CTX_create(), &EVP_MD_CTX_destroy); -#else - std::unique_ptr ctx(EVP_MD_CTX_new(), EVP_MD_CTX_free); -#endif - if(EVP_DigestInit(ctx.get(), md()) == 0) - throw signature_generation_exception("EVP_DigestInit failed"); - if(EVP_DigestUpdate(ctx.get(), data.data(), data.size()) == 0) - throw signature_generation_exception("EVP_DigestUpdate failed"); - unsigned int len = 0; - std::string res; - res.resize(EVP_MD_CTX_size(ctx.get())); - if(EVP_DigestFinal(ctx.get(), (unsigned char*)res.data(), &len) == 0) - throw signature_generation_exception("EVP_DigestFinal failed"); - res.resize(len); - return res; - } - - /// OpenSSL structure containing keys - std::shared_ptr pkey; - /// Hash generator function - const EVP_MD*(*md)(); - /// Algorithmname - const std::string alg_name; - }; - - /** - * HS256 algorithm - */ - struct hs256 : public hmacsha { - /** - * Construct new instance of algorithm - * \param key HMAC signing key - */ - explicit hs256(std::string key) - : hmacsha(std::move(key), EVP_sha256, "HS256") - {} - }; - /** - * HS384 algorithm - */ - struct hs384 : public hmacsha { - /** - * Construct new instance of algorithm - * \param key HMAC signing key - */ - explicit hs384(std::string key) - : hmacsha(std::move(key), EVP_sha384, "HS384") - {} - }; - /** - * HS512 algorithm - */ - struct hs512 : public hmacsha { - /** - * Construct new instance of algorithm - * \param key HMAC signing key - */ - explicit hs512(std::string key) - : hmacsha(std::move(key), EVP_sha512, "HS512") - {} - }; - /** - * RS256 algorithm - */ - struct rs256 : public rsa { - /** - * Construct new instance of algorithm - * \param public_key RSA public key in PEM format - * \param private_key RSA private key or empty string if not available. If empty, signing will always fail. - * \param public_key_password Password to decrypt public key pem. - * \param privat_key_password Password to decrypt private key pem. - */ - rs256(const std::string& public_key, const std::string& private_key = "", const std::string& public_key_password = "", const std::string& private_key_password = "") - : rsa(public_key, private_key, public_key_password, private_key_password, EVP_sha256, "RS256") - {} - }; - /** - * RS384 algorithm - */ - struct rs384 : public rsa { - /** - * Construct new instance of algorithm - * \param public_key RSA public key in PEM format - * \param private_key RSA private key or empty string if not available. If empty, signing will always fail. - * \param public_key_password Password to decrypt public key pem. - * \param privat_key_password Password to decrypt private key pem. - */ - rs384(const std::string& public_key, const std::string& private_key = "", const std::string& public_key_password = "", const std::string& private_key_password = "") - : rsa(public_key, private_key, public_key_password, private_key_password, EVP_sha384, "RS384") - {} - }; - /** - * RS512 algorithm - */ - struct rs512 : public rsa { - /** - * Construct new instance of algorithm - * \param public_key RSA public key in PEM format - * \param private_key RSA private key or empty string if not available. If empty, signing will always fail. - * \param public_key_password Password to decrypt public key pem. - * \param privat_key_password Password to decrypt private key pem. - */ - rs512(const std::string& public_key, const std::string& private_key = "", const std::string& public_key_password = "", const std::string& private_key_password = "") - : rsa(public_key, private_key, public_key_password, private_key_password, EVP_sha512, "RS512") - {} - }; - /** - * ES256 algorithm - */ - struct es256 : public ecdsa { - /** - * Construct new instance of algorithm - * \param public_key ECDSA public key in PEM format - * \param private_key ECDSA private key or empty string if not available. If empty, signing will always fail. - * \param public_key_password Password to decrypt public key pem. - * \param privat_key_password Password to decrypt private key pem. - */ - es256(const std::string& public_key, const std::string& private_key = "", const std::string& public_key_password = "", const std::string& private_key_password = "") - : ecdsa(public_key, private_key, public_key_password, private_key_password, EVP_sha256, "ES256") - {} - }; - /** - * ES384 algorithm - */ - struct es384 : public ecdsa { - /** - * Construct new instance of algorithm - * \param public_key ECDSA public key in PEM format - * \param private_key ECDSA private key or empty string if not available. If empty, signing will always fail. - * \param public_key_password Password to decrypt public key pem. - * \param privat_key_password Password to decrypt private key pem. - */ - es384(const std::string& public_key, const std::string& private_key = "", const std::string& public_key_password = "", const std::string& private_key_password = "") - : ecdsa(public_key, private_key, public_key_password, private_key_password, EVP_sha384, "ES384") - {} - }; - /** - * ES512 algorithm - */ - struct es512 : public ecdsa { - /** - * Construct new instance of algorithm - * \param public_key ECDSA public key in PEM format - * \param private_key ECDSA private key or empty string if not available. If empty, signing will always fail. - * \param public_key_password Password to decrypt public key pem. - * \param privat_key_password Password to decrypt private key pem. - */ - es512(const std::string& public_key, const std::string& private_key = "", const std::string& public_key_password = "", const std::string& private_key_password = "") - : ecdsa(public_key, private_key, public_key_password, private_key_password, EVP_sha512, "ES512") - {} - }; - - /** - * PS256 algorithm - */ - struct ps256 : public pss { - /** - * Construct new instance of algorithm - * \param public_key RSA public key in PEM format - * \param private_key RSA private key or empty string if not available. If empty, signing will always fail. - * \param public_key_password Password to decrypt public key pem. - * \param privat_key_password Password to decrypt private key pem. - */ - ps256(const std::string& public_key, const std::string& private_key = "", const std::string& public_key_password = "", const std::string& private_key_password = "") - : pss(public_key, private_key, public_key_password, private_key_password, EVP_sha256, "PS256") - {} - }; - /** - * PS384 algorithm - */ - struct ps384 : public pss { - /** - * Construct new instance of algorithm - * \param public_key RSA public key in PEM format - * \param private_key RSA private key or empty string if not available. If empty, signing will always fail. - * \param public_key_password Password to decrypt public key pem. - * \param privat_key_password Password to decrypt private key pem. - */ - ps384(const std::string& public_key, const std::string& private_key = "", const std::string& public_key_password = "", const std::string& private_key_password = "") - : pss(public_key, private_key, public_key_password, private_key_password, EVP_sha384, "PS384") - {} - }; - /** - * PS512 algorithm - */ - struct ps512 : public pss { - /** - * Construct new instance of algorithm - * \param public_key RSA public key in PEM format - * \param private_key RSA private key or empty string if not available. If empty, signing will always fail. - * \param public_key_password Password to decrypt public key pem. - * \param privat_key_password Password to decrypt private key pem. - */ - ps512(const std::string& public_key, const std::string& private_key = "", const std::string& public_key_password = "", const std::string& private_key_password = "") - : pss(public_key, private_key, public_key_password, private_key_password, EVP_sha512, "PS512") - {} - }; - } - - /** - * Convenience wrapper for JSON value - */ - class claim { - picojson::value val; - public: - enum class type { - null, - boolean, - number, - string, - array, - object, - int64 - }; - - claim() - : val() - {} -#if JWT_CLAIM_EXPLICIT - explicit claim(std::string s) - : val(std::move(s)) - {} - explicit claim(const date& s) - : val(int64_t(std::chrono::system_clock::to_time_t(s))) - {} - explicit claim(const std::set& s) - : val(picojson::array(s.cbegin(), s.cend())) - {} - explicit claim(const picojson::value& val) - : val(val) - {} -#else - claim(std::string s) - : val(std::move(s)) - {} - claim(const date& s) - : val(int64_t(std::chrono::system_clock::to_time_t(s))) - {} - claim(const std::set& s) - : val(picojson::array(s.cbegin(), s.cend())) - {} - claim(const picojson::value& val) - : val(val) - {} -#endif - - /** - * Get wrapped json object - * \return Wrapped json object - */ - picojson::value to_json() const { - return val; - } - - /** - * Get type of contained object - * \return Type - * \throws std::logic_error An internal error occured - */ - type get_type() const { - if (val.is()) return type::null; - else if (val.is()) return type::boolean; - else if (val.is()) return type::int64; - else if (val.is()) return type::number; - else if (val.is()) return type::string; - else if (val.is()) return type::array; - else if (val.is()) return type::object; - else throw std::logic_error("internal error"); - } - - /** - * Get the contained object as a string - * \return content as string - * \throws std::bad_cast Content was not a string - */ - const std::string& as_string() const { - if (!val.is()) - throw std::bad_cast(); - return val.get(); - } - /** - * Get the contained object as a date - * \return content as date - * \throws std::bad_cast Content was not a date - */ - date as_date() const { - return std::chrono::system_clock::from_time_t(as_int()); - } - /** - * Get the contained object as an array - * \return content as array - * \throws std::bad_cast Content was not an array - */ - const picojson::array& as_array() const { - if (!val.is()) - throw std::bad_cast(); - return val.get(); - } - /** - * Get the contained object as a set of strings - * \return content as set of strings - * \throws std::bad_cast Content was not a set - */ - const std::set as_set() const { - std::set res; - for(auto& e : as_array()) { - if(!e.is()) - throw std::bad_cast(); - res.insert(e.get()); - } - return res; - } - /** - * Get the contained object as an integer - * \return content as int - * \throws std::bad_cast Content was not an int - */ - int64_t as_int() const { - if (!val.is()) - throw std::bad_cast(); - return val.get(); - } - /** - * Get the contained object as a bool - * \return content as bool - * \throws std::bad_cast Content was not a bool - */ - bool as_bool() const { - if (!val.is()) - throw std::bad_cast(); - return val.get(); - } - /** - * Get the contained object as a number - * \return content as double - * \throws std::bad_cast Content was not a number - */ - double as_number() const { - if (!val.is()) - throw std::bad_cast(); - return val.get(); - } - }; - - /** - * Base class that represents a token payload. - * Contains Convenience accessors for common claims. - */ - class payload { - protected: - std::unordered_map payload_claims; - public: - /** - * Check if issuer is present ("iss") - * \return true if present, false otherwise - */ - bool has_issuer() const noexcept { return has_payload_claim("iss"); } - /** - * Check if subject is present ("sub") - * \return true if present, false otherwise - */ - bool has_subject() const noexcept { return has_payload_claim("sub"); } - /** - * Check if audience is present ("aud") - * \return true if present, false otherwise - */ - bool has_audience() const noexcept { return has_payload_claim("aud"); } - /** - * Check if expires is present ("exp") - * \return true if present, false otherwise - */ - bool has_expires_at() const noexcept { return has_payload_claim("exp"); } - /** - * Check if not before is present ("nbf") - * \return true if present, false otherwise - */ - bool has_not_before() const noexcept { return has_payload_claim("nbf"); } - /** - * Check if issued at is present ("iat") - * \return true if present, false otherwise - */ - bool has_issued_at() const noexcept { return has_payload_claim("iat"); } - /** - * Check if token id is present ("jti") - * \return true if present, false otherwise - */ - bool has_id() const noexcept { return has_payload_claim("jti"); } - /** - * Get issuer claim - * \return issuer as string - * \throws std::runtime_error If claim was not present - * \throws std::bad_cast Claim was present but not a string (Should not happen in a valid token) - */ - const std::string& get_issuer() const { return get_payload_claim("iss").as_string(); } - /** - * Get subject claim - * \return subject as string - * \throws std::runtime_error If claim was not present - * \throws std::bad_cast Claim was present but not a string (Should not happen in a valid token) - */ - const std::string& get_subject() const { return get_payload_claim("sub").as_string(); } - /** - * Get audience claim - * \return audience as a set of strings - * \throws std::runtime_error If claim was not present - * \throws std::bad_cast Claim was present but not a set (Should not happen in a valid token) - */ - std::set get_audience() const { - auto aud = get_payload_claim("aud"); - if(aud.get_type() == jwt::claim::type::string) return { aud.as_string()}; - else return aud.as_set(); - } - /** - * Get expires claim - * \return expires as a date in utc - * \throws std::runtime_error If claim was not present - * \throws std::bad_cast Claim was present but not a date (Should not happen in a valid token) - */ - const date get_expires_at() const { return get_payload_claim("exp").as_date(); } - /** - * Get not valid before claim - * \return nbf date in utc - * \throws std::runtime_error If claim was not present - * \throws std::bad_cast Claim was present but not a date (Should not happen in a valid token) - */ - const date get_not_before() const { return get_payload_claim("nbf").as_date(); } - /** - * Get issued at claim - * \return issued at as date in utc - * \throws std::runtime_error If claim was not present - * \throws std::bad_cast Claim was present but not a date (Should not happen in a valid token) - */ - const date get_issued_at() const { return get_payload_claim("iat").as_date(); } - /** - * Get id claim - * \return id as string - * \throws std::runtime_error If claim was not present - * \throws std::bad_cast Claim was present but not a string (Should not happen in a valid token) - */ - const std::string& get_id() const { return get_payload_claim("jti").as_string(); } - /** - * Check if a payload claim is present - * \return true if claim was present, false otherwise - */ - bool has_payload_claim(const std::string& name) const noexcept { return payload_claims.count(name) != 0; } - /** - * Get payload claim - * \return Requested claim - * \throws std::runtime_error If claim was not present - */ - const claim& get_payload_claim(const std::string& name) const { - if (!has_payload_claim(name)) - throw std::runtime_error("claim not found"); - return payload_claims.at(name); - } - /** - * Get all payload claims - * \return map of claims - */ - std::unordered_map get_payload_claims() const { return payload_claims; } - }; - - /** - * Base class that represents a token header. - * Contains Convenience accessors for common claims. - */ - class header { - protected: - std::unordered_map header_claims; - public: - /** - * Check if algortihm is present ("alg") - * \return true if present, false otherwise - */ - bool has_algorithm() const noexcept { return has_header_claim("alg"); } - /** - * Check if type is present ("typ") - * \return true if present, false otherwise - */ - bool has_type() const noexcept { return has_header_claim("typ"); } - /** - * Check if content type is present ("cty") - * \return true if present, false otherwise - */ - bool has_content_type() const noexcept { return has_header_claim("cty"); } - /** - * Check if key id is present ("kid") - * \return true if present, false otherwise - */ - bool has_key_id() const noexcept { return has_header_claim("kid"); } - /** - * Get algorithm claim - * \return algorithm as string - * \throws std::runtime_error If claim was not present - * \throws std::bad_cast Claim was present but not a string (Should not happen in a valid token) - */ - const std::string& get_algorithm() const { return get_header_claim("alg").as_string(); } - /** - * Get type claim - * \return type as a string - * \throws std::runtime_error If claim was not present - * \throws std::bad_cast Claim was present but not a string (Should not happen in a valid token) - */ - const std::string& get_type() const { return get_header_claim("typ").as_string(); } - /** - * Get content type claim - * \return content type as string - * \throws std::runtime_error If claim was not present - * \throws std::bad_cast Claim was present but not a string (Should not happen in a valid token) - */ - const std::string& get_content_type() const { return get_header_claim("cty").as_string(); } - /** - * Get key id claim - * \return key id as string - * \throws std::runtime_error If claim was not present - * \throws std::bad_cast Claim was present but not a string (Should not happen in a valid token) - */ - const std::string& get_key_id() const { return get_header_claim("kid").as_string(); } - /** - * Check if a header claim is present - * \return true if claim was present, false otherwise - */ - bool has_header_claim(const std::string& name) const noexcept { return header_claims.count(name) != 0; } - /** - * Get header claim - * \return Requested claim - * \throws std::runtime_error If claim was not present - */ - const claim& get_header_claim(const std::string& name) const { - if (!has_header_claim(name)) - throw std::runtime_error("claim not found"); - return header_claims.at(name); - } - /** - * Get all header claims - * \return map of claims - */ - std::unordered_map get_header_claims() const { return header_claims; } - }; - - /** - * Class containing all information about a decoded token - */ - class decoded_jwt : public header, public payload { - protected: - /// Unmodifed token, as passed to constructor - const std::string token; - /// Header part decoded from base64 - std::string header; - /// Unmodified header part in base64 - std::string header_base64; - /// Payload part decoded from base64 - std::string payload; - /// Unmodified payload part in base64 - std::string payload_base64; - /// Signature part decoded from base64 - std::string signature; - /// Unmodified signature part in base64 - std::string signature_base64; - public: - /** - * Constructor - * Parses a given token - * \param token The token to parse - * \throws std::invalid_argument Token is not in correct format - * \throws std::runtime_error Base64 decoding failed or invalid json - */ - explicit decoded_jwt(const std::string& token) - : token(token) - { - auto hdr_end = token.find('.'); - if (hdr_end == std::string::npos) - throw std::invalid_argument("invalid token supplied"); - auto payload_end = token.find('.', hdr_end + 1); - if (payload_end == std::string::npos) - throw std::invalid_argument("invalid token supplied"); - header = header_base64 = token.substr(0, hdr_end); - payload = payload_base64 = token.substr(hdr_end + 1, payload_end - hdr_end - 1); - signature = signature_base64 = token.substr(payload_end + 1); - - // Fix padding: JWT requires padding to get removed - auto fix_padding = [](std::string& str) { - switch (str.size() % 4) { - case 1: - str += alphabet::base64url::fill(); -#ifdef __has_cpp_attribute -#if __has_cpp_attribute(fallthrough) - [[fallthrough]]; -#endif -#endif - case 2: - str += alphabet::base64url::fill(); -#ifdef __has_cpp_attribute -#if __has_cpp_attribute(fallthrough) - [[fallthrough]]; -#endif -#endif - case 3: - str += alphabet::base64url::fill(); -#ifdef __has_cpp_attribute -#if __has_cpp_attribute(fallthrough) - [[fallthrough]]; -#endif -#endif - default: - break; - } - }; - fix_padding(header); - fix_padding(payload); - fix_padding(signature); - - header = base::decode(header); - payload = base::decode(payload); - signature = base::decode(signature); - - auto parse_claims = [](const std::string& str) { - std::unordered_map res; - picojson::value val; - if (!picojson::parse(val, str).empty()) - throw std::runtime_error("Invalid json"); - - for (auto& e : val.get()) { res.insert({ e.first, claim(e.second) }); } - - return res; - }; - - header_claims = parse_claims(header); - payload_claims = parse_claims(payload); - } - - /** - * Get token string, as passed to constructor - * \return token as passed to constructor - */ - const std::string& get_token() const { return token; } - /** - * Get header part as json string - * \return header part after base64 decoding - */ - const std::string& get_header() const { return header; } - /** - * Get payload part as json string - * \return payload part after base64 decoding - */ - const std::string& get_payload() const { return payload; } - /** - * Get signature part as json string - * \return signature part after base64 decoding - */ - const std::string& get_signature() const { return signature; } - /** - * Get header part as base64 string - * \return header part before base64 decoding - */ - const std::string& get_header_base64() const { return header_base64; } - /** - * Get payload part as base64 string - * \return payload part before base64 decoding - */ - const std::string& get_payload_base64() const { return payload_base64; } - /** - * Get signature part as base64 string - * \return signature part before base64 decoding - */ - const std::string& get_signature_base64() const { return signature_base64; } - - }; - - /** - * Builder class to build and sign a new token - * Use jwt::create() to get an instance of this class. - */ - class builder { - std::unordered_map header_claims; - std::unordered_map payload_claims; - - builder() {} - friend builder create(); - public: - /** - * Set a header claim. - * \param id Name of the claim - * \param c Claim to add - * \return *this to allow for method chaining - */ - builder& set_header_claim(const std::string& id, claim c) { header_claims[id] = std::move(c); return *this; } - /** - * Set a payload claim. - * \param id Name of the claim - * \param c Claim to add - * \return *this to allow for method chaining - */ - builder& set_payload_claim(const std::string& id, claim c) { payload_claims[id] = std::move(c); return *this; } - /** - * Set algorithm claim - * You normally don't need to do this, as the algorithm is automatically set if you don't change it. - * \param str Name of algorithm - * \return *this to allow for method chaining - */ - builder& set_algorithm(const std::string& str) { return set_header_claim("alg", claim(str)); } - /** - * Set type claim - * \param str Type to set - * \return *this to allow for method chaining - */ - builder& set_type(const std::string& str) { return set_header_claim("typ", claim(str)); } - /** - * Set content type claim - * \param str Type to set - * \return *this to allow for method chaining - */ - builder& set_content_type(const std::string& str) { return set_header_claim("cty", claim(str)); } - /** - * Set key id claim - * \param str Key id to set - * \return *this to allow for method chaining - */ - builder& set_key_id(const std::string& str) { return set_header_claim("kid", claim(str)); } - /** - * Set issuer claim - * \param str Issuer to set - * \return *this to allow for method chaining - */ - builder& set_issuer(const std::string& str) { return set_payload_claim("iss", claim(str)); } - /** - * Set subject claim - * \param str Subject to set - * \return *this to allow for method chaining - */ - builder& set_subject(const std::string& str) { return set_payload_claim("sub", claim(str)); } - /** - * Set audience claim - * \param l Audience set - * \return *this to allow for method chaining - */ - builder& set_audience(const std::set& l) { return set_payload_claim("aud", claim(l)); } - /** - * Set audience claim - * \param aud Single audience - * \return *this to allow for method chaining - */ - builder& set_audience(const std::string& aud) { return set_payload_claim("aud", claim(aud)); } - /** - * Set expires at claim - * \param d Expires time - * \return *this to allow for method chaining - */ - builder& set_expires_at(const date& d) { return set_payload_claim("exp", claim(d)); } - /** - * Set not before claim - * \param d First valid time - * \return *this to allow for method chaining - */ - builder& set_not_before(const date& d) { return set_payload_claim("nbf", claim(d)); } - /** - * Set issued at claim - * \param d Issued at time, should be current time - * \return *this to allow for method chaining - */ - builder& set_issued_at(const date& d) { return set_payload_claim("iat", claim(d)); } - /** - * Set id claim - * \param str ID to set - * \return *this to allow for method chaining - */ - builder& set_id(const std::string& str) { return set_payload_claim("jti", claim(str)); } - - /** - * Sign token and return result - * \param algo Instance of an algorithm to sign the token with - * \return Final token as a string - */ - template - std::string sign(const T& algo) { - this->set_algorithm(algo.name()); - - picojson::object obj_header; - for (auto& e : header_claims) { - obj_header.insert({ e.first, e.second.to_json() }); - } - picojson::object obj_payload; - for (auto& e : payload_claims) { - obj_payload.insert({ e.first, e.second.to_json() }); - } - - auto encode = [](const std::string& data) { - auto base = base::encode(data); - auto pos = base.find(alphabet::base64url::fill()); - base = base.substr(0, pos); - return base; - }; - - std::string header = encode(picojson::value(obj_header).serialize()); - std::string payload = encode(picojson::value(obj_payload).serialize()); - - std::string token = header + "." + payload; - - return token + "." + encode(algo.sign(token)); - } - }; - - /** - * Verifier class used to check if a decoded token contains all claims required by your application and has a valid signature. - */ - template - class verifier { - struct algo_base { - virtual ~algo_base() {} - virtual void verify(const std::string& data, const std::string& sig) = 0; - }; - template - struct algo : public algo_base { - T alg; - explicit algo(T a) : alg(a) {} - virtual void verify(const std::string& data, const std::string& sig) override { - alg.verify(data, sig); - } - }; - - /// Required claims - std::unordered_map claims; - /// Leeway time for exp, nbf and iat - size_t default_leeway = 0; - /// Instance of clock type - Clock clock; - /// Supported algorithms - std::unordered_map> algs; - public: - /** - * Constructor for building a new verifier instance - * \param c Clock instance - */ - explicit verifier(Clock c) : clock(c) {} - - /** - * Set default leeway to use. - * \param leeway Default leeway to use if not specified otherwise - * \return *this to allow chaining - */ - verifier& leeway(size_t leeway) { default_leeway = leeway; return *this; } - /** - * Set leeway for expires at. - * If not specified the default leeway will be used. - * \param leeway Set leeway to use for expires at. - * \return *this to allow chaining - */ - verifier& expires_at_leeway(size_t leeway) { return with_claim("exp", claim(std::chrono::system_clock::from_time_t(leeway))); } - /** - * Set leeway for not before. - * If not specified the default leeway will be used. - * \param leeway Set leeway to use for not before. - * \return *this to allow chaining - */ - verifier& not_before_leeway(size_t leeway) { return with_claim("nbf", claim(std::chrono::system_clock::from_time_t(leeway))); } - /** - * Set leeway for issued at. - * If not specified the default leeway will be used. - * \param leeway Set leeway to use for issued at. - * \return *this to allow chaining - */ - verifier& issued_at_leeway(size_t leeway) { return with_claim("iat", claim(std::chrono::system_clock::from_time_t(leeway))); } - /** - * Set an issuer to check for. - * Check is casesensitive. - * \param iss Issuer to check for. - * \return *this to allow chaining - */ - verifier& with_issuer(const std::string& iss) { return with_claim("iss", claim(iss)); } - /** - * Set a subject to check for. - * Check is casesensitive. - * \param sub Subject to check for. - * \return *this to allow chaining - */ - verifier& with_subject(const std::string& sub) { return with_claim("sub", claim(sub)); } - /** - * Set an audience to check for. - * If any of the specified audiences is not present in the token the check fails. - * \param aud Audience to check for. - * \return *this to allow chaining - */ - verifier& with_audience(const std::set& aud) { return with_claim("aud", claim(aud)); } - /** - * Set an id to check for. - * Check is casesensitive. - * \param id ID to check for. - * \return *this to allow chaining - */ - verifier& with_id(const std::string& id) { return with_claim("jti", claim(id)); } - /** - * Specify a claim to check for. - * \param name Name of the claim to check for - * \param c Claim to check for - * \return *this to allow chaining - */ - verifier& with_claim(const std::string& name, claim c) { claims[name] = c; return *this; } - - /** - * Add an algorithm available for checking. - * \param alg Algorithm to allow - * \return *this to allow chaining - */ - template - verifier& allow_algorithm(Algorithm alg) { - algs[alg.name()] = std::make_shared>(alg); - return *this; - } - - /** - * Verify the given token. - * \param jwt Token to check - * \throws token_verification_exception Verification failed - */ - void verify(const decoded_jwt& jwt) const { - const std::string data = jwt.get_header_base64() + "." + jwt.get_payload_base64(); - const std::string sig = jwt.get_signature(); - const std::string& algo = jwt.get_algorithm(); - if (algs.count(algo) == 0) - throw token_verification_exception("wrong algorithm"); - algs.at(algo)->verify(data, sig); - - auto assert_claim_eq = [](const decoded_jwt& jwt, const std::string& key, const claim& c) { - if (!jwt.has_payload_claim(key)) - throw token_verification_exception("decoded_jwt is missing " + key + " claim"); - auto& jc = jwt.get_payload_claim(key); - if (jc.get_type() != c.get_type()) - throw token_verification_exception("claim " + key + " type mismatch"); - if (c.get_type() == claim::type::int64) { - if (c.as_date() != jc.as_date()) - throw token_verification_exception("claim " + key + " does not match expected"); - } - else if (c.get_type() == claim::type::array) { - auto s1 = c.as_set(); - auto s2 = jc.as_set(); - if (s1.size() != s2.size()) - throw token_verification_exception("claim " + key + " does not match expected"); - auto it1 = s1.cbegin(); - auto it2 = s2.cbegin(); - while (it1 != s1.cend() && it2 != s2.cend()) { - if (*it1++ != *it2++) - throw token_verification_exception("claim " + key + " does not match expected"); - } - } - else if (c.get_type() == claim::type::string) { - if (c.as_string() != jc.as_string()) - throw token_verification_exception("claim " + key + " does not match expected"); - } - else throw token_verification_exception("internal error"); - }; - - auto time = clock.now(); - - if (jwt.has_expires_at()) { - auto leeway = claims.count("exp") == 1 ? std::chrono::system_clock::to_time_t(claims.at("exp").as_date()) : default_leeway; - auto exp = jwt.get_expires_at(); - if (time > exp + std::chrono::seconds(leeway)) - throw token_verification_exception("token expired"); - } - if (jwt.has_issued_at()) { - auto leeway = claims.count("iat") == 1 ? std::chrono::system_clock::to_time_t(claims.at("iat").as_date()) : default_leeway; - auto iat = jwt.get_issued_at(); - if (time < iat - std::chrono::seconds(leeway)) - throw token_verification_exception("token expired"); - } - if (jwt.has_not_before()) { - auto leeway = claims.count("nbf") == 1 ? std::chrono::system_clock::to_time_t(claims.at("nbf").as_date()) : default_leeway; - auto nbf = jwt.get_not_before(); - if (time < nbf - std::chrono::seconds(leeway)) - throw token_verification_exception("token expired"); - } - for (auto& c : claims) - { - if (c.first == "exp" || c.first == "iat" || c.first == "nbf") { - // Nothing to do here, already checked - } - else if (c.first == "aud") { - if (!jwt.has_audience()) - throw token_verification_exception("token doesn't contain the required audience"); - auto aud = jwt.get_audience(); - auto expected = c.second.as_set(); - for (auto& e : expected) - if (aud.count(e) == 0) - throw token_verification_exception("token doesn't contain the required audience"); - } - else { - assert_claim_eq(jwt, c.first, c.second); - } - } - } - }; - - /** - * Create a verifier using the given clock - * \param c Clock instance to use - * \return verifier instance - */ - template - verifier verify(Clock c) { - return verifier(c); - } - - /** - * Default clock class using std::chrono::system_clock as a backend. - */ - struct default_clock { - std::chrono::system_clock::time_point now() const { - return std::chrono::system_clock::now(); - } - }; - - /** - * Create a verifier using the default clock - * \return verifier instance - */ - inline - verifier verify() { - return verify({}); - } - - /** - * Return a builder instance to create a new token - */ - inline - builder create() { - return builder(); - } - - /** - * Decode a token - * \param token Token to decode - * \return Decoded token - * \throws std::invalid_argument Token is not in correct format - * \throws std::runtime_error Base64 decoding failed or invalid json - */ - inline - decoded_jwt decode(const std::string& token) { - return decoded_jwt(token); - } -} diff --git a/dep/jwt-cpp/include/nlohmann/json.hpp b/dep/jwt-cpp/include/nlohmann/json.hpp new file mode 100644 index 000000000..a70aaf8cb --- /dev/null +++ b/dep/jwt-cpp/include/nlohmann/json.hpp @@ -0,0 +1,25447 @@ +/* + __ _____ _____ _____ + __| | __| | | | JSON for Modern C++ +| | |__ | | | | | | version 3.9.1 +|_____|_____|_____|_|___| https://github.com/nlohmann/json + +Licensed under the MIT License . +SPDX-License-Identifier: MIT +Copyright (c) 2013-2019 Niels Lohmann . + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. +*/ + +#ifndef INCLUDE_NLOHMANN_JSON_HPP_ +#define INCLUDE_NLOHMANN_JSON_HPP_ + +#define NLOHMANN_JSON_VERSION_MAJOR 3 +#define NLOHMANN_JSON_VERSION_MINOR 9 +#define NLOHMANN_JSON_VERSION_PATCH 1 + +#include // all_of, find, for_each +#include // nullptr_t, ptrdiff_t, size_t +#include // hash, less +#include // initializer_list +#include // istream, ostream +#include // random_access_iterator_tag +#include // unique_ptr +#include // accumulate +#include // string, stoi, to_string +#include // declval, forward, move, pair, swap +#include // vector + +// #include + + +#include + +// #include + + +#include // transform +#include // array +#include // forward_list +#include // inserter, front_inserter, end +#include // map +#include // string +#include // tuple, make_tuple +#include // is_arithmetic, is_same, is_enum, underlying_type, is_convertible +#include // unordered_map +#include // pair, declval +#include // valarray + +// #include + + +#include // exception +#include // runtime_error +#include // to_string + +// #include + + +#include // size_t + +namespace nlohmann +{ +namespace detail +{ +/// struct to capture the start position of the current token +struct position_t +{ + /// the total number of characters read + std::size_t chars_read_total = 0; + /// the number of characters read in the current line + std::size_t chars_read_current_line = 0; + /// the number of lines read + std::size_t lines_read = 0; + + /// conversion to size_t to preserve SAX interface + constexpr operator size_t() const + { + return chars_read_total; + } +}; + +} // namespace detail +} // namespace nlohmann + +// #include + + +#include // pair +// #include +/* Hedley - https://nemequ.github.io/hedley + * Created by Evan Nemerson + * + * To the extent possible under law, the author(s) have dedicated all + * copyright and related and neighboring rights to this software to + * the public domain worldwide. This software is distributed without + * any warranty. + * + * For details, see . + * SPDX-License-Identifier: CC0-1.0 + */ + +#if !defined(JSON_HEDLEY_VERSION) || (JSON_HEDLEY_VERSION < 13) +#if defined(JSON_HEDLEY_VERSION) + #undef JSON_HEDLEY_VERSION +#endif +#define JSON_HEDLEY_VERSION 13 + +#if defined(JSON_HEDLEY_STRINGIFY_EX) + #undef JSON_HEDLEY_STRINGIFY_EX +#endif +#define JSON_HEDLEY_STRINGIFY_EX(x) #x + +#if defined(JSON_HEDLEY_STRINGIFY) + #undef JSON_HEDLEY_STRINGIFY +#endif +#define JSON_HEDLEY_STRINGIFY(x) JSON_HEDLEY_STRINGIFY_EX(x) + +#if defined(JSON_HEDLEY_CONCAT_EX) + #undef JSON_HEDLEY_CONCAT_EX +#endif +#define JSON_HEDLEY_CONCAT_EX(a,b) a##b + +#if defined(JSON_HEDLEY_CONCAT) + #undef JSON_HEDLEY_CONCAT +#endif +#define JSON_HEDLEY_CONCAT(a,b) JSON_HEDLEY_CONCAT_EX(a,b) + +#if defined(JSON_HEDLEY_CONCAT3_EX) + #undef JSON_HEDLEY_CONCAT3_EX +#endif +#define JSON_HEDLEY_CONCAT3_EX(a,b,c) a##b##c + +#if defined(JSON_HEDLEY_CONCAT3) + #undef JSON_HEDLEY_CONCAT3 +#endif +#define JSON_HEDLEY_CONCAT3(a,b,c) JSON_HEDLEY_CONCAT3_EX(a,b,c) + +#if defined(JSON_HEDLEY_VERSION_ENCODE) + #undef JSON_HEDLEY_VERSION_ENCODE +#endif +#define JSON_HEDLEY_VERSION_ENCODE(major,minor,revision) (((major) * 1000000) + ((minor) * 1000) + (revision)) + +#if defined(JSON_HEDLEY_VERSION_DECODE_MAJOR) + #undef JSON_HEDLEY_VERSION_DECODE_MAJOR +#endif +#define JSON_HEDLEY_VERSION_DECODE_MAJOR(version) ((version) / 1000000) + +#if defined(JSON_HEDLEY_VERSION_DECODE_MINOR) + #undef JSON_HEDLEY_VERSION_DECODE_MINOR +#endif +#define JSON_HEDLEY_VERSION_DECODE_MINOR(version) (((version) % 1000000) / 1000) + +#if defined(JSON_HEDLEY_VERSION_DECODE_REVISION) + #undef JSON_HEDLEY_VERSION_DECODE_REVISION +#endif +#define JSON_HEDLEY_VERSION_DECODE_REVISION(version) ((version) % 1000) + +#if defined(JSON_HEDLEY_GNUC_VERSION) + #undef JSON_HEDLEY_GNUC_VERSION +#endif +#if defined(__GNUC__) && defined(__GNUC_PATCHLEVEL__) + #define JSON_HEDLEY_GNUC_VERSION JSON_HEDLEY_VERSION_ENCODE(__GNUC__, __GNUC_MINOR__, __GNUC_PATCHLEVEL__) +#elif defined(__GNUC__) + #define JSON_HEDLEY_GNUC_VERSION JSON_HEDLEY_VERSION_ENCODE(__GNUC__, __GNUC_MINOR__, 0) +#endif + +#if defined(JSON_HEDLEY_GNUC_VERSION_CHECK) + #undef JSON_HEDLEY_GNUC_VERSION_CHECK +#endif +#if defined(JSON_HEDLEY_GNUC_VERSION) + #define JSON_HEDLEY_GNUC_VERSION_CHECK(major,minor,patch) (JSON_HEDLEY_GNUC_VERSION >= JSON_HEDLEY_VERSION_ENCODE(major, minor, patch)) +#else + #define JSON_HEDLEY_GNUC_VERSION_CHECK(major,minor,patch) (0) +#endif + +#if defined(JSON_HEDLEY_MSVC_VERSION) + #undef JSON_HEDLEY_MSVC_VERSION +#endif +#if defined(_MSC_FULL_VER) && (_MSC_FULL_VER >= 140000000) + #define JSON_HEDLEY_MSVC_VERSION JSON_HEDLEY_VERSION_ENCODE(_MSC_FULL_VER / 10000000, (_MSC_FULL_VER % 10000000) / 100000, (_MSC_FULL_VER % 100000) / 100) +#elif defined(_MSC_FULL_VER) + #define JSON_HEDLEY_MSVC_VERSION JSON_HEDLEY_VERSION_ENCODE(_MSC_FULL_VER / 1000000, (_MSC_FULL_VER % 1000000) / 10000, (_MSC_FULL_VER % 10000) / 10) +#elif defined(_MSC_VER) + #define JSON_HEDLEY_MSVC_VERSION JSON_HEDLEY_VERSION_ENCODE(_MSC_VER / 100, _MSC_VER % 100, 0) +#endif + +#if defined(JSON_HEDLEY_MSVC_VERSION_CHECK) + #undef JSON_HEDLEY_MSVC_VERSION_CHECK +#endif +#if !defined(_MSC_VER) + #define JSON_HEDLEY_MSVC_VERSION_CHECK(major,minor,patch) (0) +#elif defined(_MSC_VER) && (_MSC_VER >= 1400) + #define JSON_HEDLEY_MSVC_VERSION_CHECK(major,minor,patch) (_MSC_FULL_VER >= ((major * 10000000) + (minor * 100000) + (patch))) +#elif defined(_MSC_VER) && (_MSC_VER >= 1200) + #define JSON_HEDLEY_MSVC_VERSION_CHECK(major,minor,patch) (_MSC_FULL_VER >= ((major * 1000000) + (minor * 10000) + (patch))) +#else + #define JSON_HEDLEY_MSVC_VERSION_CHECK(major,minor,patch) (_MSC_VER >= ((major * 100) + (minor))) +#endif + +#if defined(JSON_HEDLEY_INTEL_VERSION) + #undef JSON_HEDLEY_INTEL_VERSION +#endif +#if defined(__INTEL_COMPILER) && defined(__INTEL_COMPILER_UPDATE) + #define JSON_HEDLEY_INTEL_VERSION JSON_HEDLEY_VERSION_ENCODE(__INTEL_COMPILER / 100, __INTEL_COMPILER % 100, __INTEL_COMPILER_UPDATE) +#elif defined(__INTEL_COMPILER) + #define JSON_HEDLEY_INTEL_VERSION JSON_HEDLEY_VERSION_ENCODE(__INTEL_COMPILER / 100, __INTEL_COMPILER % 100, 0) +#endif + +#if defined(JSON_HEDLEY_INTEL_VERSION_CHECK) + #undef JSON_HEDLEY_INTEL_VERSION_CHECK +#endif +#if defined(JSON_HEDLEY_INTEL_VERSION) + #define JSON_HEDLEY_INTEL_VERSION_CHECK(major,minor,patch) (JSON_HEDLEY_INTEL_VERSION >= JSON_HEDLEY_VERSION_ENCODE(major, minor, patch)) +#else + #define JSON_HEDLEY_INTEL_VERSION_CHECK(major,minor,patch) (0) +#endif + +#if defined(JSON_HEDLEY_PGI_VERSION) + #undef JSON_HEDLEY_PGI_VERSION +#endif +#if defined(__PGI) && defined(__PGIC__) && defined(__PGIC_MINOR__) && defined(__PGIC_PATCHLEVEL__) + #define JSON_HEDLEY_PGI_VERSION JSON_HEDLEY_VERSION_ENCODE(__PGIC__, __PGIC_MINOR__, __PGIC_PATCHLEVEL__) +#endif + +#if defined(JSON_HEDLEY_PGI_VERSION_CHECK) + #undef JSON_HEDLEY_PGI_VERSION_CHECK +#endif +#if defined(JSON_HEDLEY_PGI_VERSION) + #define JSON_HEDLEY_PGI_VERSION_CHECK(major,minor,patch) (JSON_HEDLEY_PGI_VERSION >= JSON_HEDLEY_VERSION_ENCODE(major, minor, patch)) +#else + #define JSON_HEDLEY_PGI_VERSION_CHECK(major,minor,patch) (0) +#endif + +#if defined(JSON_HEDLEY_SUNPRO_VERSION) + #undef JSON_HEDLEY_SUNPRO_VERSION +#endif +#if defined(__SUNPRO_C) && (__SUNPRO_C > 0x1000) + #define JSON_HEDLEY_SUNPRO_VERSION JSON_HEDLEY_VERSION_ENCODE((((__SUNPRO_C >> 16) & 0xf) * 10) + ((__SUNPRO_C >> 12) & 0xf), (((__SUNPRO_C >> 8) & 0xf) * 10) + ((__SUNPRO_C >> 4) & 0xf), (__SUNPRO_C & 0xf) * 10) +#elif defined(__SUNPRO_C) + #define JSON_HEDLEY_SUNPRO_VERSION JSON_HEDLEY_VERSION_ENCODE((__SUNPRO_C >> 8) & 0xf, (__SUNPRO_C >> 4) & 0xf, (__SUNPRO_C) & 0xf) +#elif defined(__SUNPRO_CC) && (__SUNPRO_CC > 0x1000) + #define JSON_HEDLEY_SUNPRO_VERSION JSON_HEDLEY_VERSION_ENCODE((((__SUNPRO_CC >> 16) & 0xf) * 10) + ((__SUNPRO_CC >> 12) & 0xf), (((__SUNPRO_CC >> 8) & 0xf) * 10) + ((__SUNPRO_CC >> 4) & 0xf), (__SUNPRO_CC & 0xf) * 10) +#elif defined(__SUNPRO_CC) + #define JSON_HEDLEY_SUNPRO_VERSION JSON_HEDLEY_VERSION_ENCODE((__SUNPRO_CC >> 8) & 0xf, (__SUNPRO_CC >> 4) & 0xf, (__SUNPRO_CC) & 0xf) +#endif + +#if defined(JSON_HEDLEY_SUNPRO_VERSION_CHECK) + #undef JSON_HEDLEY_SUNPRO_VERSION_CHECK +#endif +#if defined(JSON_HEDLEY_SUNPRO_VERSION) + #define JSON_HEDLEY_SUNPRO_VERSION_CHECK(major,minor,patch) (JSON_HEDLEY_SUNPRO_VERSION >= JSON_HEDLEY_VERSION_ENCODE(major, minor, patch)) +#else + #define JSON_HEDLEY_SUNPRO_VERSION_CHECK(major,minor,patch) (0) +#endif + +#if defined(JSON_HEDLEY_EMSCRIPTEN_VERSION) + #undef JSON_HEDLEY_EMSCRIPTEN_VERSION +#endif +#if defined(__EMSCRIPTEN__) + #define JSON_HEDLEY_EMSCRIPTEN_VERSION JSON_HEDLEY_VERSION_ENCODE(__EMSCRIPTEN_major__, __EMSCRIPTEN_minor__, __EMSCRIPTEN_tiny__) +#endif + +#if defined(JSON_HEDLEY_EMSCRIPTEN_VERSION_CHECK) + #undef JSON_HEDLEY_EMSCRIPTEN_VERSION_CHECK +#endif +#if defined(JSON_HEDLEY_EMSCRIPTEN_VERSION) + #define JSON_HEDLEY_EMSCRIPTEN_VERSION_CHECK(major,minor,patch) (JSON_HEDLEY_EMSCRIPTEN_VERSION >= JSON_HEDLEY_VERSION_ENCODE(major, minor, patch)) +#else + #define JSON_HEDLEY_EMSCRIPTEN_VERSION_CHECK(major,minor,patch) (0) +#endif + +#if defined(JSON_HEDLEY_ARM_VERSION) + #undef JSON_HEDLEY_ARM_VERSION +#endif +#if defined(__CC_ARM) && defined(__ARMCOMPILER_VERSION) + #define JSON_HEDLEY_ARM_VERSION JSON_HEDLEY_VERSION_ENCODE(__ARMCOMPILER_VERSION / 1000000, (__ARMCOMPILER_VERSION % 1000000) / 10000, (__ARMCOMPILER_VERSION % 10000) / 100) +#elif defined(__CC_ARM) && defined(__ARMCC_VERSION) + #define JSON_HEDLEY_ARM_VERSION JSON_HEDLEY_VERSION_ENCODE(__ARMCC_VERSION / 1000000, (__ARMCC_VERSION % 1000000) / 10000, (__ARMCC_VERSION % 10000) / 100) +#endif + +#if defined(JSON_HEDLEY_ARM_VERSION_CHECK) + #undef JSON_HEDLEY_ARM_VERSION_CHECK +#endif +#if defined(JSON_HEDLEY_ARM_VERSION) + #define JSON_HEDLEY_ARM_VERSION_CHECK(major,minor,patch) (JSON_HEDLEY_ARM_VERSION >= JSON_HEDLEY_VERSION_ENCODE(major, minor, patch)) +#else + #define JSON_HEDLEY_ARM_VERSION_CHECK(major,minor,patch) (0) +#endif + +#if defined(JSON_HEDLEY_IBM_VERSION) + #undef JSON_HEDLEY_IBM_VERSION +#endif +#if defined(__ibmxl__) + #define JSON_HEDLEY_IBM_VERSION JSON_HEDLEY_VERSION_ENCODE(__ibmxl_version__, __ibmxl_release__, __ibmxl_modification__) +#elif defined(__xlC__) && defined(__xlC_ver__) + #define JSON_HEDLEY_IBM_VERSION JSON_HEDLEY_VERSION_ENCODE(__xlC__ >> 8, __xlC__ & 0xff, (__xlC_ver__ >> 8) & 0xff) +#elif defined(__xlC__) + #define JSON_HEDLEY_IBM_VERSION JSON_HEDLEY_VERSION_ENCODE(__xlC__ >> 8, __xlC__ & 0xff, 0) +#endif + +#if defined(JSON_HEDLEY_IBM_VERSION_CHECK) + #undef JSON_HEDLEY_IBM_VERSION_CHECK +#endif +#if defined(JSON_HEDLEY_IBM_VERSION) + #define JSON_HEDLEY_IBM_VERSION_CHECK(major,minor,patch) (JSON_HEDLEY_IBM_VERSION >= JSON_HEDLEY_VERSION_ENCODE(major, minor, patch)) +#else + #define JSON_HEDLEY_IBM_VERSION_CHECK(major,minor,patch) (0) +#endif + +#if defined(JSON_HEDLEY_TI_VERSION) + #undef JSON_HEDLEY_TI_VERSION +#endif +#if \ + defined(__TI_COMPILER_VERSION__) && \ + ( \ + defined(__TMS470__) || defined(__TI_ARM__) || \ + defined(__MSP430__) || \ + defined(__TMS320C2000__) \ + ) +#if (__TI_COMPILER_VERSION__ >= 16000000) + #define JSON_HEDLEY_TI_VERSION JSON_HEDLEY_VERSION_ENCODE(__TI_COMPILER_VERSION__ / 1000000, (__TI_COMPILER_VERSION__ % 1000000) / 1000, (__TI_COMPILER_VERSION__ % 1000)) +#endif +#endif + +#if defined(JSON_HEDLEY_TI_VERSION_CHECK) + #undef JSON_HEDLEY_TI_VERSION_CHECK +#endif +#if defined(JSON_HEDLEY_TI_VERSION) + #define JSON_HEDLEY_TI_VERSION_CHECK(major,minor,patch) (JSON_HEDLEY_TI_VERSION >= JSON_HEDLEY_VERSION_ENCODE(major, minor, patch)) +#else + #define JSON_HEDLEY_TI_VERSION_CHECK(major,minor,patch) (0) +#endif + +#if defined(JSON_HEDLEY_TI_CL2000_VERSION) + #undef JSON_HEDLEY_TI_CL2000_VERSION +#endif +#if defined(__TI_COMPILER_VERSION__) && defined(__TMS320C2000__) + #define JSON_HEDLEY_TI_CL2000_VERSION JSON_HEDLEY_VERSION_ENCODE(__TI_COMPILER_VERSION__ / 1000000, (__TI_COMPILER_VERSION__ % 1000000) / 1000, (__TI_COMPILER_VERSION__ % 1000)) +#endif + +#if defined(JSON_HEDLEY_TI_CL2000_VERSION_CHECK) + #undef JSON_HEDLEY_TI_CL2000_VERSION_CHECK +#endif +#if defined(JSON_HEDLEY_TI_CL2000_VERSION) + #define JSON_HEDLEY_TI_CL2000_VERSION_CHECK(major,minor,patch) (JSON_HEDLEY_TI_CL2000_VERSION >= JSON_HEDLEY_VERSION_ENCODE(major, minor, patch)) +#else + #define JSON_HEDLEY_TI_CL2000_VERSION_CHECK(major,minor,patch) (0) +#endif + +#if defined(JSON_HEDLEY_TI_CL430_VERSION) + #undef JSON_HEDLEY_TI_CL430_VERSION +#endif +#if defined(__TI_COMPILER_VERSION__) && defined(__MSP430__) + #define JSON_HEDLEY_TI_CL430_VERSION JSON_HEDLEY_VERSION_ENCODE(__TI_COMPILER_VERSION__ / 1000000, (__TI_COMPILER_VERSION__ % 1000000) / 1000, (__TI_COMPILER_VERSION__ % 1000)) +#endif + +#if defined(JSON_HEDLEY_TI_CL430_VERSION_CHECK) + #undef JSON_HEDLEY_TI_CL430_VERSION_CHECK +#endif +#if defined(JSON_HEDLEY_TI_CL430_VERSION) + #define JSON_HEDLEY_TI_CL430_VERSION_CHECK(major,minor,patch) (JSON_HEDLEY_TI_CL430_VERSION >= JSON_HEDLEY_VERSION_ENCODE(major, minor, patch)) +#else + #define JSON_HEDLEY_TI_CL430_VERSION_CHECK(major,minor,patch) (0) +#endif + +#if defined(JSON_HEDLEY_TI_ARMCL_VERSION) + #undef JSON_HEDLEY_TI_ARMCL_VERSION +#endif +#if defined(__TI_COMPILER_VERSION__) && (defined(__TMS470__) || defined(__TI_ARM__)) + #define JSON_HEDLEY_TI_ARMCL_VERSION JSON_HEDLEY_VERSION_ENCODE(__TI_COMPILER_VERSION__ / 1000000, (__TI_COMPILER_VERSION__ % 1000000) / 1000, (__TI_COMPILER_VERSION__ % 1000)) +#endif + +#if defined(JSON_HEDLEY_TI_ARMCL_VERSION_CHECK) + #undef JSON_HEDLEY_TI_ARMCL_VERSION_CHECK +#endif +#if defined(JSON_HEDLEY_TI_ARMCL_VERSION) + #define JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(major,minor,patch) (JSON_HEDLEY_TI_ARMCL_VERSION >= JSON_HEDLEY_VERSION_ENCODE(major, minor, patch)) +#else + #define JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(major,minor,patch) (0) +#endif + +#if defined(JSON_HEDLEY_TI_CL6X_VERSION) + #undef JSON_HEDLEY_TI_CL6X_VERSION +#endif +#if defined(__TI_COMPILER_VERSION__) && defined(__TMS320C6X__) + #define JSON_HEDLEY_TI_CL6X_VERSION JSON_HEDLEY_VERSION_ENCODE(__TI_COMPILER_VERSION__ / 1000000, (__TI_COMPILER_VERSION__ % 1000000) / 1000, (__TI_COMPILER_VERSION__ % 1000)) +#endif + +#if defined(JSON_HEDLEY_TI_CL6X_VERSION_CHECK) + #undef JSON_HEDLEY_TI_CL6X_VERSION_CHECK +#endif +#if defined(JSON_HEDLEY_TI_CL6X_VERSION) + #define JSON_HEDLEY_TI_CL6X_VERSION_CHECK(major,minor,patch) (JSON_HEDLEY_TI_CL6X_VERSION >= JSON_HEDLEY_VERSION_ENCODE(major, minor, patch)) +#else + #define JSON_HEDLEY_TI_CL6X_VERSION_CHECK(major,minor,patch) (0) +#endif + +#if defined(JSON_HEDLEY_TI_CL7X_VERSION) + #undef JSON_HEDLEY_TI_CL7X_VERSION +#endif +#if defined(__TI_COMPILER_VERSION__) && defined(__C7000__) + #define JSON_HEDLEY_TI_CL7X_VERSION JSON_HEDLEY_VERSION_ENCODE(__TI_COMPILER_VERSION__ / 1000000, (__TI_COMPILER_VERSION__ % 1000000) / 1000, (__TI_COMPILER_VERSION__ % 1000)) +#endif + +#if defined(JSON_HEDLEY_TI_CL7X_VERSION_CHECK) + #undef JSON_HEDLEY_TI_CL7X_VERSION_CHECK +#endif +#if defined(JSON_HEDLEY_TI_CL7X_VERSION) + #define JSON_HEDLEY_TI_CL7X_VERSION_CHECK(major,minor,patch) (JSON_HEDLEY_TI_CL7X_VERSION >= JSON_HEDLEY_VERSION_ENCODE(major, minor, patch)) +#else + #define JSON_HEDLEY_TI_CL7X_VERSION_CHECK(major,minor,patch) (0) +#endif + +#if defined(JSON_HEDLEY_TI_CLPRU_VERSION) + #undef JSON_HEDLEY_TI_CLPRU_VERSION +#endif +#if defined(__TI_COMPILER_VERSION__) && defined(__PRU__) + #define JSON_HEDLEY_TI_CLPRU_VERSION JSON_HEDLEY_VERSION_ENCODE(__TI_COMPILER_VERSION__ / 1000000, (__TI_COMPILER_VERSION__ % 1000000) / 1000, (__TI_COMPILER_VERSION__ % 1000)) +#endif + +#if defined(JSON_HEDLEY_TI_CLPRU_VERSION_CHECK) + #undef JSON_HEDLEY_TI_CLPRU_VERSION_CHECK +#endif +#if defined(JSON_HEDLEY_TI_CLPRU_VERSION) + #define JSON_HEDLEY_TI_CLPRU_VERSION_CHECK(major,minor,patch) (JSON_HEDLEY_TI_CLPRU_VERSION >= JSON_HEDLEY_VERSION_ENCODE(major, minor, patch)) +#else + #define JSON_HEDLEY_TI_CLPRU_VERSION_CHECK(major,minor,patch) (0) +#endif + +#if defined(JSON_HEDLEY_CRAY_VERSION) + #undef JSON_HEDLEY_CRAY_VERSION +#endif +#if defined(_CRAYC) + #if defined(_RELEASE_PATCHLEVEL) + #define JSON_HEDLEY_CRAY_VERSION JSON_HEDLEY_VERSION_ENCODE(_RELEASE_MAJOR, _RELEASE_MINOR, _RELEASE_PATCHLEVEL) + #else + #define JSON_HEDLEY_CRAY_VERSION JSON_HEDLEY_VERSION_ENCODE(_RELEASE_MAJOR, _RELEASE_MINOR, 0) + #endif +#endif + +#if defined(JSON_HEDLEY_CRAY_VERSION_CHECK) + #undef JSON_HEDLEY_CRAY_VERSION_CHECK +#endif +#if defined(JSON_HEDLEY_CRAY_VERSION) + #define JSON_HEDLEY_CRAY_VERSION_CHECK(major,minor,patch) (JSON_HEDLEY_CRAY_VERSION >= JSON_HEDLEY_VERSION_ENCODE(major, minor, patch)) +#else + #define JSON_HEDLEY_CRAY_VERSION_CHECK(major,minor,patch) (0) +#endif + +#if defined(JSON_HEDLEY_IAR_VERSION) + #undef JSON_HEDLEY_IAR_VERSION +#endif +#if defined(__IAR_SYSTEMS_ICC__) + #if __VER__ > 1000 + #define JSON_HEDLEY_IAR_VERSION JSON_HEDLEY_VERSION_ENCODE((__VER__ / 1000000), ((__VER__ / 1000) % 1000), (__VER__ % 1000)) + #else + #define JSON_HEDLEY_IAR_VERSION JSON_HEDLEY_VERSION_ENCODE(VER / 100, __VER__ % 100, 0) + #endif +#endif + +#if defined(JSON_HEDLEY_IAR_VERSION_CHECK) + #undef JSON_HEDLEY_IAR_VERSION_CHECK +#endif +#if defined(JSON_HEDLEY_IAR_VERSION) + #define JSON_HEDLEY_IAR_VERSION_CHECK(major,minor,patch) (JSON_HEDLEY_IAR_VERSION >= JSON_HEDLEY_VERSION_ENCODE(major, minor, patch)) +#else + #define JSON_HEDLEY_IAR_VERSION_CHECK(major,minor,patch) (0) +#endif + +#if defined(JSON_HEDLEY_TINYC_VERSION) + #undef JSON_HEDLEY_TINYC_VERSION +#endif +#if defined(__TINYC__) + #define JSON_HEDLEY_TINYC_VERSION JSON_HEDLEY_VERSION_ENCODE(__TINYC__ / 1000, (__TINYC__ / 100) % 10, __TINYC__ % 100) +#endif + +#if defined(JSON_HEDLEY_TINYC_VERSION_CHECK) + #undef JSON_HEDLEY_TINYC_VERSION_CHECK +#endif +#if defined(JSON_HEDLEY_TINYC_VERSION) + #define JSON_HEDLEY_TINYC_VERSION_CHECK(major,minor,patch) (JSON_HEDLEY_TINYC_VERSION >= JSON_HEDLEY_VERSION_ENCODE(major, minor, patch)) +#else + #define JSON_HEDLEY_TINYC_VERSION_CHECK(major,minor,patch) (0) +#endif + +#if defined(JSON_HEDLEY_DMC_VERSION) + #undef JSON_HEDLEY_DMC_VERSION +#endif +#if defined(__DMC__) + #define JSON_HEDLEY_DMC_VERSION JSON_HEDLEY_VERSION_ENCODE(__DMC__ >> 8, (__DMC__ >> 4) & 0xf, __DMC__ & 0xf) +#endif + +#if defined(JSON_HEDLEY_DMC_VERSION_CHECK) + #undef JSON_HEDLEY_DMC_VERSION_CHECK +#endif +#if defined(JSON_HEDLEY_DMC_VERSION) + #define JSON_HEDLEY_DMC_VERSION_CHECK(major,minor,patch) (JSON_HEDLEY_DMC_VERSION >= JSON_HEDLEY_VERSION_ENCODE(major, minor, patch)) +#else + #define JSON_HEDLEY_DMC_VERSION_CHECK(major,minor,patch) (0) +#endif + +#if defined(JSON_HEDLEY_COMPCERT_VERSION) + #undef JSON_HEDLEY_COMPCERT_VERSION +#endif +#if defined(__COMPCERT_VERSION__) + #define JSON_HEDLEY_COMPCERT_VERSION JSON_HEDLEY_VERSION_ENCODE(__COMPCERT_VERSION__ / 10000, (__COMPCERT_VERSION__ / 100) % 100, __COMPCERT_VERSION__ % 100) +#endif + +#if defined(JSON_HEDLEY_COMPCERT_VERSION_CHECK) + #undef JSON_HEDLEY_COMPCERT_VERSION_CHECK +#endif +#if defined(JSON_HEDLEY_COMPCERT_VERSION) + #define JSON_HEDLEY_COMPCERT_VERSION_CHECK(major,minor,patch) (JSON_HEDLEY_COMPCERT_VERSION >= JSON_HEDLEY_VERSION_ENCODE(major, minor, patch)) +#else + #define JSON_HEDLEY_COMPCERT_VERSION_CHECK(major,minor,patch) (0) +#endif + +#if defined(JSON_HEDLEY_PELLES_VERSION) + #undef JSON_HEDLEY_PELLES_VERSION +#endif +#if defined(__POCC__) + #define JSON_HEDLEY_PELLES_VERSION JSON_HEDLEY_VERSION_ENCODE(__POCC__ / 100, __POCC__ % 100, 0) +#endif + +#if defined(JSON_HEDLEY_PELLES_VERSION_CHECK) + #undef JSON_HEDLEY_PELLES_VERSION_CHECK +#endif +#if defined(JSON_HEDLEY_PELLES_VERSION) + #define JSON_HEDLEY_PELLES_VERSION_CHECK(major,minor,patch) (JSON_HEDLEY_PELLES_VERSION >= JSON_HEDLEY_VERSION_ENCODE(major, minor, patch)) +#else + #define JSON_HEDLEY_PELLES_VERSION_CHECK(major,minor,patch) (0) +#endif + +#if defined(JSON_HEDLEY_GCC_VERSION) + #undef JSON_HEDLEY_GCC_VERSION +#endif +#if \ + defined(JSON_HEDLEY_GNUC_VERSION) && \ + !defined(__clang__) && \ + !defined(JSON_HEDLEY_INTEL_VERSION) && \ + !defined(JSON_HEDLEY_PGI_VERSION) && \ + !defined(JSON_HEDLEY_ARM_VERSION) && \ + !defined(JSON_HEDLEY_TI_VERSION) && \ + !defined(JSON_HEDLEY_TI_ARMCL_VERSION) && \ + !defined(JSON_HEDLEY_TI_CL430_VERSION) && \ + !defined(JSON_HEDLEY_TI_CL2000_VERSION) && \ + !defined(JSON_HEDLEY_TI_CL6X_VERSION) && \ + !defined(JSON_HEDLEY_TI_CL7X_VERSION) && \ + !defined(JSON_HEDLEY_TI_CLPRU_VERSION) && \ + !defined(__COMPCERT__) + #define JSON_HEDLEY_GCC_VERSION JSON_HEDLEY_GNUC_VERSION +#endif + +#if defined(JSON_HEDLEY_GCC_VERSION_CHECK) + #undef JSON_HEDLEY_GCC_VERSION_CHECK +#endif +#if defined(JSON_HEDLEY_GCC_VERSION) + #define JSON_HEDLEY_GCC_VERSION_CHECK(major,minor,patch) (JSON_HEDLEY_GCC_VERSION >= JSON_HEDLEY_VERSION_ENCODE(major, minor, patch)) +#else + #define JSON_HEDLEY_GCC_VERSION_CHECK(major,minor,patch) (0) +#endif + +#if defined(JSON_HEDLEY_HAS_ATTRIBUTE) + #undef JSON_HEDLEY_HAS_ATTRIBUTE +#endif +#if defined(__has_attribute) + #define JSON_HEDLEY_HAS_ATTRIBUTE(attribute) __has_attribute(attribute) +#else + #define JSON_HEDLEY_HAS_ATTRIBUTE(attribute) (0) +#endif + +#if defined(JSON_HEDLEY_GNUC_HAS_ATTRIBUTE) + #undef JSON_HEDLEY_GNUC_HAS_ATTRIBUTE +#endif +#if defined(__has_attribute) + #define JSON_HEDLEY_GNUC_HAS_ATTRIBUTE(attribute,major,minor,patch) __has_attribute(attribute) +#else + #define JSON_HEDLEY_GNUC_HAS_ATTRIBUTE(attribute,major,minor,patch) JSON_HEDLEY_GNUC_VERSION_CHECK(major,minor,patch) +#endif + +#if defined(JSON_HEDLEY_GCC_HAS_ATTRIBUTE) + #undef JSON_HEDLEY_GCC_HAS_ATTRIBUTE +#endif +#if defined(__has_attribute) + #define JSON_HEDLEY_GCC_HAS_ATTRIBUTE(attribute,major,minor,patch) __has_attribute(attribute) +#else + #define JSON_HEDLEY_GCC_HAS_ATTRIBUTE(attribute,major,minor,patch) JSON_HEDLEY_GCC_VERSION_CHECK(major,minor,patch) +#endif + +#if defined(JSON_HEDLEY_HAS_CPP_ATTRIBUTE) + #undef JSON_HEDLEY_HAS_CPP_ATTRIBUTE +#endif +#if \ + defined(__has_cpp_attribute) && \ + defined(__cplusplus) && \ + (!defined(JSON_HEDLEY_SUNPRO_VERSION) || JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,15,0)) + #define JSON_HEDLEY_HAS_CPP_ATTRIBUTE(attribute) __has_cpp_attribute(attribute) +#else + #define JSON_HEDLEY_HAS_CPP_ATTRIBUTE(attribute) (0) +#endif + +#if defined(JSON_HEDLEY_HAS_CPP_ATTRIBUTE_NS) + #undef JSON_HEDLEY_HAS_CPP_ATTRIBUTE_NS +#endif +#if !defined(__cplusplus) || !defined(__has_cpp_attribute) + #define JSON_HEDLEY_HAS_CPP_ATTRIBUTE_NS(ns,attribute) (0) +#elif \ + !defined(JSON_HEDLEY_PGI_VERSION) && \ + !defined(JSON_HEDLEY_IAR_VERSION) && \ + (!defined(JSON_HEDLEY_SUNPRO_VERSION) || JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,15,0)) && \ + (!defined(JSON_HEDLEY_MSVC_VERSION) || JSON_HEDLEY_MSVC_VERSION_CHECK(19,20,0)) + #define JSON_HEDLEY_HAS_CPP_ATTRIBUTE_NS(ns,attribute) JSON_HEDLEY_HAS_CPP_ATTRIBUTE(ns::attribute) +#else + #define JSON_HEDLEY_HAS_CPP_ATTRIBUTE_NS(ns,attribute) (0) +#endif + +#if defined(JSON_HEDLEY_GNUC_HAS_CPP_ATTRIBUTE) + #undef JSON_HEDLEY_GNUC_HAS_CPP_ATTRIBUTE +#endif +#if defined(__has_cpp_attribute) && defined(__cplusplus) + #define JSON_HEDLEY_GNUC_HAS_CPP_ATTRIBUTE(attribute,major,minor,patch) __has_cpp_attribute(attribute) +#else + #define JSON_HEDLEY_GNUC_HAS_CPP_ATTRIBUTE(attribute,major,minor,patch) JSON_HEDLEY_GNUC_VERSION_CHECK(major,minor,patch) +#endif + +#if defined(JSON_HEDLEY_GCC_HAS_CPP_ATTRIBUTE) + #undef JSON_HEDLEY_GCC_HAS_CPP_ATTRIBUTE +#endif +#if defined(__has_cpp_attribute) && defined(__cplusplus) + #define JSON_HEDLEY_GCC_HAS_CPP_ATTRIBUTE(attribute,major,minor,patch) __has_cpp_attribute(attribute) +#else + #define JSON_HEDLEY_GCC_HAS_CPP_ATTRIBUTE(attribute,major,minor,patch) JSON_HEDLEY_GCC_VERSION_CHECK(major,minor,patch) +#endif + +#if defined(JSON_HEDLEY_HAS_BUILTIN) + #undef JSON_HEDLEY_HAS_BUILTIN +#endif +#if defined(__has_builtin) + #define JSON_HEDLEY_HAS_BUILTIN(builtin) __has_builtin(builtin) +#else + #define JSON_HEDLEY_HAS_BUILTIN(builtin) (0) +#endif + +#if defined(JSON_HEDLEY_GNUC_HAS_BUILTIN) + #undef JSON_HEDLEY_GNUC_HAS_BUILTIN +#endif +#if defined(__has_builtin) + #define JSON_HEDLEY_GNUC_HAS_BUILTIN(builtin,major,minor,patch) __has_builtin(builtin) +#else + #define JSON_HEDLEY_GNUC_HAS_BUILTIN(builtin,major,minor,patch) JSON_HEDLEY_GNUC_VERSION_CHECK(major,minor,patch) +#endif + +#if defined(JSON_HEDLEY_GCC_HAS_BUILTIN) + #undef JSON_HEDLEY_GCC_HAS_BUILTIN +#endif +#if defined(__has_builtin) + #define JSON_HEDLEY_GCC_HAS_BUILTIN(builtin,major,minor,patch) __has_builtin(builtin) +#else + #define JSON_HEDLEY_GCC_HAS_BUILTIN(builtin,major,minor,patch) JSON_HEDLEY_GCC_VERSION_CHECK(major,minor,patch) +#endif + +#if defined(JSON_HEDLEY_HAS_FEATURE) + #undef JSON_HEDLEY_HAS_FEATURE +#endif +#if defined(__has_feature) + #define JSON_HEDLEY_HAS_FEATURE(feature) __has_feature(feature) +#else + #define JSON_HEDLEY_HAS_FEATURE(feature) (0) +#endif + +#if defined(JSON_HEDLEY_GNUC_HAS_FEATURE) + #undef JSON_HEDLEY_GNUC_HAS_FEATURE +#endif +#if defined(__has_feature) + #define JSON_HEDLEY_GNUC_HAS_FEATURE(feature,major,minor,patch) __has_feature(feature) +#else + #define JSON_HEDLEY_GNUC_HAS_FEATURE(feature,major,minor,patch) JSON_HEDLEY_GNUC_VERSION_CHECK(major,minor,patch) +#endif + +#if defined(JSON_HEDLEY_GCC_HAS_FEATURE) + #undef JSON_HEDLEY_GCC_HAS_FEATURE +#endif +#if defined(__has_feature) + #define JSON_HEDLEY_GCC_HAS_FEATURE(feature,major,minor,patch) __has_feature(feature) +#else + #define JSON_HEDLEY_GCC_HAS_FEATURE(feature,major,minor,patch) JSON_HEDLEY_GCC_VERSION_CHECK(major,minor,patch) +#endif + +#if defined(JSON_HEDLEY_HAS_EXTENSION) + #undef JSON_HEDLEY_HAS_EXTENSION +#endif +#if defined(__has_extension) + #define JSON_HEDLEY_HAS_EXTENSION(extension) __has_extension(extension) +#else + #define JSON_HEDLEY_HAS_EXTENSION(extension) (0) +#endif + +#if defined(JSON_HEDLEY_GNUC_HAS_EXTENSION) + #undef JSON_HEDLEY_GNUC_HAS_EXTENSION +#endif +#if defined(__has_extension) + #define JSON_HEDLEY_GNUC_HAS_EXTENSION(extension,major,minor,patch) __has_extension(extension) +#else + #define JSON_HEDLEY_GNUC_HAS_EXTENSION(extension,major,minor,patch) JSON_HEDLEY_GNUC_VERSION_CHECK(major,minor,patch) +#endif + +#if defined(JSON_HEDLEY_GCC_HAS_EXTENSION) + #undef JSON_HEDLEY_GCC_HAS_EXTENSION +#endif +#if defined(__has_extension) + #define JSON_HEDLEY_GCC_HAS_EXTENSION(extension,major,minor,patch) __has_extension(extension) +#else + #define JSON_HEDLEY_GCC_HAS_EXTENSION(extension,major,minor,patch) JSON_HEDLEY_GCC_VERSION_CHECK(major,minor,patch) +#endif + +#if defined(JSON_HEDLEY_HAS_DECLSPEC_ATTRIBUTE) + #undef JSON_HEDLEY_HAS_DECLSPEC_ATTRIBUTE +#endif +#if defined(__has_declspec_attribute) + #define JSON_HEDLEY_HAS_DECLSPEC_ATTRIBUTE(attribute) __has_declspec_attribute(attribute) +#else + #define JSON_HEDLEY_HAS_DECLSPEC_ATTRIBUTE(attribute) (0) +#endif + +#if defined(JSON_HEDLEY_GNUC_HAS_DECLSPEC_ATTRIBUTE) + #undef JSON_HEDLEY_GNUC_HAS_DECLSPEC_ATTRIBUTE +#endif +#if defined(__has_declspec_attribute) + #define JSON_HEDLEY_GNUC_HAS_DECLSPEC_ATTRIBUTE(attribute,major,minor,patch) __has_declspec_attribute(attribute) +#else + #define JSON_HEDLEY_GNUC_HAS_DECLSPEC_ATTRIBUTE(attribute,major,minor,patch) JSON_HEDLEY_GNUC_VERSION_CHECK(major,minor,patch) +#endif + +#if defined(JSON_HEDLEY_GCC_HAS_DECLSPEC_ATTRIBUTE) + #undef JSON_HEDLEY_GCC_HAS_DECLSPEC_ATTRIBUTE +#endif +#if defined(__has_declspec_attribute) + #define JSON_HEDLEY_GCC_HAS_DECLSPEC_ATTRIBUTE(attribute,major,minor,patch) __has_declspec_attribute(attribute) +#else + #define JSON_HEDLEY_GCC_HAS_DECLSPEC_ATTRIBUTE(attribute,major,minor,patch) JSON_HEDLEY_GCC_VERSION_CHECK(major,minor,patch) +#endif + +#if defined(JSON_HEDLEY_HAS_WARNING) + #undef JSON_HEDLEY_HAS_WARNING +#endif +#if defined(__has_warning) + #define JSON_HEDLEY_HAS_WARNING(warning) __has_warning(warning) +#else + #define JSON_HEDLEY_HAS_WARNING(warning) (0) +#endif + +#if defined(JSON_HEDLEY_GNUC_HAS_WARNING) + #undef JSON_HEDLEY_GNUC_HAS_WARNING +#endif +#if defined(__has_warning) + #define JSON_HEDLEY_GNUC_HAS_WARNING(warning,major,minor,patch) __has_warning(warning) +#else + #define JSON_HEDLEY_GNUC_HAS_WARNING(warning,major,minor,patch) JSON_HEDLEY_GNUC_VERSION_CHECK(major,minor,patch) +#endif + +#if defined(JSON_HEDLEY_GCC_HAS_WARNING) + #undef JSON_HEDLEY_GCC_HAS_WARNING +#endif +#if defined(__has_warning) + #define JSON_HEDLEY_GCC_HAS_WARNING(warning,major,minor,patch) __has_warning(warning) +#else + #define JSON_HEDLEY_GCC_HAS_WARNING(warning,major,minor,patch) JSON_HEDLEY_GCC_VERSION_CHECK(major,minor,patch) +#endif + +/* JSON_HEDLEY_DIAGNOSTIC_DISABLE_CPP98_COMPAT_WRAP_ is for + HEDLEY INTERNAL USE ONLY. API subject to change without notice. */ +#if defined(JSON_HEDLEY_DIAGNOSTIC_DISABLE_CPP98_COMPAT_WRAP_) + #undef JSON_HEDLEY_DIAGNOSTIC_DISABLE_CPP98_COMPAT_WRAP_ +#endif +#if defined(__cplusplus) +# if JSON_HEDLEY_HAS_WARNING("-Wc++98-compat") +# if JSON_HEDLEY_HAS_WARNING("-Wc++17-extensions") +# define JSON_HEDLEY_DIAGNOSTIC_DISABLE_CPP98_COMPAT_WRAP_(xpr) \ + JSON_HEDLEY_DIAGNOSTIC_PUSH \ + _Pragma("clang diagnostic ignored \"-Wc++98-compat\"") \ + _Pragma("clang diagnostic ignored \"-Wc++17-extensions\"") \ + xpr \ + JSON_HEDLEY_DIAGNOSTIC_POP +# else +# define JSON_HEDLEY_DIAGNOSTIC_DISABLE_CPP98_COMPAT_WRAP_(xpr) \ + JSON_HEDLEY_DIAGNOSTIC_PUSH \ + _Pragma("clang diagnostic ignored \"-Wc++98-compat\"") \ + xpr \ + JSON_HEDLEY_DIAGNOSTIC_POP +# endif +# endif +#endif +#if !defined(JSON_HEDLEY_DIAGNOSTIC_DISABLE_CPP98_COMPAT_WRAP_) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_CPP98_COMPAT_WRAP_(x) x +#endif + +#if defined(JSON_HEDLEY_CONST_CAST) + #undef JSON_HEDLEY_CONST_CAST +#endif +#if defined(__cplusplus) +# define JSON_HEDLEY_CONST_CAST(T, expr) (const_cast(expr)) +#elif \ + JSON_HEDLEY_HAS_WARNING("-Wcast-qual") || \ + JSON_HEDLEY_GCC_VERSION_CHECK(4,6,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) +# define JSON_HEDLEY_CONST_CAST(T, expr) (__extension__ ({ \ + JSON_HEDLEY_DIAGNOSTIC_PUSH \ + JSON_HEDLEY_DIAGNOSTIC_DISABLE_CAST_QUAL \ + ((T) (expr)); \ + JSON_HEDLEY_DIAGNOSTIC_POP \ + })) +#else +# define JSON_HEDLEY_CONST_CAST(T, expr) ((T) (expr)) +#endif + +#if defined(JSON_HEDLEY_REINTERPRET_CAST) + #undef JSON_HEDLEY_REINTERPRET_CAST +#endif +#if defined(__cplusplus) + #define JSON_HEDLEY_REINTERPRET_CAST(T, expr) (reinterpret_cast(expr)) +#else + #define JSON_HEDLEY_REINTERPRET_CAST(T, expr) ((T) (expr)) +#endif + +#if defined(JSON_HEDLEY_STATIC_CAST) + #undef JSON_HEDLEY_STATIC_CAST +#endif +#if defined(__cplusplus) + #define JSON_HEDLEY_STATIC_CAST(T, expr) (static_cast(expr)) +#else + #define JSON_HEDLEY_STATIC_CAST(T, expr) ((T) (expr)) +#endif + +#if defined(JSON_HEDLEY_CPP_CAST) + #undef JSON_HEDLEY_CPP_CAST +#endif +#if defined(__cplusplus) +# if JSON_HEDLEY_HAS_WARNING("-Wold-style-cast") +# define JSON_HEDLEY_CPP_CAST(T, expr) \ + JSON_HEDLEY_DIAGNOSTIC_PUSH \ + _Pragma("clang diagnostic ignored \"-Wold-style-cast\"") \ + ((T) (expr)) \ + JSON_HEDLEY_DIAGNOSTIC_POP +# elif JSON_HEDLEY_IAR_VERSION_CHECK(8,3,0) +# define JSON_HEDLEY_CPP_CAST(T, expr) \ + JSON_HEDLEY_DIAGNOSTIC_PUSH \ + _Pragma("diag_suppress=Pe137") \ + JSON_HEDLEY_DIAGNOSTIC_POP \ +# else +# define JSON_HEDLEY_CPP_CAST(T, expr) ((T) (expr)) +# endif +#else +# define JSON_HEDLEY_CPP_CAST(T, expr) (expr) +#endif + +#if \ + (defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L)) || \ + defined(__clang__) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(3,0,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) || \ + JSON_HEDLEY_IAR_VERSION_CHECK(8,0,0) || \ + JSON_HEDLEY_PGI_VERSION_CHECK(18,4,0) || \ + JSON_HEDLEY_ARM_VERSION_CHECK(4,1,0) || \ + JSON_HEDLEY_TI_VERSION_CHECK(15,12,0) || \ + JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(4,7,0) || \ + JSON_HEDLEY_TI_CL430_VERSION_CHECK(2,0,1) || \ + JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,1,0) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,0,0) || \ + JSON_HEDLEY_TI_CL7X_VERSION_CHECK(1,2,0) || \ + JSON_HEDLEY_TI_CLPRU_VERSION_CHECK(2,1,0) || \ + JSON_HEDLEY_CRAY_VERSION_CHECK(5,0,0) || \ + JSON_HEDLEY_TINYC_VERSION_CHECK(0,9,17) || \ + JSON_HEDLEY_SUNPRO_VERSION_CHECK(8,0,0) || \ + (JSON_HEDLEY_IBM_VERSION_CHECK(10,1,0) && defined(__C99_PRAGMA_OPERATOR)) + #define JSON_HEDLEY_PRAGMA(value) _Pragma(#value) +#elif JSON_HEDLEY_MSVC_VERSION_CHECK(15,0,0) + #define JSON_HEDLEY_PRAGMA(value) __pragma(value) +#else + #define JSON_HEDLEY_PRAGMA(value) +#endif + +#if defined(JSON_HEDLEY_DIAGNOSTIC_PUSH) + #undef JSON_HEDLEY_DIAGNOSTIC_PUSH +#endif +#if defined(JSON_HEDLEY_DIAGNOSTIC_POP) + #undef JSON_HEDLEY_DIAGNOSTIC_POP +#endif +#if defined(__clang__) + #define JSON_HEDLEY_DIAGNOSTIC_PUSH _Pragma("clang diagnostic push") + #define JSON_HEDLEY_DIAGNOSTIC_POP _Pragma("clang diagnostic pop") +#elif JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) + #define JSON_HEDLEY_DIAGNOSTIC_PUSH _Pragma("warning(push)") + #define JSON_HEDLEY_DIAGNOSTIC_POP _Pragma("warning(pop)") +#elif JSON_HEDLEY_GCC_VERSION_CHECK(4,6,0) + #define JSON_HEDLEY_DIAGNOSTIC_PUSH _Pragma("GCC diagnostic push") + #define JSON_HEDLEY_DIAGNOSTIC_POP _Pragma("GCC diagnostic pop") +#elif JSON_HEDLEY_MSVC_VERSION_CHECK(15,0,0) + #define JSON_HEDLEY_DIAGNOSTIC_PUSH __pragma(warning(push)) + #define JSON_HEDLEY_DIAGNOSTIC_POP __pragma(warning(pop)) +#elif JSON_HEDLEY_ARM_VERSION_CHECK(5,6,0) + #define JSON_HEDLEY_DIAGNOSTIC_PUSH _Pragma("push") + #define JSON_HEDLEY_DIAGNOSTIC_POP _Pragma("pop") +#elif \ + JSON_HEDLEY_TI_VERSION_CHECK(15,12,0) || \ + JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(5,2,0) || \ + JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,4,0) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(8,1,0) || \ + JSON_HEDLEY_TI_CL7X_VERSION_CHECK(1,2,0) || \ + JSON_HEDLEY_TI_CLPRU_VERSION_CHECK(2,1,0) + #define JSON_HEDLEY_DIAGNOSTIC_PUSH _Pragma("diag_push") + #define JSON_HEDLEY_DIAGNOSTIC_POP _Pragma("diag_pop") +#elif JSON_HEDLEY_PELLES_VERSION_CHECK(2,90,0) + #define JSON_HEDLEY_DIAGNOSTIC_PUSH _Pragma("warning(push)") + #define JSON_HEDLEY_DIAGNOSTIC_POP _Pragma("warning(pop)") +#else + #define JSON_HEDLEY_DIAGNOSTIC_PUSH + #define JSON_HEDLEY_DIAGNOSTIC_POP +#endif + +#if defined(JSON_HEDLEY_DIAGNOSTIC_DISABLE_DEPRECATED) + #undef JSON_HEDLEY_DIAGNOSTIC_DISABLE_DEPRECATED +#endif +#if JSON_HEDLEY_HAS_WARNING("-Wdeprecated-declarations") + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_DEPRECATED _Pragma("clang diagnostic ignored \"-Wdeprecated-declarations\"") +#elif JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_DEPRECATED _Pragma("warning(disable:1478 1786)") +#elif JSON_HEDLEY_PGI_VERSION_CHECK(17,10,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_DEPRECATED _Pragma("diag_suppress 1215,1444") +#elif JSON_HEDLEY_GCC_VERSION_CHECK(4,3,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_DEPRECATED _Pragma("GCC diagnostic ignored \"-Wdeprecated-declarations\"") +#elif JSON_HEDLEY_MSVC_VERSION_CHECK(15,0,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_DEPRECATED __pragma(warning(disable:4996)) +#elif \ + JSON_HEDLEY_TI_VERSION_CHECK(15,12,0) || \ + (JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(4,8,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(5,2,0) || \ + (JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,0,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,4,0) || \ + (JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,0,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,3,0) || \ + (JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,2,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,5,0) || \ + JSON_HEDLEY_TI_CL7X_VERSION_CHECK(1,2,0) || \ + JSON_HEDLEY_TI_CLPRU_VERSION_CHECK(2,1,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_DEPRECATED _Pragma("diag_suppress 1291,1718") +#elif JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,13,0) && !defined(__cplusplus) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_DEPRECATED _Pragma("error_messages(off,E_DEPRECATED_ATT,E_DEPRECATED_ATT_MESS)") +#elif JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,13,0) && defined(__cplusplus) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_DEPRECATED _Pragma("error_messages(off,symdeprecated,symdeprecated2)") +#elif JSON_HEDLEY_IAR_VERSION_CHECK(8,0,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_DEPRECATED _Pragma("diag_suppress=Pe1444,Pe1215") +#elif JSON_HEDLEY_PELLES_VERSION_CHECK(2,90,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_DEPRECATED _Pragma("warn(disable:2241)") +#else + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_DEPRECATED +#endif + +#if defined(JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_PRAGMAS) + #undef JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_PRAGMAS +#endif +#if JSON_HEDLEY_HAS_WARNING("-Wunknown-pragmas") + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_PRAGMAS _Pragma("clang diagnostic ignored \"-Wunknown-pragmas\"") +#elif JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_PRAGMAS _Pragma("warning(disable:161)") +#elif JSON_HEDLEY_PGI_VERSION_CHECK(17,10,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_PRAGMAS _Pragma("diag_suppress 1675") +#elif JSON_HEDLEY_GCC_VERSION_CHECK(4,3,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_PRAGMAS _Pragma("GCC diagnostic ignored \"-Wunknown-pragmas\"") +#elif JSON_HEDLEY_MSVC_VERSION_CHECK(15,0,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_PRAGMAS __pragma(warning(disable:4068)) +#elif \ + JSON_HEDLEY_TI_VERSION_CHECK(16,9,0) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(8,0,0) || \ + JSON_HEDLEY_TI_CL7X_VERSION_CHECK(1,2,0) || \ + JSON_HEDLEY_TI_CLPRU_VERSION_CHECK(2,3,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_PRAGMAS _Pragma("diag_suppress 163") +#elif JSON_HEDLEY_TI_CL6X_VERSION_CHECK(8,0,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_PRAGMAS _Pragma("diag_suppress 163") +#elif JSON_HEDLEY_IAR_VERSION_CHECK(8,0,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_PRAGMAS _Pragma("diag_suppress=Pe161") +#else + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_PRAGMAS +#endif + +#if defined(JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_CPP_ATTRIBUTES) + #undef JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_CPP_ATTRIBUTES +#endif +#if JSON_HEDLEY_HAS_WARNING("-Wunknown-attributes") + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_CPP_ATTRIBUTES _Pragma("clang diagnostic ignored \"-Wunknown-attributes\"") +#elif JSON_HEDLEY_GCC_VERSION_CHECK(4,6,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_CPP_ATTRIBUTES _Pragma("GCC diagnostic ignored \"-Wdeprecated-declarations\"") +#elif JSON_HEDLEY_INTEL_VERSION_CHECK(17,0,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_CPP_ATTRIBUTES _Pragma("warning(disable:1292)") +#elif JSON_HEDLEY_MSVC_VERSION_CHECK(19,0,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_CPP_ATTRIBUTES __pragma(warning(disable:5030)) +#elif JSON_HEDLEY_PGI_VERSION_CHECK(17,10,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_CPP_ATTRIBUTES _Pragma("diag_suppress 1097") +#elif JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,14,0) && defined(__cplusplus) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_CPP_ATTRIBUTES _Pragma("error_messages(off,attrskipunsup)") +#elif \ + JSON_HEDLEY_TI_VERSION_CHECK(18,1,0) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(8,3,0) || \ + JSON_HEDLEY_TI_CL7X_VERSION_CHECK(1,2,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_CPP_ATTRIBUTES _Pragma("diag_suppress 1173") +#elif JSON_HEDLEY_IAR_VERSION_CHECK(8,0,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_CPP_ATTRIBUTES _Pragma("diag_suppress=Pe1097") +#else + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_CPP_ATTRIBUTES +#endif + +#if defined(JSON_HEDLEY_DIAGNOSTIC_DISABLE_CAST_QUAL) + #undef JSON_HEDLEY_DIAGNOSTIC_DISABLE_CAST_QUAL +#endif +#if JSON_HEDLEY_HAS_WARNING("-Wcast-qual") + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_CAST_QUAL _Pragma("clang diagnostic ignored \"-Wcast-qual\"") +#elif JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_CAST_QUAL _Pragma("warning(disable:2203 2331)") +#elif JSON_HEDLEY_GCC_VERSION_CHECK(3,0,0) + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_CAST_QUAL _Pragma("GCC diagnostic ignored \"-Wcast-qual\"") +#else + #define JSON_HEDLEY_DIAGNOSTIC_DISABLE_CAST_QUAL +#endif + +#if defined(JSON_HEDLEY_DEPRECATED) + #undef JSON_HEDLEY_DEPRECATED +#endif +#if defined(JSON_HEDLEY_DEPRECATED_FOR) + #undef JSON_HEDLEY_DEPRECATED_FOR +#endif +#if JSON_HEDLEY_MSVC_VERSION_CHECK(14,0,0) + #define JSON_HEDLEY_DEPRECATED(since) __declspec(deprecated("Since " # since)) + #define JSON_HEDLEY_DEPRECATED_FOR(since, replacement) __declspec(deprecated("Since " #since "; use " #replacement)) +#elif defined(__cplusplus) && (__cplusplus >= 201402L) + #define JSON_HEDLEY_DEPRECATED(since) JSON_HEDLEY_DIAGNOSTIC_DISABLE_CPP98_COMPAT_WRAP_([[deprecated("Since " #since)]]) + #define JSON_HEDLEY_DEPRECATED_FOR(since, replacement) JSON_HEDLEY_DIAGNOSTIC_DISABLE_CPP98_COMPAT_WRAP_([[deprecated("Since " #since "; use " #replacement)]]) +#elif \ + JSON_HEDLEY_HAS_EXTENSION(attribute_deprecated_with_message) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(4,5,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) || \ + JSON_HEDLEY_ARM_VERSION_CHECK(5,6,0) || \ + JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,13,0) || \ + JSON_HEDLEY_PGI_VERSION_CHECK(17,10,0) || \ + JSON_HEDLEY_TI_VERSION_CHECK(18,1,0) || \ + JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(18,1,0) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(8,3,0) || \ + JSON_HEDLEY_TI_CL7X_VERSION_CHECK(1,2,0) || \ + JSON_HEDLEY_TI_CLPRU_VERSION_CHECK(2,3,0) + #define JSON_HEDLEY_DEPRECATED(since) __attribute__((__deprecated__("Since " #since))) + #define JSON_HEDLEY_DEPRECATED_FOR(since, replacement) __attribute__((__deprecated__("Since " #since "; use " #replacement))) +#elif \ + JSON_HEDLEY_HAS_ATTRIBUTE(deprecated) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(3,1,0) || \ + JSON_HEDLEY_ARM_VERSION_CHECK(4,1,0) || \ + JSON_HEDLEY_TI_VERSION_CHECK(15,12,0) || \ + (JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(4,8,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(5,2,0) || \ + (JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,0,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,4,0) || \ + (JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,0,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,3,0) || \ + (JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,2,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,5,0) || \ + JSON_HEDLEY_TI_CL7X_VERSION_CHECK(1,2,0) || \ + JSON_HEDLEY_TI_CLPRU_VERSION_CHECK(2,1,0) + #define JSON_HEDLEY_DEPRECATED(since) __attribute__((__deprecated__)) + #define JSON_HEDLEY_DEPRECATED_FOR(since, replacement) __attribute__((__deprecated__)) +#elif \ + JSON_HEDLEY_MSVC_VERSION_CHECK(13,10,0) || \ + JSON_HEDLEY_PELLES_VERSION_CHECK(6,50,0) + #define JSON_HEDLEY_DEPRECATED(since) __declspec(deprecated) + #define JSON_HEDLEY_DEPRECATED_FOR(since, replacement) __declspec(deprecated) +#elif JSON_HEDLEY_IAR_VERSION_CHECK(8,0,0) + #define JSON_HEDLEY_DEPRECATED(since) _Pragma("deprecated") + #define JSON_HEDLEY_DEPRECATED_FOR(since, replacement) _Pragma("deprecated") +#else + #define JSON_HEDLEY_DEPRECATED(since) + #define JSON_HEDLEY_DEPRECATED_FOR(since, replacement) +#endif + +#if defined(JSON_HEDLEY_UNAVAILABLE) + #undef JSON_HEDLEY_UNAVAILABLE +#endif +#if \ + JSON_HEDLEY_HAS_ATTRIBUTE(warning) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(4,3,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) + #define JSON_HEDLEY_UNAVAILABLE(available_since) __attribute__((__warning__("Not available until " #available_since))) +#else + #define JSON_HEDLEY_UNAVAILABLE(available_since) +#endif + +#if defined(JSON_HEDLEY_WARN_UNUSED_RESULT) + #undef JSON_HEDLEY_WARN_UNUSED_RESULT +#endif +#if defined(JSON_HEDLEY_WARN_UNUSED_RESULT_MSG) + #undef JSON_HEDLEY_WARN_UNUSED_RESULT_MSG +#endif +#if (JSON_HEDLEY_HAS_CPP_ATTRIBUTE(nodiscard) >= 201907L) + #define JSON_HEDLEY_WARN_UNUSED_RESULT JSON_HEDLEY_DIAGNOSTIC_DISABLE_CPP98_COMPAT_WRAP_([[nodiscard]]) + #define JSON_HEDLEY_WARN_UNUSED_RESULT_MSG(msg) JSON_HEDLEY_DIAGNOSTIC_DISABLE_CPP98_COMPAT_WRAP_([[nodiscard(msg)]]) +#elif JSON_HEDLEY_HAS_CPP_ATTRIBUTE(nodiscard) + #define JSON_HEDLEY_WARN_UNUSED_RESULT JSON_HEDLEY_DIAGNOSTIC_DISABLE_CPP98_COMPAT_WRAP_([[nodiscard]]) + #define JSON_HEDLEY_WARN_UNUSED_RESULT_MSG(msg) JSON_HEDLEY_DIAGNOSTIC_DISABLE_CPP98_COMPAT_WRAP_([[nodiscard]]) +#elif \ + JSON_HEDLEY_HAS_ATTRIBUTE(warn_unused_result) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(3,4,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) || \ + JSON_HEDLEY_TI_VERSION_CHECK(15,12,0) || \ + (JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(4,8,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(5,2,0) || \ + (JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,0,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,4,0) || \ + (JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,0,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,3,0) || \ + (JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,2,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,5,0) || \ + JSON_HEDLEY_TI_CL7X_VERSION_CHECK(1,2,0) || \ + JSON_HEDLEY_TI_CLPRU_VERSION_CHECK(2,1,0) || \ + (JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,15,0) && defined(__cplusplus)) || \ + JSON_HEDLEY_PGI_VERSION_CHECK(17,10,0) + #define JSON_HEDLEY_WARN_UNUSED_RESULT __attribute__((__warn_unused_result__)) + #define JSON_HEDLEY_WARN_UNUSED_RESULT_MSG(msg) __attribute__((__warn_unused_result__)) +#elif defined(_Check_return_) /* SAL */ + #define JSON_HEDLEY_WARN_UNUSED_RESULT _Check_return_ + #define JSON_HEDLEY_WARN_UNUSED_RESULT_MSG(msg) _Check_return_ +#else + #define JSON_HEDLEY_WARN_UNUSED_RESULT + #define JSON_HEDLEY_WARN_UNUSED_RESULT_MSG(msg) +#endif + +#if defined(JSON_HEDLEY_SENTINEL) + #undef JSON_HEDLEY_SENTINEL +#endif +#if \ + JSON_HEDLEY_HAS_ATTRIBUTE(sentinel) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(4,0,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) || \ + JSON_HEDLEY_ARM_VERSION_CHECK(5,4,0) + #define JSON_HEDLEY_SENTINEL(position) __attribute__((__sentinel__(position))) +#else + #define JSON_HEDLEY_SENTINEL(position) +#endif + +#if defined(JSON_HEDLEY_NO_RETURN) + #undef JSON_HEDLEY_NO_RETURN +#endif +#if JSON_HEDLEY_IAR_VERSION_CHECK(8,0,0) + #define JSON_HEDLEY_NO_RETURN __noreturn +#elif JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) + #define JSON_HEDLEY_NO_RETURN __attribute__((__noreturn__)) +#elif defined(__STDC_VERSION__) && __STDC_VERSION__ >= 201112L + #define JSON_HEDLEY_NO_RETURN _Noreturn +#elif defined(__cplusplus) && (__cplusplus >= 201103L) + #define JSON_HEDLEY_NO_RETURN JSON_HEDLEY_DIAGNOSTIC_DISABLE_CPP98_COMPAT_WRAP_([[noreturn]]) +#elif \ + JSON_HEDLEY_HAS_ATTRIBUTE(noreturn) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(3,2,0) || \ + JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,11,0) || \ + JSON_HEDLEY_ARM_VERSION_CHECK(4,1,0) || \ + JSON_HEDLEY_IBM_VERSION_CHECK(10,1,0) || \ + JSON_HEDLEY_TI_VERSION_CHECK(15,12,0) || \ + (JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(4,8,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(5,2,0) || \ + (JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,0,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,4,0) || \ + (JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,0,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,3,0) || \ + (JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,2,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,5,0) || \ + JSON_HEDLEY_TI_CL7X_VERSION_CHECK(1,2,0) || \ + JSON_HEDLEY_TI_CLPRU_VERSION_CHECK(2,1,0) + #define JSON_HEDLEY_NO_RETURN __attribute__((__noreturn__)) +#elif JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,10,0) + #define JSON_HEDLEY_NO_RETURN _Pragma("does_not_return") +#elif JSON_HEDLEY_MSVC_VERSION_CHECK(13,10,0) + #define JSON_HEDLEY_NO_RETURN __declspec(noreturn) +#elif JSON_HEDLEY_TI_CL6X_VERSION_CHECK(6,0,0) && defined(__cplusplus) + #define JSON_HEDLEY_NO_RETURN _Pragma("FUNC_NEVER_RETURNS;") +#elif JSON_HEDLEY_COMPCERT_VERSION_CHECK(3,2,0) + #define JSON_HEDLEY_NO_RETURN __attribute((noreturn)) +#elif JSON_HEDLEY_PELLES_VERSION_CHECK(9,0,0) + #define JSON_HEDLEY_NO_RETURN __declspec(noreturn) +#else + #define JSON_HEDLEY_NO_RETURN +#endif + +#if defined(JSON_HEDLEY_NO_ESCAPE) + #undef JSON_HEDLEY_NO_ESCAPE +#endif +#if JSON_HEDLEY_HAS_ATTRIBUTE(noescape) + #define JSON_HEDLEY_NO_ESCAPE __attribute__((__noescape__)) +#else + #define JSON_HEDLEY_NO_ESCAPE +#endif + +#if defined(JSON_HEDLEY_UNREACHABLE) + #undef JSON_HEDLEY_UNREACHABLE +#endif +#if defined(JSON_HEDLEY_UNREACHABLE_RETURN) + #undef JSON_HEDLEY_UNREACHABLE_RETURN +#endif +#if defined(JSON_HEDLEY_ASSUME) + #undef JSON_HEDLEY_ASSUME +#endif +#if \ + JSON_HEDLEY_MSVC_VERSION_CHECK(13,10,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) + #define JSON_HEDLEY_ASSUME(expr) __assume(expr) +#elif JSON_HEDLEY_HAS_BUILTIN(__builtin_assume) + #define JSON_HEDLEY_ASSUME(expr) __builtin_assume(expr) +#elif \ + JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,2,0) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(4,0,0) + #if defined(__cplusplus) + #define JSON_HEDLEY_ASSUME(expr) std::_nassert(expr) + #else + #define JSON_HEDLEY_ASSUME(expr) _nassert(expr) + #endif +#endif +#if \ + (JSON_HEDLEY_HAS_BUILTIN(__builtin_unreachable) && (!defined(JSON_HEDLEY_ARM_VERSION))) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(4,5,0) || \ + JSON_HEDLEY_PGI_VERSION_CHECK(18,10,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) || \ + JSON_HEDLEY_IBM_VERSION_CHECK(13,1,5) + #define JSON_HEDLEY_UNREACHABLE() __builtin_unreachable() +#elif defined(JSON_HEDLEY_ASSUME) + #define JSON_HEDLEY_UNREACHABLE() JSON_HEDLEY_ASSUME(0) +#endif +#if !defined(JSON_HEDLEY_ASSUME) + #if defined(JSON_HEDLEY_UNREACHABLE) + #define JSON_HEDLEY_ASSUME(expr) JSON_HEDLEY_STATIC_CAST(void, ((expr) ? 1 : (JSON_HEDLEY_UNREACHABLE(), 1))) + #else + #define JSON_HEDLEY_ASSUME(expr) JSON_HEDLEY_STATIC_CAST(void, expr) + #endif +#endif +#if defined(JSON_HEDLEY_UNREACHABLE) + #if \ + JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,2,0) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(4,0,0) + #define JSON_HEDLEY_UNREACHABLE_RETURN(value) return (JSON_HEDLEY_STATIC_CAST(void, JSON_HEDLEY_ASSUME(0)), (value)) + #else + #define JSON_HEDLEY_UNREACHABLE_RETURN(value) JSON_HEDLEY_UNREACHABLE() + #endif +#else + #define JSON_HEDLEY_UNREACHABLE_RETURN(value) return (value) +#endif +#if !defined(JSON_HEDLEY_UNREACHABLE) + #define JSON_HEDLEY_UNREACHABLE() JSON_HEDLEY_ASSUME(0) +#endif + +JSON_HEDLEY_DIAGNOSTIC_PUSH +#if JSON_HEDLEY_HAS_WARNING("-Wpedantic") + #pragma clang diagnostic ignored "-Wpedantic" +#endif +#if JSON_HEDLEY_HAS_WARNING("-Wc++98-compat-pedantic") && defined(__cplusplus) + #pragma clang diagnostic ignored "-Wc++98-compat-pedantic" +#endif +#if JSON_HEDLEY_GCC_HAS_WARNING("-Wvariadic-macros",4,0,0) + #if defined(__clang__) + #pragma clang diagnostic ignored "-Wvariadic-macros" + #elif defined(JSON_HEDLEY_GCC_VERSION) + #pragma GCC diagnostic ignored "-Wvariadic-macros" + #endif +#endif +#if defined(JSON_HEDLEY_NON_NULL) + #undef JSON_HEDLEY_NON_NULL +#endif +#if \ + JSON_HEDLEY_HAS_ATTRIBUTE(nonnull) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(3,3,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) || \ + JSON_HEDLEY_ARM_VERSION_CHECK(4,1,0) + #define JSON_HEDLEY_NON_NULL(...) __attribute__((__nonnull__(__VA_ARGS__))) +#else + #define JSON_HEDLEY_NON_NULL(...) +#endif +JSON_HEDLEY_DIAGNOSTIC_POP + +#if defined(JSON_HEDLEY_PRINTF_FORMAT) + #undef JSON_HEDLEY_PRINTF_FORMAT +#endif +#if defined(__MINGW32__) && JSON_HEDLEY_GCC_HAS_ATTRIBUTE(format,4,4,0) && !defined(__USE_MINGW_ANSI_STDIO) + #define JSON_HEDLEY_PRINTF_FORMAT(string_idx,first_to_check) __attribute__((__format__(ms_printf, string_idx, first_to_check))) +#elif defined(__MINGW32__) && JSON_HEDLEY_GCC_HAS_ATTRIBUTE(format,4,4,0) && defined(__USE_MINGW_ANSI_STDIO) + #define JSON_HEDLEY_PRINTF_FORMAT(string_idx,first_to_check) __attribute__((__format__(gnu_printf, string_idx, first_to_check))) +#elif \ + JSON_HEDLEY_HAS_ATTRIBUTE(format) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(3,1,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) || \ + JSON_HEDLEY_ARM_VERSION_CHECK(5,6,0) || \ + JSON_HEDLEY_IBM_VERSION_CHECK(10,1,0) || \ + JSON_HEDLEY_TI_VERSION_CHECK(15,12,0) || \ + (JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(4,8,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(5,2,0) || \ + (JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,0,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,4,0) || \ + (JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,0,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,3,0) || \ + (JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,2,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,5,0) || \ + JSON_HEDLEY_TI_CL7X_VERSION_CHECK(1,2,0) || \ + JSON_HEDLEY_TI_CLPRU_VERSION_CHECK(2,1,0) + #define JSON_HEDLEY_PRINTF_FORMAT(string_idx,first_to_check) __attribute__((__format__(__printf__, string_idx, first_to_check))) +#elif JSON_HEDLEY_PELLES_VERSION_CHECK(6,0,0) + #define JSON_HEDLEY_PRINTF_FORMAT(string_idx,first_to_check) __declspec(vaformat(printf,string_idx,first_to_check)) +#else + #define JSON_HEDLEY_PRINTF_FORMAT(string_idx,first_to_check) +#endif + +#if defined(JSON_HEDLEY_CONSTEXPR) + #undef JSON_HEDLEY_CONSTEXPR +#endif +#if defined(__cplusplus) + #if __cplusplus >= 201103L + #define JSON_HEDLEY_CONSTEXPR JSON_HEDLEY_DIAGNOSTIC_DISABLE_CPP98_COMPAT_WRAP_(constexpr) + #endif +#endif +#if !defined(JSON_HEDLEY_CONSTEXPR) + #define JSON_HEDLEY_CONSTEXPR +#endif + +#if defined(JSON_HEDLEY_PREDICT) + #undef JSON_HEDLEY_PREDICT +#endif +#if defined(JSON_HEDLEY_LIKELY) + #undef JSON_HEDLEY_LIKELY +#endif +#if defined(JSON_HEDLEY_UNLIKELY) + #undef JSON_HEDLEY_UNLIKELY +#endif +#if defined(JSON_HEDLEY_UNPREDICTABLE) + #undef JSON_HEDLEY_UNPREDICTABLE +#endif +#if JSON_HEDLEY_HAS_BUILTIN(__builtin_unpredictable) + #define JSON_HEDLEY_UNPREDICTABLE(expr) __builtin_unpredictable((expr)) +#endif +#if \ + JSON_HEDLEY_HAS_BUILTIN(__builtin_expect_with_probability) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(9,0,0) +# define JSON_HEDLEY_PREDICT(expr, value, probability) __builtin_expect_with_probability( (expr), (value), (probability)) +# define JSON_HEDLEY_PREDICT_TRUE(expr, probability) __builtin_expect_with_probability(!!(expr), 1 , (probability)) +# define JSON_HEDLEY_PREDICT_FALSE(expr, probability) __builtin_expect_with_probability(!!(expr), 0 , (probability)) +# define JSON_HEDLEY_LIKELY(expr) __builtin_expect (!!(expr), 1 ) +# define JSON_HEDLEY_UNLIKELY(expr) __builtin_expect (!!(expr), 0 ) +#elif \ + JSON_HEDLEY_HAS_BUILTIN(__builtin_expect) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(3,0,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) || \ + (JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,15,0) && defined(__cplusplus)) || \ + JSON_HEDLEY_ARM_VERSION_CHECK(4,1,0) || \ + JSON_HEDLEY_IBM_VERSION_CHECK(10,1,0) || \ + JSON_HEDLEY_TI_VERSION_CHECK(15,12,0) || \ + JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(4,7,0) || \ + JSON_HEDLEY_TI_CL430_VERSION_CHECK(3,1,0) || \ + JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,1,0) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(6,1,0) || \ + JSON_HEDLEY_TI_CL7X_VERSION_CHECK(1,2,0) || \ + JSON_HEDLEY_TI_CLPRU_VERSION_CHECK(2,1,0) || \ + JSON_HEDLEY_TINYC_VERSION_CHECK(0,9,27) || \ + JSON_HEDLEY_CRAY_VERSION_CHECK(8,1,0) +# define JSON_HEDLEY_PREDICT(expr, expected, probability) \ + (((probability) >= 0.9) ? __builtin_expect((expr), (expected)) : (JSON_HEDLEY_STATIC_CAST(void, expected), (expr))) +# define JSON_HEDLEY_PREDICT_TRUE(expr, probability) \ + (__extension__ ({ \ + double hedley_probability_ = (probability); \ + ((hedley_probability_ >= 0.9) ? __builtin_expect(!!(expr), 1) : ((hedley_probability_ <= 0.1) ? __builtin_expect(!!(expr), 0) : !!(expr))); \ + })) +# define JSON_HEDLEY_PREDICT_FALSE(expr, probability) \ + (__extension__ ({ \ + double hedley_probability_ = (probability); \ + ((hedley_probability_ >= 0.9) ? __builtin_expect(!!(expr), 0) : ((hedley_probability_ <= 0.1) ? __builtin_expect(!!(expr), 1) : !!(expr))); \ + })) +# define JSON_HEDLEY_LIKELY(expr) __builtin_expect(!!(expr), 1) +# define JSON_HEDLEY_UNLIKELY(expr) __builtin_expect(!!(expr), 0) +#else +# define JSON_HEDLEY_PREDICT(expr, expected, probability) (JSON_HEDLEY_STATIC_CAST(void, expected), (expr)) +# define JSON_HEDLEY_PREDICT_TRUE(expr, probability) (!!(expr)) +# define JSON_HEDLEY_PREDICT_FALSE(expr, probability) (!!(expr)) +# define JSON_HEDLEY_LIKELY(expr) (!!(expr)) +# define JSON_HEDLEY_UNLIKELY(expr) (!!(expr)) +#endif +#if !defined(JSON_HEDLEY_UNPREDICTABLE) + #define JSON_HEDLEY_UNPREDICTABLE(expr) JSON_HEDLEY_PREDICT(expr, 1, 0.5) +#endif + +#if defined(JSON_HEDLEY_MALLOC) + #undef JSON_HEDLEY_MALLOC +#endif +#if \ + JSON_HEDLEY_HAS_ATTRIBUTE(malloc) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(3,1,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) || \ + JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,11,0) || \ + JSON_HEDLEY_ARM_VERSION_CHECK(4,1,0) || \ + JSON_HEDLEY_IBM_VERSION_CHECK(12,1,0) || \ + JSON_HEDLEY_TI_VERSION_CHECK(15,12,0) || \ + (JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(4,8,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(5,2,0) || \ + (JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,0,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,4,0) || \ + (JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,0,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,3,0) || \ + (JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,2,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,5,0) || \ + JSON_HEDLEY_TI_CL7X_VERSION_CHECK(1,2,0) || \ + JSON_HEDLEY_TI_CLPRU_VERSION_CHECK(2,1,0) + #define JSON_HEDLEY_MALLOC __attribute__((__malloc__)) +#elif JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,10,0) + #define JSON_HEDLEY_MALLOC _Pragma("returns_new_memory") +#elif JSON_HEDLEY_MSVC_VERSION_CHECK(14, 0, 0) + #define JSON_HEDLEY_MALLOC __declspec(restrict) +#else + #define JSON_HEDLEY_MALLOC +#endif + +#if defined(JSON_HEDLEY_PURE) + #undef JSON_HEDLEY_PURE +#endif +#if \ + JSON_HEDLEY_HAS_ATTRIBUTE(pure) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(2,96,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) || \ + JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,11,0) || \ + JSON_HEDLEY_ARM_VERSION_CHECK(4,1,0) || \ + JSON_HEDLEY_IBM_VERSION_CHECK(10,1,0) || \ + JSON_HEDLEY_TI_VERSION_CHECK(15,12,0) || \ + (JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(4,8,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(5,2,0) || \ + (JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,0,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,4,0) || \ + (JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,0,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,3,0) || \ + (JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,2,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,5,0) || \ + JSON_HEDLEY_TI_CL7X_VERSION_CHECK(1,2,0) || \ + JSON_HEDLEY_TI_CLPRU_VERSION_CHECK(2,1,0) || \ + JSON_HEDLEY_PGI_VERSION_CHECK(17,10,0) +# define JSON_HEDLEY_PURE __attribute__((__pure__)) +#elif JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,10,0) +# define JSON_HEDLEY_PURE _Pragma("does_not_write_global_data") +#elif defined(__cplusplus) && \ + ( \ + JSON_HEDLEY_TI_CL430_VERSION_CHECK(2,0,1) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(4,0,0) || \ + JSON_HEDLEY_TI_CL7X_VERSION_CHECK(1,2,0) \ + ) +# define JSON_HEDLEY_PURE _Pragma("FUNC_IS_PURE;") +#else +# define JSON_HEDLEY_PURE +#endif + +#if defined(JSON_HEDLEY_CONST) + #undef JSON_HEDLEY_CONST +#endif +#if \ + JSON_HEDLEY_HAS_ATTRIBUTE(const) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(2,5,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) || \ + JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,11,0) || \ + JSON_HEDLEY_ARM_VERSION_CHECK(4,1,0) || \ + JSON_HEDLEY_IBM_VERSION_CHECK(10,1,0) || \ + JSON_HEDLEY_TI_VERSION_CHECK(15,12,0) || \ + (JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(4,8,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(5,2,0) || \ + (JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,0,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,4,0) || \ + (JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,0,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,3,0) || \ + (JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,2,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,5,0) || \ + JSON_HEDLEY_TI_CL7X_VERSION_CHECK(1,2,0) || \ + JSON_HEDLEY_TI_CLPRU_VERSION_CHECK(2,1,0) || \ + JSON_HEDLEY_PGI_VERSION_CHECK(17,10,0) + #define JSON_HEDLEY_CONST __attribute__((__const__)) +#elif \ + JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,10,0) + #define JSON_HEDLEY_CONST _Pragma("no_side_effect") +#else + #define JSON_HEDLEY_CONST JSON_HEDLEY_PURE +#endif + +#if defined(JSON_HEDLEY_RESTRICT) + #undef JSON_HEDLEY_RESTRICT +#endif +#if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) && !defined(__cplusplus) + #define JSON_HEDLEY_RESTRICT restrict +#elif \ + JSON_HEDLEY_GCC_VERSION_CHECK(3,1,0) || \ + JSON_HEDLEY_MSVC_VERSION_CHECK(14,0,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) || \ + JSON_HEDLEY_ARM_VERSION_CHECK(4,1,0) || \ + JSON_HEDLEY_IBM_VERSION_CHECK(10,1,0) || \ + JSON_HEDLEY_PGI_VERSION_CHECK(17,10,0) || \ + JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,3,0) || \ + JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,2,4) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(8,1,0) || \ + JSON_HEDLEY_TI_CL7X_VERSION_CHECK(1,2,0) || \ + (JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,14,0) && defined(__cplusplus)) || \ + JSON_HEDLEY_IAR_VERSION_CHECK(8,0,0) || \ + defined(__clang__) + #define JSON_HEDLEY_RESTRICT __restrict +#elif JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,3,0) && !defined(__cplusplus) + #define JSON_HEDLEY_RESTRICT _Restrict +#else + #define JSON_HEDLEY_RESTRICT +#endif + +#if defined(JSON_HEDLEY_INLINE) + #undef JSON_HEDLEY_INLINE +#endif +#if \ + (defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L)) || \ + (defined(__cplusplus) && (__cplusplus >= 199711L)) + #define JSON_HEDLEY_INLINE inline +#elif \ + defined(JSON_HEDLEY_GCC_VERSION) || \ + JSON_HEDLEY_ARM_VERSION_CHECK(6,2,0) + #define JSON_HEDLEY_INLINE __inline__ +#elif \ + JSON_HEDLEY_MSVC_VERSION_CHECK(12,0,0) || \ + JSON_HEDLEY_ARM_VERSION_CHECK(4,1,0) || \ + JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(5,1,0) || \ + JSON_HEDLEY_TI_CL430_VERSION_CHECK(3,1,0) || \ + JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,2,0) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(8,0,0) || \ + JSON_HEDLEY_TI_CL7X_VERSION_CHECK(1,2,0) || \ + JSON_HEDLEY_TI_CLPRU_VERSION_CHECK(2,1,0) + #define JSON_HEDLEY_INLINE __inline +#else + #define JSON_HEDLEY_INLINE +#endif + +#if defined(JSON_HEDLEY_ALWAYS_INLINE) + #undef JSON_HEDLEY_ALWAYS_INLINE +#endif +#if \ + JSON_HEDLEY_HAS_ATTRIBUTE(always_inline) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(4,0,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) || \ + JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,11,0) || \ + JSON_HEDLEY_ARM_VERSION_CHECK(4,1,0) || \ + JSON_HEDLEY_IBM_VERSION_CHECK(10,1,0) || \ + JSON_HEDLEY_TI_VERSION_CHECK(15,12,0) || \ + (JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(4,8,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(5,2,0) || \ + (JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,0,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,4,0) || \ + (JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,0,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,3,0) || \ + (JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,2,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,5,0) || \ + JSON_HEDLEY_TI_CL7X_VERSION_CHECK(1,2,0) || \ + JSON_HEDLEY_TI_CLPRU_VERSION_CHECK(2,1,0) +# define JSON_HEDLEY_ALWAYS_INLINE __attribute__((__always_inline__)) JSON_HEDLEY_INLINE +#elif JSON_HEDLEY_MSVC_VERSION_CHECK(12,0,0) +# define JSON_HEDLEY_ALWAYS_INLINE __forceinline +#elif defined(__cplusplus) && \ + ( \ + JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(5,2,0) || \ + JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,3,0) || \ + JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,4,0) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(6,1,0) || \ + JSON_HEDLEY_TI_CL7X_VERSION_CHECK(1,2,0) || \ + JSON_HEDLEY_TI_CLPRU_VERSION_CHECK(2,1,0) \ + ) +# define JSON_HEDLEY_ALWAYS_INLINE _Pragma("FUNC_ALWAYS_INLINE;") +#elif JSON_HEDLEY_IAR_VERSION_CHECK(8,0,0) +# define JSON_HEDLEY_ALWAYS_INLINE _Pragma("inline=forced") +#else +# define JSON_HEDLEY_ALWAYS_INLINE JSON_HEDLEY_INLINE +#endif + +#if defined(JSON_HEDLEY_NEVER_INLINE) + #undef JSON_HEDLEY_NEVER_INLINE +#endif +#if \ + JSON_HEDLEY_HAS_ATTRIBUTE(noinline) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(4,0,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) || \ + JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,11,0) || \ + JSON_HEDLEY_ARM_VERSION_CHECK(4,1,0) || \ + JSON_HEDLEY_IBM_VERSION_CHECK(10,1,0) || \ + JSON_HEDLEY_TI_VERSION_CHECK(15,12,0) || \ + (JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(4,8,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_ARMCL_VERSION_CHECK(5,2,0) || \ + (JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,0,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL2000_VERSION_CHECK(6,4,0) || \ + (JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,0,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL430_VERSION_CHECK(4,3,0) || \ + (JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,2,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,5,0) || \ + JSON_HEDLEY_TI_CL7X_VERSION_CHECK(1,2,0) || \ + JSON_HEDLEY_TI_CLPRU_VERSION_CHECK(2,1,0) + #define JSON_HEDLEY_NEVER_INLINE __attribute__((__noinline__)) +#elif JSON_HEDLEY_MSVC_VERSION_CHECK(13,10,0) + #define JSON_HEDLEY_NEVER_INLINE __declspec(noinline) +#elif JSON_HEDLEY_PGI_VERSION_CHECK(10,2,0) + #define JSON_HEDLEY_NEVER_INLINE _Pragma("noinline") +#elif JSON_HEDLEY_TI_CL6X_VERSION_CHECK(6,0,0) && defined(__cplusplus) + #define JSON_HEDLEY_NEVER_INLINE _Pragma("FUNC_CANNOT_INLINE;") +#elif JSON_HEDLEY_IAR_VERSION_CHECK(8,0,0) + #define JSON_HEDLEY_NEVER_INLINE _Pragma("inline=never") +#elif JSON_HEDLEY_COMPCERT_VERSION_CHECK(3,2,0) + #define JSON_HEDLEY_NEVER_INLINE __attribute((noinline)) +#elif JSON_HEDLEY_PELLES_VERSION_CHECK(9,0,0) + #define JSON_HEDLEY_NEVER_INLINE __declspec(noinline) +#else + #define JSON_HEDLEY_NEVER_INLINE +#endif + +#if defined(JSON_HEDLEY_PRIVATE) + #undef JSON_HEDLEY_PRIVATE +#endif +#if defined(JSON_HEDLEY_PUBLIC) + #undef JSON_HEDLEY_PUBLIC +#endif +#if defined(JSON_HEDLEY_IMPORT) + #undef JSON_HEDLEY_IMPORT +#endif +#if defined(_WIN32) || defined(__CYGWIN__) +# define JSON_HEDLEY_PRIVATE +# define JSON_HEDLEY_PUBLIC __declspec(dllexport) +# define JSON_HEDLEY_IMPORT __declspec(dllimport) +#else +# if \ + JSON_HEDLEY_HAS_ATTRIBUTE(visibility) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(3,3,0) || \ + JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,11,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) || \ + JSON_HEDLEY_ARM_VERSION_CHECK(4,1,0) || \ + JSON_HEDLEY_IBM_VERSION_CHECK(13,1,0) || \ + ( \ + defined(__TI_EABI__) && \ + ( \ + (JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,2,0) && defined(__TI_GNU_ATTRIBUTE_SUPPORT__)) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(7,5,0) \ + ) \ + ) +# define JSON_HEDLEY_PRIVATE __attribute__((__visibility__("hidden"))) +# define JSON_HEDLEY_PUBLIC __attribute__((__visibility__("default"))) +# else +# define JSON_HEDLEY_PRIVATE +# define JSON_HEDLEY_PUBLIC +# endif +# define JSON_HEDLEY_IMPORT extern +#endif + +#if defined(JSON_HEDLEY_NO_THROW) + #undef JSON_HEDLEY_NO_THROW +#endif +#if \ + JSON_HEDLEY_HAS_ATTRIBUTE(nothrow) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(3,3,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) + #define JSON_HEDLEY_NO_THROW __attribute__((__nothrow__)) +#elif \ + JSON_HEDLEY_MSVC_VERSION_CHECK(13,1,0) || \ + JSON_HEDLEY_ARM_VERSION_CHECK(4,1,0) + #define JSON_HEDLEY_NO_THROW __declspec(nothrow) +#else + #define JSON_HEDLEY_NO_THROW +#endif + +#if defined(JSON_HEDLEY_FALL_THROUGH) + #undef JSON_HEDLEY_FALL_THROUGH +#endif +#if \ + JSON_HEDLEY_HAS_ATTRIBUTE(fallthrough) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(7,0,0) + #define JSON_HEDLEY_FALL_THROUGH __attribute__((__fallthrough__)) +#elif JSON_HEDLEY_HAS_CPP_ATTRIBUTE_NS(clang,fallthrough) + #define JSON_HEDLEY_FALL_THROUGH JSON_HEDLEY_DIAGNOSTIC_DISABLE_CPP98_COMPAT_WRAP_([[clang::fallthrough]]) +#elif JSON_HEDLEY_HAS_CPP_ATTRIBUTE(fallthrough) + #define JSON_HEDLEY_FALL_THROUGH JSON_HEDLEY_DIAGNOSTIC_DISABLE_CPP98_COMPAT_WRAP_([[fallthrough]]) +#elif defined(__fallthrough) /* SAL */ + #define JSON_HEDLEY_FALL_THROUGH __fallthrough +#else + #define JSON_HEDLEY_FALL_THROUGH +#endif + +#if defined(JSON_HEDLEY_RETURNS_NON_NULL) + #undef JSON_HEDLEY_RETURNS_NON_NULL +#endif +#if \ + JSON_HEDLEY_HAS_ATTRIBUTE(returns_nonnull) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(4,9,0) + #define JSON_HEDLEY_RETURNS_NON_NULL __attribute__((__returns_nonnull__)) +#elif defined(_Ret_notnull_) /* SAL */ + #define JSON_HEDLEY_RETURNS_NON_NULL _Ret_notnull_ +#else + #define JSON_HEDLEY_RETURNS_NON_NULL +#endif + +#if defined(JSON_HEDLEY_ARRAY_PARAM) + #undef JSON_HEDLEY_ARRAY_PARAM +#endif +#if \ + defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) && \ + !defined(__STDC_NO_VLA__) && \ + !defined(__cplusplus) && \ + !defined(JSON_HEDLEY_PGI_VERSION) && \ + !defined(JSON_HEDLEY_TINYC_VERSION) + #define JSON_HEDLEY_ARRAY_PARAM(name) (name) +#else + #define JSON_HEDLEY_ARRAY_PARAM(name) +#endif + +#if defined(JSON_HEDLEY_IS_CONSTANT) + #undef JSON_HEDLEY_IS_CONSTANT +#endif +#if defined(JSON_HEDLEY_REQUIRE_CONSTEXPR) + #undef JSON_HEDLEY_REQUIRE_CONSTEXPR +#endif +/* JSON_HEDLEY_IS_CONSTEXPR_ is for + HEDLEY INTERNAL USE ONLY. API subject to change without notice. */ +#if defined(JSON_HEDLEY_IS_CONSTEXPR_) + #undef JSON_HEDLEY_IS_CONSTEXPR_ +#endif +#if \ + JSON_HEDLEY_HAS_BUILTIN(__builtin_constant_p) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(3,4,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) || \ + JSON_HEDLEY_TINYC_VERSION_CHECK(0,9,19) || \ + JSON_HEDLEY_ARM_VERSION_CHECK(4,1,0) || \ + JSON_HEDLEY_IBM_VERSION_CHECK(13,1,0) || \ + JSON_HEDLEY_TI_CL6X_VERSION_CHECK(6,1,0) || \ + (JSON_HEDLEY_SUNPRO_VERSION_CHECK(5,10,0) && !defined(__cplusplus)) || \ + JSON_HEDLEY_CRAY_VERSION_CHECK(8,1,0) + #define JSON_HEDLEY_IS_CONSTANT(expr) __builtin_constant_p(expr) +#endif +#if !defined(__cplusplus) +# if \ + JSON_HEDLEY_HAS_BUILTIN(__builtin_types_compatible_p) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(3,4,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) || \ + JSON_HEDLEY_IBM_VERSION_CHECK(13,1,0) || \ + JSON_HEDLEY_CRAY_VERSION_CHECK(8,1,0) || \ + JSON_HEDLEY_ARM_VERSION_CHECK(5,4,0) || \ + JSON_HEDLEY_TINYC_VERSION_CHECK(0,9,24) +#if defined(__INTPTR_TYPE__) + #define JSON_HEDLEY_IS_CONSTEXPR_(expr) __builtin_types_compatible_p(__typeof__((1 ? (void*) ((__INTPTR_TYPE__) ((expr) * 0)) : (int*) 0)), int*) +#else + #include + #define JSON_HEDLEY_IS_CONSTEXPR_(expr) __builtin_types_compatible_p(__typeof__((1 ? (void*) ((intptr_t) ((expr) * 0)) : (int*) 0)), int*) +#endif +# elif \ + ( \ + defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 201112L) && \ + !defined(JSON_HEDLEY_SUNPRO_VERSION) && \ + !defined(JSON_HEDLEY_PGI_VERSION) && \ + !defined(JSON_HEDLEY_IAR_VERSION)) || \ + JSON_HEDLEY_HAS_EXTENSION(c_generic_selections) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(4,9,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(17,0,0) || \ + JSON_HEDLEY_IBM_VERSION_CHECK(12,1,0) || \ + JSON_HEDLEY_ARM_VERSION_CHECK(5,3,0) +#if defined(__INTPTR_TYPE__) + #define JSON_HEDLEY_IS_CONSTEXPR_(expr) _Generic((1 ? (void*) ((__INTPTR_TYPE__) ((expr) * 0)) : (int*) 0), int*: 1, void*: 0) +#else + #include + #define JSON_HEDLEY_IS_CONSTEXPR_(expr) _Generic((1 ? (void*) ((intptr_t) * 0) : (int*) 0), int*: 1, void*: 0) +#endif +# elif \ + defined(JSON_HEDLEY_GCC_VERSION) || \ + defined(JSON_HEDLEY_INTEL_VERSION) || \ + defined(JSON_HEDLEY_TINYC_VERSION) || \ + defined(JSON_HEDLEY_TI_ARMCL_VERSION) || \ + JSON_HEDLEY_TI_CL430_VERSION_CHECK(18,12,0) || \ + defined(JSON_HEDLEY_TI_CL2000_VERSION) || \ + defined(JSON_HEDLEY_TI_CL6X_VERSION) || \ + defined(JSON_HEDLEY_TI_CL7X_VERSION) || \ + defined(JSON_HEDLEY_TI_CLPRU_VERSION) || \ + defined(__clang__) +# define JSON_HEDLEY_IS_CONSTEXPR_(expr) ( \ + sizeof(void) != \ + sizeof(*( \ + 1 ? \ + ((void*) ((expr) * 0L) ) : \ +((struct { char v[sizeof(void) * 2]; } *) 1) \ + ) \ + ) \ + ) +# endif +#endif +#if defined(JSON_HEDLEY_IS_CONSTEXPR_) + #if !defined(JSON_HEDLEY_IS_CONSTANT) + #define JSON_HEDLEY_IS_CONSTANT(expr) JSON_HEDLEY_IS_CONSTEXPR_(expr) + #endif + #define JSON_HEDLEY_REQUIRE_CONSTEXPR(expr) (JSON_HEDLEY_IS_CONSTEXPR_(expr) ? (expr) : (-1)) +#else + #if !defined(JSON_HEDLEY_IS_CONSTANT) + #define JSON_HEDLEY_IS_CONSTANT(expr) (0) + #endif + #define JSON_HEDLEY_REQUIRE_CONSTEXPR(expr) (expr) +#endif + +#if defined(JSON_HEDLEY_BEGIN_C_DECLS) + #undef JSON_HEDLEY_BEGIN_C_DECLS +#endif +#if defined(JSON_HEDLEY_END_C_DECLS) + #undef JSON_HEDLEY_END_C_DECLS +#endif +#if defined(JSON_HEDLEY_C_DECL) + #undef JSON_HEDLEY_C_DECL +#endif +#if defined(__cplusplus) + #define JSON_HEDLEY_BEGIN_C_DECLS extern "C" { + #define JSON_HEDLEY_END_C_DECLS } + #define JSON_HEDLEY_C_DECL extern "C" +#else + #define JSON_HEDLEY_BEGIN_C_DECLS + #define JSON_HEDLEY_END_C_DECLS + #define JSON_HEDLEY_C_DECL +#endif + +#if defined(JSON_HEDLEY_STATIC_ASSERT) + #undef JSON_HEDLEY_STATIC_ASSERT +#endif +#if \ + !defined(__cplusplus) && ( \ + (defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 201112L)) || \ + JSON_HEDLEY_HAS_FEATURE(c_static_assert) || \ + JSON_HEDLEY_GCC_VERSION_CHECK(6,0,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) || \ + defined(_Static_assert) \ + ) +# define JSON_HEDLEY_STATIC_ASSERT(expr, message) _Static_assert(expr, message) +#elif \ + (defined(__cplusplus) && (__cplusplus >= 201103L)) || \ + JSON_HEDLEY_MSVC_VERSION_CHECK(16,0,0) +# define JSON_HEDLEY_STATIC_ASSERT(expr, message) JSON_HEDLEY_DIAGNOSTIC_DISABLE_CPP98_COMPAT_WRAP_(static_assert(expr, message)) +#else +# define JSON_HEDLEY_STATIC_ASSERT(expr, message) +#endif + +#if defined(JSON_HEDLEY_NULL) + #undef JSON_HEDLEY_NULL +#endif +#if defined(__cplusplus) + #if __cplusplus >= 201103L + #define JSON_HEDLEY_NULL JSON_HEDLEY_DIAGNOSTIC_DISABLE_CPP98_COMPAT_WRAP_(nullptr) + #elif defined(NULL) + #define JSON_HEDLEY_NULL NULL + #else + #define JSON_HEDLEY_NULL JSON_HEDLEY_STATIC_CAST(void*, 0) + #endif +#elif defined(NULL) + #define JSON_HEDLEY_NULL NULL +#else + #define JSON_HEDLEY_NULL ((void*) 0) +#endif + +#if defined(JSON_HEDLEY_MESSAGE) + #undef JSON_HEDLEY_MESSAGE +#endif +#if JSON_HEDLEY_HAS_WARNING("-Wunknown-pragmas") +# define JSON_HEDLEY_MESSAGE(msg) \ + JSON_HEDLEY_DIAGNOSTIC_PUSH \ + JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_PRAGMAS \ + JSON_HEDLEY_PRAGMA(message msg) \ + JSON_HEDLEY_DIAGNOSTIC_POP +#elif \ + JSON_HEDLEY_GCC_VERSION_CHECK(4,4,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) +# define JSON_HEDLEY_MESSAGE(msg) JSON_HEDLEY_PRAGMA(message msg) +#elif JSON_HEDLEY_CRAY_VERSION_CHECK(5,0,0) +# define JSON_HEDLEY_MESSAGE(msg) JSON_HEDLEY_PRAGMA(_CRI message msg) +#elif JSON_HEDLEY_IAR_VERSION_CHECK(8,0,0) +# define JSON_HEDLEY_MESSAGE(msg) JSON_HEDLEY_PRAGMA(message(msg)) +#elif JSON_HEDLEY_PELLES_VERSION_CHECK(2,0,0) +# define JSON_HEDLEY_MESSAGE(msg) JSON_HEDLEY_PRAGMA(message(msg)) +#else +# define JSON_HEDLEY_MESSAGE(msg) +#endif + +#if defined(JSON_HEDLEY_WARNING) + #undef JSON_HEDLEY_WARNING +#endif +#if JSON_HEDLEY_HAS_WARNING("-Wunknown-pragmas") +# define JSON_HEDLEY_WARNING(msg) \ + JSON_HEDLEY_DIAGNOSTIC_PUSH \ + JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_PRAGMAS \ + JSON_HEDLEY_PRAGMA(clang warning msg) \ + JSON_HEDLEY_DIAGNOSTIC_POP +#elif \ + JSON_HEDLEY_GCC_VERSION_CHECK(4,8,0) || \ + JSON_HEDLEY_PGI_VERSION_CHECK(18,4,0) || \ + JSON_HEDLEY_INTEL_VERSION_CHECK(13,0,0) +# define JSON_HEDLEY_WARNING(msg) JSON_HEDLEY_PRAGMA(GCC warning msg) +#elif JSON_HEDLEY_MSVC_VERSION_CHECK(15,0,0) +# define JSON_HEDLEY_WARNING(msg) JSON_HEDLEY_PRAGMA(message(msg)) +#else +# define JSON_HEDLEY_WARNING(msg) JSON_HEDLEY_MESSAGE(msg) +#endif + +#if defined(JSON_HEDLEY_REQUIRE) + #undef JSON_HEDLEY_REQUIRE +#endif +#if defined(JSON_HEDLEY_REQUIRE_MSG) + #undef JSON_HEDLEY_REQUIRE_MSG +#endif +#if JSON_HEDLEY_HAS_ATTRIBUTE(diagnose_if) +# if JSON_HEDLEY_HAS_WARNING("-Wgcc-compat") +# define JSON_HEDLEY_REQUIRE(expr) \ + JSON_HEDLEY_DIAGNOSTIC_PUSH \ + _Pragma("clang diagnostic ignored \"-Wgcc-compat\"") \ + __attribute__((diagnose_if(!(expr), #expr, "error"))) \ + JSON_HEDLEY_DIAGNOSTIC_POP +# define JSON_HEDLEY_REQUIRE_MSG(expr,msg) \ + JSON_HEDLEY_DIAGNOSTIC_PUSH \ + _Pragma("clang diagnostic ignored \"-Wgcc-compat\"") \ + __attribute__((diagnose_if(!(expr), msg, "error"))) \ + JSON_HEDLEY_DIAGNOSTIC_POP +# else +# define JSON_HEDLEY_REQUIRE(expr) __attribute__((diagnose_if(!(expr), #expr, "error"))) +# define JSON_HEDLEY_REQUIRE_MSG(expr,msg) __attribute__((diagnose_if(!(expr), msg, "error"))) +# endif +#else +# define JSON_HEDLEY_REQUIRE(expr) +# define JSON_HEDLEY_REQUIRE_MSG(expr,msg) +#endif + +#if defined(JSON_HEDLEY_FLAGS) + #undef JSON_HEDLEY_FLAGS +#endif +#if JSON_HEDLEY_HAS_ATTRIBUTE(flag_enum) + #define JSON_HEDLEY_FLAGS __attribute__((__flag_enum__)) +#endif + +#if defined(JSON_HEDLEY_FLAGS_CAST) + #undef JSON_HEDLEY_FLAGS_CAST +#endif +#if JSON_HEDLEY_INTEL_VERSION_CHECK(19,0,0) +# define JSON_HEDLEY_FLAGS_CAST(T, expr) (__extension__ ({ \ + JSON_HEDLEY_DIAGNOSTIC_PUSH \ + _Pragma("warning(disable:188)") \ + ((T) (expr)); \ + JSON_HEDLEY_DIAGNOSTIC_POP \ + })) +#else +# define JSON_HEDLEY_FLAGS_CAST(T, expr) JSON_HEDLEY_STATIC_CAST(T, expr) +#endif + +#if defined(JSON_HEDLEY_EMPTY_BASES) + #undef JSON_HEDLEY_EMPTY_BASES +#endif +#if JSON_HEDLEY_MSVC_VERSION_CHECK(19,0,23918) && !JSON_HEDLEY_MSVC_VERSION_CHECK(20,0,0) + #define JSON_HEDLEY_EMPTY_BASES __declspec(empty_bases) +#else + #define JSON_HEDLEY_EMPTY_BASES +#endif + +/* Remaining macros are deprecated. */ + +#if defined(JSON_HEDLEY_GCC_NOT_CLANG_VERSION_CHECK) + #undef JSON_HEDLEY_GCC_NOT_CLANG_VERSION_CHECK +#endif +#if defined(__clang__) + #define JSON_HEDLEY_GCC_NOT_CLANG_VERSION_CHECK(major,minor,patch) (0) +#else + #define JSON_HEDLEY_GCC_NOT_CLANG_VERSION_CHECK(major,minor,patch) JSON_HEDLEY_GCC_VERSION_CHECK(major,minor,patch) +#endif + +#if defined(JSON_HEDLEY_CLANG_HAS_ATTRIBUTE) + #undef JSON_HEDLEY_CLANG_HAS_ATTRIBUTE +#endif +#define JSON_HEDLEY_CLANG_HAS_ATTRIBUTE(attribute) JSON_HEDLEY_HAS_ATTRIBUTE(attribute) + +#if defined(JSON_HEDLEY_CLANG_HAS_CPP_ATTRIBUTE) + #undef JSON_HEDLEY_CLANG_HAS_CPP_ATTRIBUTE +#endif +#define JSON_HEDLEY_CLANG_HAS_CPP_ATTRIBUTE(attribute) JSON_HEDLEY_HAS_CPP_ATTRIBUTE(attribute) + +#if defined(JSON_HEDLEY_CLANG_HAS_BUILTIN) + #undef JSON_HEDLEY_CLANG_HAS_BUILTIN +#endif +#define JSON_HEDLEY_CLANG_HAS_BUILTIN(builtin) JSON_HEDLEY_HAS_BUILTIN(builtin) + +#if defined(JSON_HEDLEY_CLANG_HAS_FEATURE) + #undef JSON_HEDLEY_CLANG_HAS_FEATURE +#endif +#define JSON_HEDLEY_CLANG_HAS_FEATURE(feature) JSON_HEDLEY_HAS_FEATURE(feature) + +#if defined(JSON_HEDLEY_CLANG_HAS_EXTENSION) + #undef JSON_HEDLEY_CLANG_HAS_EXTENSION +#endif +#define JSON_HEDLEY_CLANG_HAS_EXTENSION(extension) JSON_HEDLEY_HAS_EXTENSION(extension) + +#if defined(JSON_HEDLEY_CLANG_HAS_DECLSPEC_DECLSPEC_ATTRIBUTE) + #undef JSON_HEDLEY_CLANG_HAS_DECLSPEC_DECLSPEC_ATTRIBUTE +#endif +#define JSON_HEDLEY_CLANG_HAS_DECLSPEC_ATTRIBUTE(attribute) JSON_HEDLEY_HAS_DECLSPEC_ATTRIBUTE(attribute) + +#if defined(JSON_HEDLEY_CLANG_HAS_WARNING) + #undef JSON_HEDLEY_CLANG_HAS_WARNING +#endif +#define JSON_HEDLEY_CLANG_HAS_WARNING(warning) JSON_HEDLEY_HAS_WARNING(warning) + +#endif /* !defined(JSON_HEDLEY_VERSION) || (JSON_HEDLEY_VERSION < X) */ + + +// This file contains all internal macro definitions +// You MUST include macro_unscope.hpp at the end of json.hpp to undef all of them + +// exclude unsupported compilers +#if !defined(JSON_SKIP_UNSUPPORTED_COMPILER_CHECK) + #if defined(__clang__) + #if (__clang_major__ * 10000 + __clang_minor__ * 100 + __clang_patchlevel__) < 30400 + #error "unsupported Clang version - see https://github.com/nlohmann/json#supported-compilers" + #endif + #elif defined(__GNUC__) && !(defined(__ICC) || defined(__INTEL_COMPILER)) + #if (__GNUC__ * 10000 + __GNUC_MINOR__ * 100 + __GNUC_PATCHLEVEL__) < 40800 + #error "unsupported GCC version - see https://github.com/nlohmann/json#supported-compilers" + #endif + #endif +#endif + +// C++ language standard detection +#if (defined(__cplusplus) && __cplusplus >= 202002L) || (defined(_MSVC_LANG) && _MSVC_LANG >= 202002L) + #define JSON_HAS_CPP_20 + #define JSON_HAS_CPP_17 + #define JSON_HAS_CPP_14 +#elif (defined(__cplusplus) && __cplusplus >= 201703L) || (defined(_HAS_CXX17) && _HAS_CXX17 == 1) // fix for issue #464 + #define JSON_HAS_CPP_17 + #define JSON_HAS_CPP_14 +#elif (defined(__cplusplus) && __cplusplus >= 201402L) || (defined(_HAS_CXX14) && _HAS_CXX14 == 1) + #define JSON_HAS_CPP_14 +#endif + +// disable float-equal warnings on GCC/clang +#if defined(__clang__) || defined(__GNUC__) || defined(__GNUG__) + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wfloat-equal" +#endif + +// disable documentation warnings on clang +#if defined(__clang__) + #pragma GCC diagnostic push + #pragma GCC diagnostic ignored "-Wdocumentation" +#endif + +// allow to disable exceptions +#if (defined(__cpp_exceptions) || defined(__EXCEPTIONS) || defined(_CPPUNWIND)) && !defined(JSON_NOEXCEPTION) + #define JSON_THROW(exception) throw exception + #define JSON_TRY try + #define JSON_CATCH(exception) catch(exception) + #define JSON_INTERNAL_CATCH(exception) catch(exception) +#else + #include + #define JSON_THROW(exception) std::abort() + #define JSON_TRY if(true) + #define JSON_CATCH(exception) if(false) + #define JSON_INTERNAL_CATCH(exception) if(false) +#endif + +// override exception macros +#if defined(JSON_THROW_USER) + #undef JSON_THROW + #define JSON_THROW JSON_THROW_USER +#endif +#if defined(JSON_TRY_USER) + #undef JSON_TRY + #define JSON_TRY JSON_TRY_USER +#endif +#if defined(JSON_CATCH_USER) + #undef JSON_CATCH + #define JSON_CATCH JSON_CATCH_USER + #undef JSON_INTERNAL_CATCH + #define JSON_INTERNAL_CATCH JSON_CATCH_USER +#endif +#if defined(JSON_INTERNAL_CATCH_USER) + #undef JSON_INTERNAL_CATCH + #define JSON_INTERNAL_CATCH JSON_INTERNAL_CATCH_USER +#endif + +// allow to override assert +#if !defined(JSON_ASSERT) + #include // assert + #define JSON_ASSERT(x) assert(x) +#endif + +/*! +@brief macro to briefly define a mapping between an enum and JSON +@def NLOHMANN_JSON_SERIALIZE_ENUM +@since version 3.4.0 +*/ +#define NLOHMANN_JSON_SERIALIZE_ENUM(ENUM_TYPE, ...) \ + template \ + inline void to_json(BasicJsonType& j, const ENUM_TYPE& e) \ + { \ + static_assert(std::is_enum::value, #ENUM_TYPE " must be an enum!"); \ + static const std::pair m[] = __VA_ARGS__; \ + auto it = std::find_if(std::begin(m), std::end(m), \ + [e](const std::pair& ej_pair) -> bool \ + { \ + return ej_pair.first == e; \ + }); \ + j = ((it != std::end(m)) ? it : std::begin(m))->second; \ + } \ + template \ + inline void from_json(const BasicJsonType& j, ENUM_TYPE& e) \ + { \ + static_assert(std::is_enum::value, #ENUM_TYPE " must be an enum!"); \ + static const std::pair m[] = __VA_ARGS__; \ + auto it = std::find_if(std::begin(m), std::end(m), \ + [&j](const std::pair& ej_pair) -> bool \ + { \ + return ej_pair.second == j; \ + }); \ + e = ((it != std::end(m)) ? it : std::begin(m))->first; \ + } + +// Ugly macros to avoid uglier copy-paste when specializing basic_json. They +// may be removed in the future once the class is split. + +#define NLOHMANN_BASIC_JSON_TPL_DECLARATION \ + template class ObjectType, \ + template class ArrayType, \ + class StringType, class BooleanType, class NumberIntegerType, \ + class NumberUnsignedType, class NumberFloatType, \ + template class AllocatorType, \ + template class JSONSerializer, \ + class BinaryType> + +#define NLOHMANN_BASIC_JSON_TPL \ + basic_json + +// Macros to simplify conversion from/to types + +#define NLOHMANN_JSON_EXPAND( x ) x +#define NLOHMANN_JSON_GET_MACRO(_1, _2, _3, _4, _5, _6, _7, _8, _9, _10, _11, _12, _13, _14, _15, _16, _17, _18, _19, _20, _21, _22, _23, _24, _25, _26, _27, _28, _29, _30, _31, _32, _33, _34, _35, _36, _37, _38, _39, _40, _41, _42, _43, _44, _45, _46, _47, _48, _49, _50, _51, _52, _53, _54, _55, _56, _57, _58, _59, _60, _61, _62, _63, _64, NAME,...) NAME +#define NLOHMANN_JSON_PASTE(...) NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_GET_MACRO(__VA_ARGS__, \ + NLOHMANN_JSON_PASTE64, \ + NLOHMANN_JSON_PASTE63, \ + NLOHMANN_JSON_PASTE62, \ + NLOHMANN_JSON_PASTE61, \ + NLOHMANN_JSON_PASTE60, \ + NLOHMANN_JSON_PASTE59, \ + NLOHMANN_JSON_PASTE58, \ + NLOHMANN_JSON_PASTE57, \ + NLOHMANN_JSON_PASTE56, \ + NLOHMANN_JSON_PASTE55, \ + NLOHMANN_JSON_PASTE54, \ + NLOHMANN_JSON_PASTE53, \ + NLOHMANN_JSON_PASTE52, \ + NLOHMANN_JSON_PASTE51, \ + NLOHMANN_JSON_PASTE50, \ + NLOHMANN_JSON_PASTE49, \ + NLOHMANN_JSON_PASTE48, \ + NLOHMANN_JSON_PASTE47, \ + NLOHMANN_JSON_PASTE46, \ + NLOHMANN_JSON_PASTE45, \ + NLOHMANN_JSON_PASTE44, \ + NLOHMANN_JSON_PASTE43, \ + NLOHMANN_JSON_PASTE42, \ + NLOHMANN_JSON_PASTE41, \ + NLOHMANN_JSON_PASTE40, \ + NLOHMANN_JSON_PASTE39, \ + NLOHMANN_JSON_PASTE38, \ + NLOHMANN_JSON_PASTE37, \ + NLOHMANN_JSON_PASTE36, \ + NLOHMANN_JSON_PASTE35, \ + NLOHMANN_JSON_PASTE34, \ + NLOHMANN_JSON_PASTE33, \ + NLOHMANN_JSON_PASTE32, \ + NLOHMANN_JSON_PASTE31, \ + NLOHMANN_JSON_PASTE30, \ + NLOHMANN_JSON_PASTE29, \ + NLOHMANN_JSON_PASTE28, \ + NLOHMANN_JSON_PASTE27, \ + NLOHMANN_JSON_PASTE26, \ + NLOHMANN_JSON_PASTE25, \ + NLOHMANN_JSON_PASTE24, \ + NLOHMANN_JSON_PASTE23, \ + NLOHMANN_JSON_PASTE22, \ + NLOHMANN_JSON_PASTE21, \ + NLOHMANN_JSON_PASTE20, \ + NLOHMANN_JSON_PASTE19, \ + NLOHMANN_JSON_PASTE18, \ + NLOHMANN_JSON_PASTE17, \ + NLOHMANN_JSON_PASTE16, \ + NLOHMANN_JSON_PASTE15, \ + NLOHMANN_JSON_PASTE14, \ + NLOHMANN_JSON_PASTE13, \ + NLOHMANN_JSON_PASTE12, \ + NLOHMANN_JSON_PASTE11, \ + NLOHMANN_JSON_PASTE10, \ + NLOHMANN_JSON_PASTE9, \ + NLOHMANN_JSON_PASTE8, \ + NLOHMANN_JSON_PASTE7, \ + NLOHMANN_JSON_PASTE6, \ + NLOHMANN_JSON_PASTE5, \ + NLOHMANN_JSON_PASTE4, \ + NLOHMANN_JSON_PASTE3, \ + NLOHMANN_JSON_PASTE2, \ + NLOHMANN_JSON_PASTE1)(__VA_ARGS__)) +#define NLOHMANN_JSON_PASTE2(func, v1) func(v1) +#define NLOHMANN_JSON_PASTE3(func, v1, v2) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE2(func, v2) +#define NLOHMANN_JSON_PASTE4(func, v1, v2, v3) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE3(func, v2, v3) +#define NLOHMANN_JSON_PASTE5(func, v1, v2, v3, v4) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE4(func, v2, v3, v4) +#define NLOHMANN_JSON_PASTE6(func, v1, v2, v3, v4, v5) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE5(func, v2, v3, v4, v5) +#define NLOHMANN_JSON_PASTE7(func, v1, v2, v3, v4, v5, v6) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE6(func, v2, v3, v4, v5, v6) +#define NLOHMANN_JSON_PASTE8(func, v1, v2, v3, v4, v5, v6, v7) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE7(func, v2, v3, v4, v5, v6, v7) +#define NLOHMANN_JSON_PASTE9(func, v1, v2, v3, v4, v5, v6, v7, v8) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE8(func, v2, v3, v4, v5, v6, v7, v8) +#define NLOHMANN_JSON_PASTE10(func, v1, v2, v3, v4, v5, v6, v7, v8, v9) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE9(func, v2, v3, v4, v5, v6, v7, v8, v9) +#define NLOHMANN_JSON_PASTE11(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE10(func, v2, v3, v4, v5, v6, v7, v8, v9, v10) +#define NLOHMANN_JSON_PASTE12(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE11(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11) +#define NLOHMANN_JSON_PASTE13(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE12(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12) +#define NLOHMANN_JSON_PASTE14(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE13(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13) +#define NLOHMANN_JSON_PASTE15(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE14(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14) +#define NLOHMANN_JSON_PASTE16(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE15(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15) +#define NLOHMANN_JSON_PASTE17(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE16(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16) +#define NLOHMANN_JSON_PASTE18(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE17(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17) +#define NLOHMANN_JSON_PASTE19(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE18(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18) +#define NLOHMANN_JSON_PASTE20(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE19(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19) +#define NLOHMANN_JSON_PASTE21(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE20(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20) +#define NLOHMANN_JSON_PASTE22(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE21(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21) +#define NLOHMANN_JSON_PASTE23(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE22(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22) +#define NLOHMANN_JSON_PASTE24(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE23(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23) +#define NLOHMANN_JSON_PASTE25(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE24(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24) +#define NLOHMANN_JSON_PASTE26(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE25(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25) +#define NLOHMANN_JSON_PASTE27(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE26(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26) +#define NLOHMANN_JSON_PASTE28(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE27(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27) +#define NLOHMANN_JSON_PASTE29(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE28(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28) +#define NLOHMANN_JSON_PASTE30(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE29(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29) +#define NLOHMANN_JSON_PASTE31(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE30(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30) +#define NLOHMANN_JSON_PASTE32(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE31(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31) +#define NLOHMANN_JSON_PASTE33(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE32(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32) +#define NLOHMANN_JSON_PASTE34(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE33(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33) +#define NLOHMANN_JSON_PASTE35(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE34(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34) +#define NLOHMANN_JSON_PASTE36(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE35(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35) +#define NLOHMANN_JSON_PASTE37(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE36(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36) +#define NLOHMANN_JSON_PASTE38(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE37(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37) +#define NLOHMANN_JSON_PASTE39(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE38(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38) +#define NLOHMANN_JSON_PASTE40(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE39(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39) +#define NLOHMANN_JSON_PASTE41(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE40(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40) +#define NLOHMANN_JSON_PASTE42(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE41(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41) +#define NLOHMANN_JSON_PASTE43(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE42(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42) +#define NLOHMANN_JSON_PASTE44(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE43(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43) +#define NLOHMANN_JSON_PASTE45(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE44(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44) +#define NLOHMANN_JSON_PASTE46(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE45(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45) +#define NLOHMANN_JSON_PASTE47(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE46(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46) +#define NLOHMANN_JSON_PASTE48(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE47(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47) +#define NLOHMANN_JSON_PASTE49(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE48(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48) +#define NLOHMANN_JSON_PASTE50(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE49(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49) +#define NLOHMANN_JSON_PASTE51(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE50(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50) +#define NLOHMANN_JSON_PASTE52(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE51(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51) +#define NLOHMANN_JSON_PASTE53(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE52(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52) +#define NLOHMANN_JSON_PASTE54(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE53(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53) +#define NLOHMANN_JSON_PASTE55(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53, v54) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE54(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53, v54) +#define NLOHMANN_JSON_PASTE56(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53, v54, v55) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE55(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53, v54, v55) +#define NLOHMANN_JSON_PASTE57(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53, v54, v55, v56) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE56(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53, v54, v55, v56) +#define NLOHMANN_JSON_PASTE58(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53, v54, v55, v56, v57) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE57(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53, v54, v55, v56, v57) +#define NLOHMANN_JSON_PASTE59(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53, v54, v55, v56, v57, v58) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE58(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53, v54, v55, v56, v57, v58) +#define NLOHMANN_JSON_PASTE60(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53, v54, v55, v56, v57, v58, v59) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE59(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53, v54, v55, v56, v57, v58, v59) +#define NLOHMANN_JSON_PASTE61(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53, v54, v55, v56, v57, v58, v59, v60) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE60(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53, v54, v55, v56, v57, v58, v59, v60) +#define NLOHMANN_JSON_PASTE62(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53, v54, v55, v56, v57, v58, v59, v60, v61) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE61(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53, v54, v55, v56, v57, v58, v59, v60, v61) +#define NLOHMANN_JSON_PASTE63(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53, v54, v55, v56, v57, v58, v59, v60, v61, v62) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE62(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53, v54, v55, v56, v57, v58, v59, v60, v61, v62) +#define NLOHMANN_JSON_PASTE64(func, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53, v54, v55, v56, v57, v58, v59, v60, v61, v62, v63) NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE63(func, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v50, v51, v52, v53, v54, v55, v56, v57, v58, v59, v60, v61, v62, v63) + +#define NLOHMANN_JSON_TO(v1) nlohmann_json_j[#v1] = nlohmann_json_t.v1; +#define NLOHMANN_JSON_FROM(v1) nlohmann_json_j.at(#v1).get_to(nlohmann_json_t.v1); + +/*! +@brief macro +@def NLOHMANN_DEFINE_TYPE_INTRUSIVE +@since version 3.9.0 +*/ +#define NLOHMANN_DEFINE_TYPE_INTRUSIVE(Type, ...) \ + friend void to_json(nlohmann::json& nlohmann_json_j, const Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_TO, __VA_ARGS__)) } \ + friend void from_json(const nlohmann::json& nlohmann_json_j, Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_FROM, __VA_ARGS__)) } + +/*! +@brief macro +@def NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE +@since version 3.9.0 +*/ +#define NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE(Type, ...) \ + inline void to_json(nlohmann::json& nlohmann_json_j, const Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_TO, __VA_ARGS__)) } \ + inline void from_json(const nlohmann::json& nlohmann_json_j, Type& nlohmann_json_t) { NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_PASTE(NLOHMANN_JSON_FROM, __VA_ARGS__)) } + +#ifndef JSON_USE_IMPLICIT_CONVERSIONS + #define JSON_USE_IMPLICIT_CONVERSIONS 1 +#endif + +#if JSON_USE_IMPLICIT_CONVERSIONS + #define JSON_EXPLICIT +#else + #define JSON_EXPLICIT explicit +#endif + + +namespace nlohmann +{ +namespace detail +{ +//////////////// +// exceptions // +//////////////// + +/*! +@brief general exception of the @ref basic_json class + +This class is an extension of `std::exception` objects with a member @a id for +exception ids. It is used as the base class for all exceptions thrown by the +@ref basic_json class. This class can hence be used as "wildcard" to catch +exceptions. + +Subclasses: +- @ref parse_error for exceptions indicating a parse error +- @ref invalid_iterator for exceptions indicating errors with iterators +- @ref type_error for exceptions indicating executing a member function with + a wrong type +- @ref out_of_range for exceptions indicating access out of the defined range +- @ref other_error for exceptions indicating other library errors + +@internal +@note To have nothrow-copy-constructible exceptions, we internally use + `std::runtime_error` which can cope with arbitrary-length error messages. + Intermediate strings are built with static functions and then passed to + the actual constructor. +@endinternal + +@liveexample{The following code shows how arbitrary library exceptions can be +caught.,exception} + +@since version 3.0.0 +*/ +class exception : public std::exception +{ + public: + /// returns the explanatory string + JSON_HEDLEY_RETURNS_NON_NULL + const char* what() const noexcept override + { + return m.what(); + } + + /// the id of the exception + const int id; + + protected: + JSON_HEDLEY_NON_NULL(3) + exception(int id_, const char* what_arg) : id(id_), m(what_arg) {} + + static std::string name(const std::string& ename, int id_) + { + return "[json.exception." + ename + "." + std::to_string(id_) + "] "; + } + + private: + /// an exception object as storage for error messages + std::runtime_error m; +}; + +/*! +@brief exception indicating a parse error + +This exception is thrown by the library when a parse error occurs. Parse errors +can occur during the deserialization of JSON text, CBOR, MessagePack, as well +as when using JSON Patch. + +Member @a byte holds the byte index of the last read character in the input +file. + +Exceptions have ids 1xx. + +name / id | example message | description +------------------------------ | --------------- | ------------------------- +json.exception.parse_error.101 | parse error at 2: unexpected end of input; expected string literal | This error indicates a syntax error while deserializing a JSON text. The error message describes that an unexpected token (character) was encountered, and the member @a byte indicates the error position. +json.exception.parse_error.102 | parse error at 14: missing or wrong low surrogate | JSON uses the `\uxxxx` format to describe Unicode characters. Code points above above 0xFFFF are split into two `\uxxxx` entries ("surrogate pairs"). This error indicates that the surrogate pair is incomplete or contains an invalid code point. +json.exception.parse_error.103 | parse error: code points above 0x10FFFF are invalid | Unicode supports code points up to 0x10FFFF. Code points above 0x10FFFF are invalid. +json.exception.parse_error.104 | parse error: JSON patch must be an array of objects | [RFC 6902](https://tools.ietf.org/html/rfc6902) requires a JSON Patch document to be a JSON document that represents an array of objects. +json.exception.parse_error.105 | parse error: operation must have string member 'op' | An operation of a JSON Patch document must contain exactly one "op" member, whose value indicates the operation to perform. Its value must be one of "add", "remove", "replace", "move", "copy", or "test"; other values are errors. +json.exception.parse_error.106 | parse error: array index '01' must not begin with '0' | An array index in a JSON Pointer ([RFC 6901](https://tools.ietf.org/html/rfc6901)) may be `0` or any number without a leading `0`. +json.exception.parse_error.107 | parse error: JSON pointer must be empty or begin with '/' - was: 'foo' | A JSON Pointer must be a Unicode string containing a sequence of zero or more reference tokens, each prefixed by a `/` character. +json.exception.parse_error.108 | parse error: escape character '~' must be followed with '0' or '1' | In a JSON Pointer, only `~0` and `~1` are valid escape sequences. +json.exception.parse_error.109 | parse error: array index 'one' is not a number | A JSON Pointer array index must be a number. +json.exception.parse_error.110 | parse error at 1: cannot read 2 bytes from vector | When parsing CBOR or MessagePack, the byte vector ends before the complete value has been read. +json.exception.parse_error.112 | parse error at 1: error reading CBOR; last byte: 0xF8 | Not all types of CBOR or MessagePack are supported. This exception occurs if an unsupported byte was read. +json.exception.parse_error.113 | parse error at 2: expected a CBOR string; last byte: 0x98 | While parsing a map key, a value that is not a string has been read. +json.exception.parse_error.114 | parse error: Unsupported BSON record type 0x0F | The parsing of the corresponding BSON record type is not implemented (yet). +json.exception.parse_error.115 | parse error at byte 5: syntax error while parsing UBJSON high-precision number: invalid number text: 1A | A UBJSON high-precision number could not be parsed. + +@note For an input with n bytes, 1 is the index of the first character and n+1 + is the index of the terminating null byte or the end of file. This also + holds true when reading a byte vector (CBOR or MessagePack). + +@liveexample{The following code shows how a `parse_error` exception can be +caught.,parse_error} + +@sa - @ref exception for the base class of the library exceptions +@sa - @ref invalid_iterator for exceptions indicating errors with iterators +@sa - @ref type_error for exceptions indicating executing a member function with + a wrong type +@sa - @ref out_of_range for exceptions indicating access out of the defined range +@sa - @ref other_error for exceptions indicating other library errors + +@since version 3.0.0 +*/ +class parse_error : public exception +{ + public: + /*! + @brief create a parse error exception + @param[in] id_ the id of the exception + @param[in] pos the position where the error occurred (or with + chars_read_total=0 if the position cannot be + determined) + @param[in] what_arg the explanatory string + @return parse_error object + */ + static parse_error create(int id_, const position_t& pos, const std::string& what_arg) + { + std::string w = exception::name("parse_error", id_) + "parse error" + + position_string(pos) + ": " + what_arg; + return parse_error(id_, pos.chars_read_total, w.c_str()); + } + + static parse_error create(int id_, std::size_t byte_, const std::string& what_arg) + { + std::string w = exception::name("parse_error", id_) + "parse error" + + (byte_ != 0 ? (" at byte " + std::to_string(byte_)) : "") + + ": " + what_arg; + return parse_error(id_, byte_, w.c_str()); + } + + /*! + @brief byte index of the parse error + + The byte index of the last read character in the input file. + + @note For an input with n bytes, 1 is the index of the first character and + n+1 is the index of the terminating null byte or the end of file. + This also holds true when reading a byte vector (CBOR or MessagePack). + */ + const std::size_t byte; + + private: + parse_error(int id_, std::size_t byte_, const char* what_arg) + : exception(id_, what_arg), byte(byte_) {} + + static std::string position_string(const position_t& pos) + { + return " at line " + std::to_string(pos.lines_read + 1) + + ", column " + std::to_string(pos.chars_read_current_line); + } +}; + +/*! +@brief exception indicating errors with iterators + +This exception is thrown if iterators passed to a library function do not match +the expected semantics. + +Exceptions have ids 2xx. + +name / id | example message | description +----------------------------------- | --------------- | ------------------------- +json.exception.invalid_iterator.201 | iterators are not compatible | The iterators passed to constructor @ref basic_json(InputIT first, InputIT last) are not compatible, meaning they do not belong to the same container. Therefore, the range (@a first, @a last) is invalid. +json.exception.invalid_iterator.202 | iterator does not fit current value | In an erase or insert function, the passed iterator @a pos does not belong to the JSON value for which the function was called. It hence does not define a valid position for the deletion/insertion. +json.exception.invalid_iterator.203 | iterators do not fit current value | Either iterator passed to function @ref erase(IteratorType first, IteratorType last) does not belong to the JSON value from which values shall be erased. It hence does not define a valid range to delete values from. +json.exception.invalid_iterator.204 | iterators out of range | When an iterator range for a primitive type (number, boolean, or string) is passed to a constructor or an erase function, this range has to be exactly (@ref begin(), @ref end()), because this is the only way the single stored value is expressed. All other ranges are invalid. +json.exception.invalid_iterator.205 | iterator out of range | When an iterator for a primitive type (number, boolean, or string) is passed to an erase function, the iterator has to be the @ref begin() iterator, because it is the only way to address the stored value. All other iterators are invalid. +json.exception.invalid_iterator.206 | cannot construct with iterators from null | The iterators passed to constructor @ref basic_json(InputIT first, InputIT last) belong to a JSON null value and hence to not define a valid range. +json.exception.invalid_iterator.207 | cannot use key() for non-object iterators | The key() member function can only be used on iterators belonging to a JSON object, because other types do not have a concept of a key. +json.exception.invalid_iterator.208 | cannot use operator[] for object iterators | The operator[] to specify a concrete offset cannot be used on iterators belonging to a JSON object, because JSON objects are unordered. +json.exception.invalid_iterator.209 | cannot use offsets with object iterators | The offset operators (+, -, +=, -=) cannot be used on iterators belonging to a JSON object, because JSON objects are unordered. +json.exception.invalid_iterator.210 | iterators do not fit | The iterator range passed to the insert function are not compatible, meaning they do not belong to the same container. Therefore, the range (@a first, @a last) is invalid. +json.exception.invalid_iterator.211 | passed iterators may not belong to container | The iterator range passed to the insert function must not be a subrange of the container to insert to. +json.exception.invalid_iterator.212 | cannot compare iterators of different containers | When two iterators are compared, they must belong to the same container. +json.exception.invalid_iterator.213 | cannot compare order of object iterators | The order of object iterators cannot be compared, because JSON objects are unordered. +json.exception.invalid_iterator.214 | cannot get value | Cannot get value for iterator: Either the iterator belongs to a null value or it is an iterator to a primitive type (number, boolean, or string), but the iterator is different to @ref begin(). + +@liveexample{The following code shows how an `invalid_iterator` exception can be +caught.,invalid_iterator} + +@sa - @ref exception for the base class of the library exceptions +@sa - @ref parse_error for exceptions indicating a parse error +@sa - @ref type_error for exceptions indicating executing a member function with + a wrong type +@sa - @ref out_of_range for exceptions indicating access out of the defined range +@sa - @ref other_error for exceptions indicating other library errors + +@since version 3.0.0 +*/ +class invalid_iterator : public exception +{ + public: + static invalid_iterator create(int id_, const std::string& what_arg) + { + std::string w = exception::name("invalid_iterator", id_) + what_arg; + return invalid_iterator(id_, w.c_str()); + } + + private: + JSON_HEDLEY_NON_NULL(3) + invalid_iterator(int id_, const char* what_arg) + : exception(id_, what_arg) {} +}; + +/*! +@brief exception indicating executing a member function with a wrong type + +This exception is thrown in case of a type error; that is, a library function is +executed on a JSON value whose type does not match the expected semantics. + +Exceptions have ids 3xx. + +name / id | example message | description +----------------------------- | --------------- | ------------------------- +json.exception.type_error.301 | cannot create object from initializer list | To create an object from an initializer list, the initializer list must consist only of a list of pairs whose first element is a string. When this constraint is violated, an array is created instead. +json.exception.type_error.302 | type must be object, but is array | During implicit or explicit value conversion, the JSON type must be compatible to the target type. For instance, a JSON string can only be converted into string types, but not into numbers or boolean types. +json.exception.type_error.303 | incompatible ReferenceType for get_ref, actual type is object | To retrieve a reference to a value stored in a @ref basic_json object with @ref get_ref, the type of the reference must match the value type. For instance, for a JSON array, the @a ReferenceType must be @ref array_t &. +json.exception.type_error.304 | cannot use at() with string | The @ref at() member functions can only be executed for certain JSON types. +json.exception.type_error.305 | cannot use operator[] with string | The @ref operator[] member functions can only be executed for certain JSON types. +json.exception.type_error.306 | cannot use value() with string | The @ref value() member functions can only be executed for certain JSON types. +json.exception.type_error.307 | cannot use erase() with string | The @ref erase() member functions can only be executed for certain JSON types. +json.exception.type_error.308 | cannot use push_back() with string | The @ref push_back() and @ref operator+= member functions can only be executed for certain JSON types. +json.exception.type_error.309 | cannot use insert() with | The @ref insert() member functions can only be executed for certain JSON types. +json.exception.type_error.310 | cannot use swap() with number | The @ref swap() member functions can only be executed for certain JSON types. +json.exception.type_error.311 | cannot use emplace_back() with string | The @ref emplace_back() member function can only be executed for certain JSON types. +json.exception.type_error.312 | cannot use update() with string | The @ref update() member functions can only be executed for certain JSON types. +json.exception.type_error.313 | invalid value to unflatten | The @ref unflatten function converts an object whose keys are JSON Pointers back into an arbitrary nested JSON value. The JSON Pointers must not overlap, because then the resulting value would not be well defined. +json.exception.type_error.314 | only objects can be unflattened | The @ref unflatten function only works for an object whose keys are JSON Pointers. +json.exception.type_error.315 | values in object must be primitive | The @ref unflatten function only works for an object whose keys are JSON Pointers and whose values are primitive. +json.exception.type_error.316 | invalid UTF-8 byte at index 10: 0x7E | The @ref dump function only works with UTF-8 encoded strings; that is, if you assign a `std::string` to a JSON value, make sure it is UTF-8 encoded. | +json.exception.type_error.317 | JSON value cannot be serialized to requested format | The dynamic type of the object cannot be represented in the requested serialization format (e.g. a raw `true` or `null` JSON object cannot be serialized to BSON) | + +@liveexample{The following code shows how a `type_error` exception can be +caught.,type_error} + +@sa - @ref exception for the base class of the library exceptions +@sa - @ref parse_error for exceptions indicating a parse error +@sa - @ref invalid_iterator for exceptions indicating errors with iterators +@sa - @ref out_of_range for exceptions indicating access out of the defined range +@sa - @ref other_error for exceptions indicating other library errors + +@since version 3.0.0 +*/ +class type_error : public exception +{ + public: + static type_error create(int id_, const std::string& what_arg) + { + std::string w = exception::name("type_error", id_) + what_arg; + return type_error(id_, w.c_str()); + } + + private: + JSON_HEDLEY_NON_NULL(3) + type_error(int id_, const char* what_arg) : exception(id_, what_arg) {} +}; + +/*! +@brief exception indicating access out of the defined range + +This exception is thrown in case a library function is called on an input +parameter that exceeds the expected range, for instance in case of array +indices or nonexisting object keys. + +Exceptions have ids 4xx. + +name / id | example message | description +------------------------------- | --------------- | ------------------------- +json.exception.out_of_range.401 | array index 3 is out of range | The provided array index @a i is larger than @a size-1. +json.exception.out_of_range.402 | array index '-' (3) is out of range | The special array index `-` in a JSON Pointer never describes a valid element of the array, but the index past the end. That is, it can only be used to add elements at this position, but not to read it. +json.exception.out_of_range.403 | key 'foo' not found | The provided key was not found in the JSON object. +json.exception.out_of_range.404 | unresolved reference token 'foo' | A reference token in a JSON Pointer could not be resolved. +json.exception.out_of_range.405 | JSON pointer has no parent | The JSON Patch operations 'remove' and 'add' can not be applied to the root element of the JSON value. +json.exception.out_of_range.406 | number overflow parsing '10E1000' | A parsed number could not be stored as without changing it to NaN or INF. +json.exception.out_of_range.407 | number overflow serializing '9223372036854775808' | UBJSON and BSON only support integer numbers up to 9223372036854775807. (until version 3.8.0) | +json.exception.out_of_range.408 | excessive array size: 8658170730974374167 | The size (following `#`) of an UBJSON array or object exceeds the maximal capacity. | +json.exception.out_of_range.409 | BSON key cannot contain code point U+0000 (at byte 2) | Key identifiers to be serialized to BSON cannot contain code point U+0000, since the key is stored as zero-terminated c-string | + +@liveexample{The following code shows how an `out_of_range` exception can be +caught.,out_of_range} + +@sa - @ref exception for the base class of the library exceptions +@sa - @ref parse_error for exceptions indicating a parse error +@sa - @ref invalid_iterator for exceptions indicating errors with iterators +@sa - @ref type_error for exceptions indicating executing a member function with + a wrong type +@sa - @ref other_error for exceptions indicating other library errors + +@since version 3.0.0 +*/ +class out_of_range : public exception +{ + public: + static out_of_range create(int id_, const std::string& what_arg) + { + std::string w = exception::name("out_of_range", id_) + what_arg; + return out_of_range(id_, w.c_str()); + } + + private: + JSON_HEDLEY_NON_NULL(3) + out_of_range(int id_, const char* what_arg) : exception(id_, what_arg) {} +}; + +/*! +@brief exception indicating other library errors + +This exception is thrown in case of errors that cannot be classified with the +other exception types. + +Exceptions have ids 5xx. + +name / id | example message | description +------------------------------ | --------------- | ------------------------- +json.exception.other_error.501 | unsuccessful: {"op":"test","path":"/baz", "value":"bar"} | A JSON Patch operation 'test' failed. The unsuccessful operation is also printed. + +@sa - @ref exception for the base class of the library exceptions +@sa - @ref parse_error for exceptions indicating a parse error +@sa - @ref invalid_iterator for exceptions indicating errors with iterators +@sa - @ref type_error for exceptions indicating executing a member function with + a wrong type +@sa - @ref out_of_range for exceptions indicating access out of the defined range + +@liveexample{The following code shows how an `other_error` exception can be +caught.,other_error} + +@since version 3.0.0 +*/ +class other_error : public exception +{ + public: + static other_error create(int id_, const std::string& what_arg) + { + std::string w = exception::name("other_error", id_) + what_arg; + return other_error(id_, w.c_str()); + } + + private: + JSON_HEDLEY_NON_NULL(3) + other_error(int id_, const char* what_arg) : exception(id_, what_arg) {} +}; +} // namespace detail +} // namespace nlohmann + +// #include + +// #include + + +#include // size_t +#include // conditional, enable_if, false_type, integral_constant, is_constructible, is_integral, is_same, remove_cv, remove_reference, true_type + +namespace nlohmann +{ +namespace detail +{ +// alias templates to reduce boilerplate +template +using enable_if_t = typename std::enable_if::type; + +template +using uncvref_t = typename std::remove_cv::type>::type; + +// implementation of C++14 index_sequence and affiliates +// source: https://stackoverflow.com/a/32223343 +template +struct index_sequence +{ + using type = index_sequence; + using value_type = std::size_t; + static constexpr std::size_t size() noexcept + { + return sizeof...(Ints); + } +}; + +template +struct merge_and_renumber; + +template +struct merge_and_renumber, index_sequence> + : index_sequence < I1..., (sizeof...(I1) + I2)... > {}; + +template +struct make_index_sequence + : merge_and_renumber < typename make_index_sequence < N / 2 >::type, + typename make_index_sequence < N - N / 2 >::type > {}; + +template<> struct make_index_sequence<0> : index_sequence<> {}; +template<> struct make_index_sequence<1> : index_sequence<0> {}; + +template +using index_sequence_for = make_index_sequence; + +// dispatch utility (taken from ranges-v3) +template struct priority_tag : priority_tag < N - 1 > {}; +template<> struct priority_tag<0> {}; + +// taken from ranges-v3 +template +struct static_const +{ + static constexpr T value{}; +}; + +template +constexpr T static_const::value; +} // namespace detail +} // namespace nlohmann + +// #include + + +#include // numeric_limits +#include // false_type, is_constructible, is_integral, is_same, true_type +#include // declval + +// #include + + +#include // random_access_iterator_tag + +// #include + + +namespace nlohmann +{ +namespace detail +{ +template struct make_void +{ + using type = void; +}; +template using void_t = typename make_void::type; +} // namespace detail +} // namespace nlohmann + +// #include + + +namespace nlohmann +{ +namespace detail +{ +template +struct iterator_types {}; + +template +struct iterator_types < + It, + void_t> +{ + using difference_type = typename It::difference_type; + using value_type = typename It::value_type; + using pointer = typename It::pointer; + using reference = typename It::reference; + using iterator_category = typename It::iterator_category; +}; + +// This is required as some compilers implement std::iterator_traits in a way that +// doesn't work with SFINAE. See https://github.com/nlohmann/json/issues/1341. +template +struct iterator_traits +{ +}; + +template +struct iterator_traits < T, enable_if_t < !std::is_pointer::value >> + : iterator_types +{ +}; + +template +struct iterator_traits::value>> +{ + using iterator_category = std::random_access_iterator_tag; + using value_type = T; + using difference_type = ptrdiff_t; + using pointer = T*; + using reference = T&; +}; +} // namespace detail +} // namespace nlohmann + +// #include + +// #include + +// #include + + +#include + +// #include + + +// https://en.cppreference.com/w/cpp/experimental/is_detected +namespace nlohmann +{ +namespace detail +{ +struct nonesuch +{ + nonesuch() = delete; + ~nonesuch() = delete; + nonesuch(nonesuch const&) = delete; + nonesuch(nonesuch const&&) = delete; + void operator=(nonesuch const&) = delete; + void operator=(nonesuch&&) = delete; +}; + +template class Op, + class... Args> +struct detector +{ + using value_t = std::false_type; + using type = Default; +}; + +template class Op, class... Args> +struct detector>, Op, Args...> +{ + using value_t = std::true_type; + using type = Op; +}; + +template class Op, class... Args> +using is_detected = typename detector::value_t; + +template class Op, class... Args> +using detected_t = typename detector::type; + +template class Op, class... Args> +using detected_or = detector; + +template class Op, class... Args> +using detected_or_t = typename detected_or::type; + +template class Op, class... Args> +using is_detected_exact = std::is_same>; + +template class Op, class... Args> +using is_detected_convertible = + std::is_convertible, To>; +} // namespace detail +} // namespace nlohmann + +// #include +#ifndef INCLUDE_NLOHMANN_JSON_FWD_HPP_ +#define INCLUDE_NLOHMANN_JSON_FWD_HPP_ + +#include // int64_t, uint64_t +#include // map +#include // allocator +#include // string +#include // vector + +/*! +@brief namespace for Niels Lohmann +@see https://github.com/nlohmann +@since version 1.0.0 +*/ +namespace nlohmann +{ +/*! +@brief default JSONSerializer template argument + +This serializer ignores the template arguments and uses ADL +([argument-dependent lookup](https://en.cppreference.com/w/cpp/language/adl)) +for serialization. +*/ +template +struct adl_serializer; + +template class ObjectType = + std::map, + template class ArrayType = std::vector, + class StringType = std::string, class BooleanType = bool, + class NumberIntegerType = std::int64_t, + class NumberUnsignedType = std::uint64_t, + class NumberFloatType = double, + template class AllocatorType = std::allocator, + template class JSONSerializer = + adl_serializer, + class BinaryType = std::vector> +class basic_json; + +/*! +@brief JSON Pointer + +A JSON pointer defines a string syntax for identifying a specific value +within a JSON document. It can be used with functions `at` and +`operator[]`. Furthermore, JSON pointers are the base for JSON patches. + +@sa [RFC 6901](https://tools.ietf.org/html/rfc6901) + +@since version 2.0.0 +*/ +template +class json_pointer; + +/*! +@brief default JSON class + +This type is the default specialization of the @ref basic_json class which +uses the standard template types. + +@since version 1.0.0 +*/ +using json = basic_json<>; + +template +struct ordered_map; + +/*! +@brief ordered JSON class + +This type preserves the insertion order of object keys. + +@since version 3.9.0 +*/ +using ordered_json = basic_json; + +} // namespace nlohmann + +#endif // INCLUDE_NLOHMANN_JSON_FWD_HPP_ + + +namespace nlohmann +{ +/*! +@brief detail namespace with internal helper functions + +This namespace collects functions that should not be exposed, +implementations of some @ref basic_json methods, and meta-programming helpers. + +@since version 2.1.0 +*/ +namespace detail +{ +///////////// +// helpers // +///////////// + +// Note to maintainers: +// +// Every trait in this file expects a non CV-qualified type. +// The only exceptions are in the 'aliases for detected' section +// (i.e. those of the form: decltype(T::member_function(std::declval()))) +// +// In this case, T has to be properly CV-qualified to constraint the function arguments +// (e.g. to_json(BasicJsonType&, const T&)) + +template struct is_basic_json : std::false_type {}; + +NLOHMANN_BASIC_JSON_TPL_DECLARATION +struct is_basic_json : std::true_type {}; + +////////////////////// +// json_ref helpers // +////////////////////// + +template +class json_ref; + +template +struct is_json_ref : std::false_type {}; + +template +struct is_json_ref> : std::true_type {}; + +////////////////////////// +// aliases for detected // +////////////////////////// + +template +using mapped_type_t = typename T::mapped_type; + +template +using key_type_t = typename T::key_type; + +template +using value_type_t = typename T::value_type; + +template +using difference_type_t = typename T::difference_type; + +template +using pointer_t = typename T::pointer; + +template +using reference_t = typename T::reference; + +template +using iterator_category_t = typename T::iterator_category; + +template +using iterator_t = typename T::iterator; + +template +using to_json_function = decltype(T::to_json(std::declval()...)); + +template +using from_json_function = decltype(T::from_json(std::declval()...)); + +template +using get_template_function = decltype(std::declval().template get()); + +// trait checking if JSONSerializer::from_json(json const&, udt&) exists +template +struct has_from_json : std::false_type {}; + +// trait checking if j.get is valid +// use this trait instead of std::is_constructible or std::is_convertible, +// both rely on, or make use of implicit conversions, and thus fail when T +// has several constructors/operator= (see https://github.com/nlohmann/json/issues/958) +template +struct is_getable +{ + static constexpr bool value = is_detected::value; +}; + +template +struct has_from_json < BasicJsonType, T, + enable_if_t < !is_basic_json::value >> +{ + using serializer = typename BasicJsonType::template json_serializer; + + static constexpr bool value = + is_detected_exact::value; +}; + +// This trait checks if JSONSerializer::from_json(json const&) exists +// this overload is used for non-default-constructible user-defined-types +template +struct has_non_default_from_json : std::false_type {}; + +template +struct has_non_default_from_json < BasicJsonType, T, enable_if_t < !is_basic_json::value >> +{ + using serializer = typename BasicJsonType::template json_serializer; + + static constexpr bool value = + is_detected_exact::value; +}; + +// This trait checks if BasicJsonType::json_serializer::to_json exists +// Do not evaluate the trait when T is a basic_json type, to avoid template instantiation infinite recursion. +template +struct has_to_json : std::false_type {}; + +template +struct has_to_json < BasicJsonType, T, enable_if_t < !is_basic_json::value >> +{ + using serializer = typename BasicJsonType::template json_serializer; + + static constexpr bool value = + is_detected_exact::value; +}; + + +/////////////////// +// is_ functions // +/////////////////// + +template +struct is_iterator_traits : std::false_type {}; + +template +struct is_iterator_traits> +{ + private: + using traits = iterator_traits; + + public: + static constexpr auto value = + is_detected::value && + is_detected::value && + is_detected::value && + is_detected::value && + is_detected::value; +}; + +// source: https://stackoverflow.com/a/37193089/4116453 + +template +struct is_complete_type : std::false_type {}; + +template +struct is_complete_type : std::true_type {}; + +template +struct is_compatible_object_type_impl : std::false_type {}; + +template +struct is_compatible_object_type_impl < + BasicJsonType, CompatibleObjectType, + enable_if_t < is_detected::value&& + is_detected::value >> +{ + + using object_t = typename BasicJsonType::object_t; + + // macOS's is_constructible does not play well with nonesuch... + static constexpr bool value = + std::is_constructible::value && + std::is_constructible::value; +}; + +template +struct is_compatible_object_type + : is_compatible_object_type_impl {}; + +template +struct is_constructible_object_type_impl : std::false_type {}; + +template +struct is_constructible_object_type_impl < + BasicJsonType, ConstructibleObjectType, + enable_if_t < is_detected::value&& + is_detected::value >> +{ + using object_t = typename BasicJsonType::object_t; + + static constexpr bool value = + (std::is_default_constructible::value && + (std::is_move_assignable::value || + std::is_copy_assignable::value) && + (std::is_constructible::value && + std::is_same < + typename object_t::mapped_type, + typename ConstructibleObjectType::mapped_type >::value)) || + (has_from_json::value || + has_non_default_from_json < + BasicJsonType, + typename ConstructibleObjectType::mapped_type >::value); +}; + +template +struct is_constructible_object_type + : is_constructible_object_type_impl {}; + +template +struct is_compatible_string_type_impl : std::false_type {}; + +template +struct is_compatible_string_type_impl < + BasicJsonType, CompatibleStringType, + enable_if_t::value >> +{ + static constexpr auto value = + std::is_constructible::value; +}; + +template +struct is_compatible_string_type + : is_compatible_string_type_impl {}; + +template +struct is_constructible_string_type_impl : std::false_type {}; + +template +struct is_constructible_string_type_impl < + BasicJsonType, ConstructibleStringType, + enable_if_t::value >> +{ + static constexpr auto value = + std::is_constructible::value; +}; + +template +struct is_constructible_string_type + : is_constructible_string_type_impl {}; + +template +struct is_compatible_array_type_impl : std::false_type {}; + +template +struct is_compatible_array_type_impl < + BasicJsonType, CompatibleArrayType, + enable_if_t < is_detected::value&& + is_detected::value&& +// This is needed because json_reverse_iterator has a ::iterator type... +// Therefore it is detected as a CompatibleArrayType. +// The real fix would be to have an Iterable concept. + !is_iterator_traits < + iterator_traits>::value >> +{ + static constexpr bool value = + std::is_constructible::value; +}; + +template +struct is_compatible_array_type + : is_compatible_array_type_impl {}; + +template +struct is_constructible_array_type_impl : std::false_type {}; + +template +struct is_constructible_array_type_impl < + BasicJsonType, ConstructibleArrayType, + enable_if_t::value >> + : std::true_type {}; + +template +struct is_constructible_array_type_impl < + BasicJsonType, ConstructibleArrayType, + enable_if_t < !std::is_same::value&& + std::is_default_constructible::value&& +(std::is_move_assignable::value || + std::is_copy_assignable::value)&& +is_detected::value&& +is_detected::value&& +is_complete_type < +detected_t>::value >> +{ + static constexpr bool value = + // This is needed because json_reverse_iterator has a ::iterator type, + // furthermore, std::back_insert_iterator (and other iterators) have a + // base class `iterator`... Therefore it is detected as a + // ConstructibleArrayType. The real fix would be to have an Iterable + // concept. + !is_iterator_traits>::value && + + (std::is_same::value || + has_from_json::value || + has_non_default_from_json < + BasicJsonType, typename ConstructibleArrayType::value_type >::value); +}; + +template +struct is_constructible_array_type + : is_constructible_array_type_impl {}; + +template +struct is_compatible_integer_type_impl : std::false_type {}; + +template +struct is_compatible_integer_type_impl < + RealIntegerType, CompatibleNumberIntegerType, + enable_if_t < std::is_integral::value&& + std::is_integral::value&& + !std::is_same::value >> +{ + // is there an assert somewhere on overflows? + using RealLimits = std::numeric_limits; + using CompatibleLimits = std::numeric_limits; + + static constexpr auto value = + std::is_constructible::value && + CompatibleLimits::is_integer && + RealLimits::is_signed == CompatibleLimits::is_signed; +}; + +template +struct is_compatible_integer_type + : is_compatible_integer_type_impl {}; + +template +struct is_compatible_type_impl: std::false_type {}; + +template +struct is_compatible_type_impl < + BasicJsonType, CompatibleType, + enable_if_t::value >> +{ + static constexpr bool value = + has_to_json::value; +}; + +template +struct is_compatible_type + : is_compatible_type_impl {}; + +// https://en.cppreference.com/w/cpp/types/conjunction +template struct conjunction : std::true_type { }; +template struct conjunction : B1 { }; +template +struct conjunction +: std::conditional, B1>::type {}; + +template +struct is_constructible_tuple : std::false_type {}; + +template +struct is_constructible_tuple> : conjunction...> {}; +} // namespace detail +} // namespace nlohmann + +// #include + + +#include // array +#include // size_t +#include // uint8_t +#include // string + +namespace nlohmann +{ +namespace detail +{ +/////////////////////////// +// JSON type enumeration // +/////////////////////////// + +/*! +@brief the JSON type enumeration + +This enumeration collects the different JSON types. It is internally used to +distinguish the stored values, and the functions @ref basic_json::is_null(), +@ref basic_json::is_object(), @ref basic_json::is_array(), +@ref basic_json::is_string(), @ref basic_json::is_boolean(), +@ref basic_json::is_number() (with @ref basic_json::is_number_integer(), +@ref basic_json::is_number_unsigned(), and @ref basic_json::is_number_float()), +@ref basic_json::is_discarded(), @ref basic_json::is_primitive(), and +@ref basic_json::is_structured() rely on it. + +@note There are three enumeration entries (number_integer, number_unsigned, and +number_float), because the library distinguishes these three types for numbers: +@ref basic_json::number_unsigned_t is used for unsigned integers, +@ref basic_json::number_integer_t is used for signed integers, and +@ref basic_json::number_float_t is used for floating-point numbers or to +approximate integers which do not fit in the limits of their respective type. + +@sa @ref basic_json::basic_json(const value_t value_type) -- create a JSON +value with the default value for a given type + +@since version 1.0.0 +*/ +enum class value_t : std::uint8_t +{ + null, ///< null value + object, ///< object (unordered set of name/value pairs) + array, ///< array (ordered collection of values) + string, ///< string value + boolean, ///< boolean value + number_integer, ///< number value (signed integer) + number_unsigned, ///< number value (unsigned integer) + number_float, ///< number value (floating-point) + binary, ///< binary array (ordered collection of bytes) + discarded ///< discarded by the parser callback function +}; + +/*! +@brief comparison operator for JSON types + +Returns an ordering that is similar to Python: +- order: null < boolean < number < object < array < string < binary +- furthermore, each type is not smaller than itself +- discarded values are not comparable +- binary is represented as a b"" string in python and directly comparable to a + string; however, making a binary array directly comparable with a string would + be surprising behavior in a JSON file. + +@since version 1.0.0 +*/ +inline bool operator<(const value_t lhs, const value_t rhs) noexcept +{ + static constexpr std::array order = {{ + 0 /* null */, 3 /* object */, 4 /* array */, 5 /* string */, + 1 /* boolean */, 2 /* integer */, 2 /* unsigned */, 2 /* float */, + 6 /* binary */ + } + }; + + const auto l_index = static_cast(lhs); + const auto r_index = static_cast(rhs); + return l_index < order.size() && r_index < order.size() && order[l_index] < order[r_index]; +} +} // namespace detail +} // namespace nlohmann + + +namespace nlohmann +{ +namespace detail +{ +template +void from_json(const BasicJsonType& j, typename std::nullptr_t& n) +{ + if (JSON_HEDLEY_UNLIKELY(!j.is_null())) + { + JSON_THROW(type_error::create(302, "type must be null, but is " + std::string(j.type_name()))); + } + n = nullptr; +} + +// overloads for basic_json template parameters +template < typename BasicJsonType, typename ArithmeticType, + enable_if_t < std::is_arithmetic::value&& + !std::is_same::value, + int > = 0 > +void get_arithmetic_value(const BasicJsonType& j, ArithmeticType& val) +{ + switch (static_cast(j)) + { + case value_t::number_unsigned: + { + val = static_cast(*j.template get_ptr()); + break; + } + case value_t::number_integer: + { + val = static_cast(*j.template get_ptr()); + break; + } + case value_t::number_float: + { + val = static_cast(*j.template get_ptr()); + break; + } + + default: + JSON_THROW(type_error::create(302, "type must be number, but is " + std::string(j.type_name()))); + } +} + +template +void from_json(const BasicJsonType& j, typename BasicJsonType::boolean_t& b) +{ + if (JSON_HEDLEY_UNLIKELY(!j.is_boolean())) + { + JSON_THROW(type_error::create(302, "type must be boolean, but is " + std::string(j.type_name()))); + } + b = *j.template get_ptr(); +} + +template +void from_json(const BasicJsonType& j, typename BasicJsonType::string_t& s) +{ + if (JSON_HEDLEY_UNLIKELY(!j.is_string())) + { + JSON_THROW(type_error::create(302, "type must be string, but is " + std::string(j.type_name()))); + } + s = *j.template get_ptr(); +} + +template < + typename BasicJsonType, typename ConstructibleStringType, + enable_if_t < + is_constructible_string_type::value&& + !std::is_same::value, + int > = 0 > +void from_json(const BasicJsonType& j, ConstructibleStringType& s) +{ + if (JSON_HEDLEY_UNLIKELY(!j.is_string())) + { + JSON_THROW(type_error::create(302, "type must be string, but is " + std::string(j.type_name()))); + } + + s = *j.template get_ptr(); +} + +template +void from_json(const BasicJsonType& j, typename BasicJsonType::number_float_t& val) +{ + get_arithmetic_value(j, val); +} + +template +void from_json(const BasicJsonType& j, typename BasicJsonType::number_unsigned_t& val) +{ + get_arithmetic_value(j, val); +} + +template +void from_json(const BasicJsonType& j, typename BasicJsonType::number_integer_t& val) +{ + get_arithmetic_value(j, val); +} + +template::value, int> = 0> +void from_json(const BasicJsonType& j, EnumType& e) +{ + typename std::underlying_type::type val; + get_arithmetic_value(j, val); + e = static_cast(val); +} + +// forward_list doesn't have an insert method +template::value, int> = 0> +void from_json(const BasicJsonType& j, std::forward_list& l) +{ + if (JSON_HEDLEY_UNLIKELY(!j.is_array())) + { + JSON_THROW(type_error::create(302, "type must be array, but is " + std::string(j.type_name()))); + } + l.clear(); + std::transform(j.rbegin(), j.rend(), + std::front_inserter(l), [](const BasicJsonType & i) + { + return i.template get(); + }); +} + +// valarray doesn't have an insert method +template::value, int> = 0> +void from_json(const BasicJsonType& j, std::valarray& l) +{ + if (JSON_HEDLEY_UNLIKELY(!j.is_array())) + { + JSON_THROW(type_error::create(302, "type must be array, but is " + std::string(j.type_name()))); + } + l.resize(j.size()); + std::transform(j.begin(), j.end(), std::begin(l), + [](const BasicJsonType & elem) + { + return elem.template get(); + }); +} + +template +auto from_json(const BasicJsonType& j, T (&arr)[N]) +-> decltype(j.template get(), void()) +{ + for (std::size_t i = 0; i < N; ++i) + { + arr[i] = j.at(i).template get(); + } +} + +template +void from_json_array_impl(const BasicJsonType& j, typename BasicJsonType::array_t& arr, priority_tag<3> /*unused*/) +{ + arr = *j.template get_ptr(); +} + +template +auto from_json_array_impl(const BasicJsonType& j, std::array& arr, + priority_tag<2> /*unused*/) +-> decltype(j.template get(), void()) +{ + for (std::size_t i = 0; i < N; ++i) + { + arr[i] = j.at(i).template get(); + } +} + +template +auto from_json_array_impl(const BasicJsonType& j, ConstructibleArrayType& arr, priority_tag<1> /*unused*/) +-> decltype( + arr.reserve(std::declval()), + j.template get(), + void()) +{ + using std::end; + + ConstructibleArrayType ret; + ret.reserve(j.size()); + std::transform(j.begin(), j.end(), + std::inserter(ret, end(ret)), [](const BasicJsonType & i) + { + // get() returns *this, this won't call a from_json + // method when value_type is BasicJsonType + return i.template get(); + }); + arr = std::move(ret); +} + +template +void from_json_array_impl(const BasicJsonType& j, ConstructibleArrayType& arr, + priority_tag<0> /*unused*/) +{ + using std::end; + + ConstructibleArrayType ret; + std::transform( + j.begin(), j.end(), std::inserter(ret, end(ret)), + [](const BasicJsonType & i) + { + // get() returns *this, this won't call a from_json + // method when value_type is BasicJsonType + return i.template get(); + }); + arr = std::move(ret); +} + +template < typename BasicJsonType, typename ConstructibleArrayType, + enable_if_t < + is_constructible_array_type::value&& + !is_constructible_object_type::value&& + !is_constructible_string_type::value&& + !std::is_same::value&& + !is_basic_json::value, + int > = 0 > +auto from_json(const BasicJsonType& j, ConstructibleArrayType& arr) +-> decltype(from_json_array_impl(j, arr, priority_tag<3> {}), +j.template get(), +void()) +{ + if (JSON_HEDLEY_UNLIKELY(!j.is_array())) + { + JSON_THROW(type_error::create(302, "type must be array, but is " + + std::string(j.type_name()))); + } + + from_json_array_impl(j, arr, priority_tag<3> {}); +} + +template +void from_json(const BasicJsonType& j, typename BasicJsonType::binary_t& bin) +{ + if (JSON_HEDLEY_UNLIKELY(!j.is_binary())) + { + JSON_THROW(type_error::create(302, "type must be binary, but is " + std::string(j.type_name()))); + } + + bin = *j.template get_ptr(); +} + +template::value, int> = 0> +void from_json(const BasicJsonType& j, ConstructibleObjectType& obj) +{ + if (JSON_HEDLEY_UNLIKELY(!j.is_object())) + { + JSON_THROW(type_error::create(302, "type must be object, but is " + std::string(j.type_name()))); + } + + ConstructibleObjectType ret; + auto inner_object = j.template get_ptr(); + using value_type = typename ConstructibleObjectType::value_type; + std::transform( + inner_object->begin(), inner_object->end(), + std::inserter(ret, ret.begin()), + [](typename BasicJsonType::object_t::value_type const & p) + { + return value_type(p.first, p.second.template get()); + }); + obj = std::move(ret); +} + +// overload for arithmetic types, not chosen for basic_json template arguments +// (BooleanType, etc..); note: Is it really necessary to provide explicit +// overloads for boolean_t etc. in case of a custom BooleanType which is not +// an arithmetic type? +template < typename BasicJsonType, typename ArithmeticType, + enable_if_t < + std::is_arithmetic::value&& + !std::is_same::value&& + !std::is_same::value&& + !std::is_same::value&& + !std::is_same::value, + int > = 0 > +void from_json(const BasicJsonType& j, ArithmeticType& val) +{ + switch (static_cast(j)) + { + case value_t::number_unsigned: + { + val = static_cast(*j.template get_ptr()); + break; + } + case value_t::number_integer: + { + val = static_cast(*j.template get_ptr()); + break; + } + case value_t::number_float: + { + val = static_cast(*j.template get_ptr()); + break; + } + case value_t::boolean: + { + val = static_cast(*j.template get_ptr()); + break; + } + + default: + JSON_THROW(type_error::create(302, "type must be number, but is " + std::string(j.type_name()))); + } +} + +template +void from_json(const BasicJsonType& j, std::pair& p) +{ + p = {j.at(0).template get(), j.at(1).template get()}; +} + +template +void from_json_tuple_impl(const BasicJsonType& j, Tuple& t, index_sequence /*unused*/) +{ + t = std::make_tuple(j.at(Idx).template get::type>()...); +} + +template +void from_json(const BasicJsonType& j, std::tuple& t) +{ + from_json_tuple_impl(j, t, index_sequence_for {}); +} + +template < typename BasicJsonType, typename Key, typename Value, typename Compare, typename Allocator, + typename = enable_if_t < !std::is_constructible < + typename BasicJsonType::string_t, Key >::value >> +void from_json(const BasicJsonType& j, std::map& m) +{ + if (JSON_HEDLEY_UNLIKELY(!j.is_array())) + { + JSON_THROW(type_error::create(302, "type must be array, but is " + std::string(j.type_name()))); + } + m.clear(); + for (const auto& p : j) + { + if (JSON_HEDLEY_UNLIKELY(!p.is_array())) + { + JSON_THROW(type_error::create(302, "type must be array, but is " + std::string(p.type_name()))); + } + m.emplace(p.at(0).template get(), p.at(1).template get()); + } +} + +template < typename BasicJsonType, typename Key, typename Value, typename Hash, typename KeyEqual, typename Allocator, + typename = enable_if_t < !std::is_constructible < + typename BasicJsonType::string_t, Key >::value >> +void from_json(const BasicJsonType& j, std::unordered_map& m) +{ + if (JSON_HEDLEY_UNLIKELY(!j.is_array())) + { + JSON_THROW(type_error::create(302, "type must be array, but is " + std::string(j.type_name()))); + } + m.clear(); + for (const auto& p : j) + { + if (JSON_HEDLEY_UNLIKELY(!p.is_array())) + { + JSON_THROW(type_error::create(302, "type must be array, but is " + std::string(p.type_name()))); + } + m.emplace(p.at(0).template get(), p.at(1).template get()); + } +} + +struct from_json_fn +{ + template + auto operator()(const BasicJsonType& j, T& val) const + noexcept(noexcept(from_json(j, val))) + -> decltype(from_json(j, val), void()) + { + return from_json(j, val); + } +}; +} // namespace detail + +/// namespace to hold default `from_json` function +/// to see why this is required: +/// http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n4381.html +namespace +{ +constexpr const auto& from_json = detail::static_const::value; +} // namespace +} // namespace nlohmann + +// #include + + +#include // copy +#include // begin, end +#include // string +#include // tuple, get +#include // is_same, is_constructible, is_floating_point, is_enum, underlying_type +#include // move, forward, declval, pair +#include // valarray +#include // vector + +// #include + + +#include // size_t +#include // input_iterator_tag +#include // string, to_string +#include // tuple_size, get, tuple_element + +// #include + +// #include + + +namespace nlohmann +{ +namespace detail +{ +template +void int_to_string( string_type& target, std::size_t value ) +{ + // For ADL + using std::to_string; + target = to_string(value); +} +template class iteration_proxy_value +{ + public: + using difference_type = std::ptrdiff_t; + using value_type = iteration_proxy_value; + using pointer = value_type * ; + using reference = value_type & ; + using iterator_category = std::input_iterator_tag; + using string_type = typename std::remove_cv< typename std::remove_reference().key() ) >::type >::type; + + private: + /// the iterator + IteratorType anchor; + /// an index for arrays (used to create key names) + std::size_t array_index = 0; + /// last stringified array index + mutable std::size_t array_index_last = 0; + /// a string representation of the array index + mutable string_type array_index_str = "0"; + /// an empty string (to return a reference for primitive values) + const string_type empty_str = ""; + + public: + explicit iteration_proxy_value(IteratorType it) noexcept : anchor(it) {} + + /// dereference operator (needed for range-based for) + iteration_proxy_value& operator*() + { + return *this; + } + + /// increment operator (needed for range-based for) + iteration_proxy_value& operator++() + { + ++anchor; + ++array_index; + + return *this; + } + + /// equality operator (needed for InputIterator) + bool operator==(const iteration_proxy_value& o) const + { + return anchor == o.anchor; + } + + /// inequality operator (needed for range-based for) + bool operator!=(const iteration_proxy_value& o) const + { + return anchor != o.anchor; + } + + /// return key of the iterator + const string_type& key() const + { + JSON_ASSERT(anchor.m_object != nullptr); + + switch (anchor.m_object->type()) + { + // use integer array index as key + case value_t::array: + { + if (array_index != array_index_last) + { + int_to_string( array_index_str, array_index ); + array_index_last = array_index; + } + return array_index_str; + } + + // use key from the object + case value_t::object: + return anchor.key(); + + // use an empty key for all primitive types + default: + return empty_str; + } + } + + /// return value of the iterator + typename IteratorType::reference value() const + { + return anchor.value(); + } +}; + +/// proxy class for the items() function +template class iteration_proxy +{ + private: + /// the container to iterate + typename IteratorType::reference container; + + public: + /// construct iteration proxy from a container + explicit iteration_proxy(typename IteratorType::reference cont) noexcept + : container(cont) {} + + /// return iterator begin (needed for range-based for) + iteration_proxy_value begin() noexcept + { + return iteration_proxy_value(container.begin()); + } + + /// return iterator end (needed for range-based for) + iteration_proxy_value end() noexcept + { + return iteration_proxy_value(container.end()); + } +}; +// Structured Bindings Support +// For further reference see https://blog.tartanllama.xyz/structured-bindings/ +// And see https://github.com/nlohmann/json/pull/1391 +template = 0> +auto get(const nlohmann::detail::iteration_proxy_value& i) -> decltype(i.key()) +{ + return i.key(); +} +// Structured Bindings Support +// For further reference see https://blog.tartanllama.xyz/structured-bindings/ +// And see https://github.com/nlohmann/json/pull/1391 +template = 0> +auto get(const nlohmann::detail::iteration_proxy_value& i) -> decltype(i.value()) +{ + return i.value(); +} +} // namespace detail +} // namespace nlohmann + +// The Addition to the STD Namespace is required to add +// Structured Bindings Support to the iteration_proxy_value class +// For further reference see https://blog.tartanllama.xyz/structured-bindings/ +// And see https://github.com/nlohmann/json/pull/1391 +namespace std +{ +#if defined(__clang__) + // Fix: https://github.com/nlohmann/json/issues/1401 + #pragma clang diagnostic push + #pragma clang diagnostic ignored "-Wmismatched-tags" +#endif +template +class tuple_size<::nlohmann::detail::iteration_proxy_value> + : public std::integral_constant {}; + +template +class tuple_element> +{ + public: + using type = decltype( + get(std::declval < + ::nlohmann::detail::iteration_proxy_value> ())); +}; +#if defined(__clang__) + #pragma clang diagnostic pop +#endif +} // namespace std + +// #include + +// #include + +// #include + + +namespace nlohmann +{ +namespace detail +{ +////////////////// +// constructors // +////////////////// + +template struct external_constructor; + +template<> +struct external_constructor +{ + template + static void construct(BasicJsonType& j, typename BasicJsonType::boolean_t b) noexcept + { + j.m_type = value_t::boolean; + j.m_value = b; + j.assert_invariant(); + } +}; + +template<> +struct external_constructor +{ + template + static void construct(BasicJsonType& j, const typename BasicJsonType::string_t& s) + { + j.m_type = value_t::string; + j.m_value = s; + j.assert_invariant(); + } + + template + static void construct(BasicJsonType& j, typename BasicJsonType::string_t&& s) + { + j.m_type = value_t::string; + j.m_value = std::move(s); + j.assert_invariant(); + } + + template < typename BasicJsonType, typename CompatibleStringType, + enable_if_t < !std::is_same::value, + int > = 0 > + static void construct(BasicJsonType& j, const CompatibleStringType& str) + { + j.m_type = value_t::string; + j.m_value.string = j.template create(str); + j.assert_invariant(); + } +}; + +template<> +struct external_constructor +{ + template + static void construct(BasicJsonType& j, const typename BasicJsonType::binary_t& b) + { + j.m_type = value_t::binary; + typename BasicJsonType::binary_t value{b}; + j.m_value = value; + j.assert_invariant(); + } + + template + static void construct(BasicJsonType& j, typename BasicJsonType::binary_t&& b) + { + j.m_type = value_t::binary; + typename BasicJsonType::binary_t value{std::move(b)}; + j.m_value = value; + j.assert_invariant(); + } +}; + +template<> +struct external_constructor +{ + template + static void construct(BasicJsonType& j, typename BasicJsonType::number_float_t val) noexcept + { + j.m_type = value_t::number_float; + j.m_value = val; + j.assert_invariant(); + } +}; + +template<> +struct external_constructor +{ + template + static void construct(BasicJsonType& j, typename BasicJsonType::number_unsigned_t val) noexcept + { + j.m_type = value_t::number_unsigned; + j.m_value = val; + j.assert_invariant(); + } +}; + +template<> +struct external_constructor +{ + template + static void construct(BasicJsonType& j, typename BasicJsonType::number_integer_t val) noexcept + { + j.m_type = value_t::number_integer; + j.m_value = val; + j.assert_invariant(); + } +}; + +template<> +struct external_constructor +{ + template + static void construct(BasicJsonType& j, const typename BasicJsonType::array_t& arr) + { + j.m_type = value_t::array; + j.m_value = arr; + j.assert_invariant(); + } + + template + static void construct(BasicJsonType& j, typename BasicJsonType::array_t&& arr) + { + j.m_type = value_t::array; + j.m_value = std::move(arr); + j.assert_invariant(); + } + + template < typename BasicJsonType, typename CompatibleArrayType, + enable_if_t < !std::is_same::value, + int > = 0 > + static void construct(BasicJsonType& j, const CompatibleArrayType& arr) + { + using std::begin; + using std::end; + j.m_type = value_t::array; + j.m_value.array = j.template create(begin(arr), end(arr)); + j.assert_invariant(); + } + + template + static void construct(BasicJsonType& j, const std::vector& arr) + { + j.m_type = value_t::array; + j.m_value = value_t::array; + j.m_value.array->reserve(arr.size()); + for (const bool x : arr) + { + j.m_value.array->push_back(x); + } + j.assert_invariant(); + } + + template::value, int> = 0> + static void construct(BasicJsonType& j, const std::valarray& arr) + { + j.m_type = value_t::array; + j.m_value = value_t::array; + j.m_value.array->resize(arr.size()); + if (arr.size() > 0) + { + std::copy(std::begin(arr), std::end(arr), j.m_value.array->begin()); + } + j.assert_invariant(); + } +}; + +template<> +struct external_constructor +{ + template + static void construct(BasicJsonType& j, const typename BasicJsonType::object_t& obj) + { + j.m_type = value_t::object; + j.m_value = obj; + j.assert_invariant(); + } + + template + static void construct(BasicJsonType& j, typename BasicJsonType::object_t&& obj) + { + j.m_type = value_t::object; + j.m_value = std::move(obj); + j.assert_invariant(); + } + + template < typename BasicJsonType, typename CompatibleObjectType, + enable_if_t < !std::is_same::value, int > = 0 > + static void construct(BasicJsonType& j, const CompatibleObjectType& obj) + { + using std::begin; + using std::end; + + j.m_type = value_t::object; + j.m_value.object = j.template create(begin(obj), end(obj)); + j.assert_invariant(); + } +}; + +///////////// +// to_json // +///////////// + +template::value, int> = 0> +void to_json(BasicJsonType& j, T b) noexcept +{ + external_constructor::construct(j, b); +} + +template::value, int> = 0> +void to_json(BasicJsonType& j, const CompatibleString& s) +{ + external_constructor::construct(j, s); +} + +template +void to_json(BasicJsonType& j, typename BasicJsonType::string_t&& s) +{ + external_constructor::construct(j, std::move(s)); +} + +template::value, int> = 0> +void to_json(BasicJsonType& j, FloatType val) noexcept +{ + external_constructor::construct(j, static_cast(val)); +} + +template::value, int> = 0> +void to_json(BasicJsonType& j, CompatibleNumberUnsignedType val) noexcept +{ + external_constructor::construct(j, static_cast(val)); +} + +template::value, int> = 0> +void to_json(BasicJsonType& j, CompatibleNumberIntegerType val) noexcept +{ + external_constructor::construct(j, static_cast(val)); +} + +template::value, int> = 0> +void to_json(BasicJsonType& j, EnumType e) noexcept +{ + using underlying_type = typename std::underlying_type::type; + external_constructor::construct(j, static_cast(e)); +} + +template +void to_json(BasicJsonType& j, const std::vector& e) +{ + external_constructor::construct(j, e); +} + +template < typename BasicJsonType, typename CompatibleArrayType, + enable_if_t < is_compatible_array_type::value&& + !is_compatible_object_type::value&& + !is_compatible_string_type::value&& + !std::is_same::value&& + !is_basic_json::value, + int > = 0 > +void to_json(BasicJsonType& j, const CompatibleArrayType& arr) +{ + external_constructor::construct(j, arr); +} + +template +void to_json(BasicJsonType& j, const typename BasicJsonType::binary_t& bin) +{ + external_constructor::construct(j, bin); +} + +template::value, int> = 0> +void to_json(BasicJsonType& j, const std::valarray& arr) +{ + external_constructor::construct(j, std::move(arr)); +} + +template +void to_json(BasicJsonType& j, typename BasicJsonType::array_t&& arr) +{ + external_constructor::construct(j, std::move(arr)); +} + +template < typename BasicJsonType, typename CompatibleObjectType, + enable_if_t < is_compatible_object_type::value&& !is_basic_json::value, int > = 0 > +void to_json(BasicJsonType& j, const CompatibleObjectType& obj) +{ + external_constructor::construct(j, obj); +} + +template +void to_json(BasicJsonType& j, typename BasicJsonType::object_t&& obj) +{ + external_constructor::construct(j, std::move(obj)); +} + +template < + typename BasicJsonType, typename T, std::size_t N, + enable_if_t < !std::is_constructible::value, + int > = 0 > +void to_json(BasicJsonType& j, const T(&arr)[N]) +{ + external_constructor::construct(j, arr); +} + +template < typename BasicJsonType, typename T1, typename T2, enable_if_t < std::is_constructible::value&& std::is_constructible::value, int > = 0 > +void to_json(BasicJsonType& j, const std::pair& p) +{ + j = { p.first, p.second }; +} + +// for https://github.com/nlohmann/json/pull/1134 +template>::value, int> = 0> +void to_json(BasicJsonType& j, const T& b) +{ + j = { {b.key(), b.value()} }; +} + +template +void to_json_tuple_impl(BasicJsonType& j, const Tuple& t, index_sequence /*unused*/) +{ + j = { std::get(t)... }; +} + +template::value, int > = 0> +void to_json(BasicJsonType& j, const T& t) +{ + to_json_tuple_impl(j, t, make_index_sequence::value> {}); +} + +struct to_json_fn +{ + template + auto operator()(BasicJsonType& j, T&& val) const noexcept(noexcept(to_json(j, std::forward(val)))) + -> decltype(to_json(j, std::forward(val)), void()) + { + return to_json(j, std::forward(val)); + } +}; +} // namespace detail + +/// namespace to hold default `to_json` function +namespace +{ +constexpr const auto& to_json = detail::static_const::value; +} // namespace +} // namespace nlohmann + + +namespace nlohmann +{ + +template +struct adl_serializer +{ + /*! + @brief convert a JSON value to any value type + + This function is usually called by the `get()` function of the + @ref basic_json class (either explicit or via conversion operators). + + @param[in] j JSON value to read from + @param[in,out] val value to write to + */ + template + static auto from_json(BasicJsonType&& j, ValueType& val) noexcept( + noexcept(::nlohmann::from_json(std::forward(j), val))) + -> decltype(::nlohmann::from_json(std::forward(j), val), void()) + { + ::nlohmann::from_json(std::forward(j), val); + } + + /*! + @brief convert any value type to a JSON value + + This function is usually called by the constructors of the @ref basic_json + class. + + @param[in,out] j JSON value to write to + @param[in] val value to read from + */ + template + static auto to_json(BasicJsonType& j, ValueType&& val) noexcept( + noexcept(::nlohmann::to_json(j, std::forward(val)))) + -> decltype(::nlohmann::to_json(j, std::forward(val)), void()) + { + ::nlohmann::to_json(j, std::forward(val)); + } +}; + +} // namespace nlohmann + +// #include + + +#include // uint8_t +#include // tie +#include // move + +namespace nlohmann +{ + +/*! +@brief an internal type for a backed binary type + +This type extends the template parameter @a BinaryType provided to `basic_json` +with a subtype used by BSON and MessagePack. This type exists so that the user +does not have to specify a type themselves with a specific naming scheme in +order to override the binary type. + +@tparam BinaryType container to store bytes (`std::vector` by + default) + +@since version 3.8.0 +*/ +template +class byte_container_with_subtype : public BinaryType +{ + public: + /// the type of the underlying container + using container_type = BinaryType; + + byte_container_with_subtype() noexcept(noexcept(container_type())) + : container_type() + {} + + byte_container_with_subtype(const container_type& b) noexcept(noexcept(container_type(b))) + : container_type(b) + {} + + byte_container_with_subtype(container_type&& b) noexcept(noexcept(container_type(std::move(b)))) + : container_type(std::move(b)) + {} + + byte_container_with_subtype(const container_type& b, std::uint8_t subtype) noexcept(noexcept(container_type(b))) + : container_type(b) + , m_subtype(subtype) + , m_has_subtype(true) + {} + + byte_container_with_subtype(container_type&& b, std::uint8_t subtype) noexcept(noexcept(container_type(std::move(b)))) + : container_type(std::move(b)) + , m_subtype(subtype) + , m_has_subtype(true) + {} + + bool operator==(const byte_container_with_subtype& rhs) const + { + return std::tie(static_cast(*this), m_subtype, m_has_subtype) == + std::tie(static_cast(rhs), rhs.m_subtype, rhs.m_has_subtype); + } + + bool operator!=(const byte_container_with_subtype& rhs) const + { + return !(rhs == *this); + } + + /*! + @brief sets the binary subtype + + Sets the binary subtype of the value, also flags a binary JSON value as + having a subtype, which has implications for serialization. + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this member function never throws + exceptions. + + @sa @ref subtype() -- return the binary subtype + @sa @ref clear_subtype() -- clears the binary subtype + @sa @ref has_subtype() -- returns whether or not the binary value has a + subtype + + @since version 3.8.0 + */ + void set_subtype(std::uint8_t subtype) noexcept + { + m_subtype = subtype; + m_has_subtype = true; + } + + /*! + @brief return the binary subtype + + Returns the numerical subtype of the value if it has a subtype. If it does + not have a subtype, this function will return size_t(-1) as a sentinel + value. + + @return the numerical subtype of the binary value + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this member function never throws + exceptions. + + @sa @ref set_subtype() -- sets the binary subtype + @sa @ref clear_subtype() -- clears the binary subtype + @sa @ref has_subtype() -- returns whether or not the binary value has a + subtype + + @since version 3.8.0 + */ + constexpr std::uint8_t subtype() const noexcept + { + return m_subtype; + } + + /*! + @brief return whether the value has a subtype + + @return whether the value has a subtype + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this member function never throws + exceptions. + + @sa @ref subtype() -- return the binary subtype + @sa @ref set_subtype() -- sets the binary subtype + @sa @ref clear_subtype() -- clears the binary subtype + + @since version 3.8.0 + */ + constexpr bool has_subtype() const noexcept + { + return m_has_subtype; + } + + /*! + @brief clears the binary subtype + + Clears the binary subtype and flags the value as not having a subtype, which + has implications for serialization; for instance MessagePack will prefer the + bin family over the ext family. + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this member function never throws + exceptions. + + @sa @ref subtype() -- return the binary subtype + @sa @ref set_subtype() -- sets the binary subtype + @sa @ref has_subtype() -- returns whether or not the binary value has a + subtype + + @since version 3.8.0 + */ + void clear_subtype() noexcept + { + m_subtype = 0; + m_has_subtype = false; + } + + private: + std::uint8_t m_subtype = 0; + bool m_has_subtype = false; +}; + +} // namespace nlohmann + +// #include + +// #include + +// #include + +// #include + + +#include // size_t, uint8_t +#include // hash + +namespace nlohmann +{ +namespace detail +{ + +// boost::hash_combine +inline std::size_t combine(std::size_t seed, std::size_t h) noexcept +{ + seed ^= h + 0x9e3779b9 + (seed << 6U) + (seed >> 2U); + return seed; +} + +/*! +@brief hash a JSON value + +The hash function tries to rely on std::hash where possible. Furthermore, the +type of the JSON value is taken into account to have different hash values for +null, 0, 0U, and false, etc. + +@tparam BasicJsonType basic_json specialization +@param j JSON value to hash +@return hash value of j +*/ +template +std::size_t hash(const BasicJsonType& j) +{ + using string_t = typename BasicJsonType::string_t; + using number_integer_t = typename BasicJsonType::number_integer_t; + using number_unsigned_t = typename BasicJsonType::number_unsigned_t; + using number_float_t = typename BasicJsonType::number_float_t; + + const auto type = static_cast(j.type()); + switch (j.type()) + { + case BasicJsonType::value_t::null: + case BasicJsonType::value_t::discarded: + { + return combine(type, 0); + } + + case BasicJsonType::value_t::object: + { + auto seed = combine(type, j.size()); + for (const auto& element : j.items()) + { + const auto h = std::hash {}(element.key()); + seed = combine(seed, h); + seed = combine(seed, hash(element.value())); + } + return seed; + } + + case BasicJsonType::value_t::array: + { + auto seed = combine(type, j.size()); + for (const auto& element : j) + { + seed = combine(seed, hash(element)); + } + return seed; + } + + case BasicJsonType::value_t::string: + { + const auto h = std::hash {}(j.template get_ref()); + return combine(type, h); + } + + case BasicJsonType::value_t::boolean: + { + const auto h = std::hash {}(j.template get()); + return combine(type, h); + } + + case BasicJsonType::value_t::number_integer: + { + const auto h = std::hash {}(j.template get()); + return combine(type, h); + } + + case nlohmann::detail::value_t::number_unsigned: + { + const auto h = std::hash {}(j.template get()); + return combine(type, h); + } + + case nlohmann::detail::value_t::number_float: + { + const auto h = std::hash {}(j.template get()); + return combine(type, h); + } + + case nlohmann::detail::value_t::binary: + { + auto seed = combine(type, j.get_binary().size()); + const auto h = std::hash {}(j.get_binary().has_subtype()); + seed = combine(seed, h); + seed = combine(seed, j.get_binary().subtype()); + for (const auto byte : j.get_binary()) + { + seed = combine(seed, std::hash {}(byte)); + } + return seed; + } + + default: // LCOV_EXCL_LINE + JSON_ASSERT(false); // LCOV_EXCL_LINE + } +} + +} // namespace detail +} // namespace nlohmann + +// #include + + +#include // generate_n +#include // array +#include // ldexp +#include // size_t +#include // uint8_t, uint16_t, uint32_t, uint64_t +#include // snprintf +#include // memcpy +#include // back_inserter +#include // numeric_limits +#include // char_traits, string +#include // make_pair, move + +// #include + +// #include + + +#include // array +#include // size_t +#include //FILE * +#include // strlen +#include // istream +#include // begin, end, iterator_traits, random_access_iterator_tag, distance, next +#include // shared_ptr, make_shared, addressof +#include // accumulate +#include // string, char_traits +#include // enable_if, is_base_of, is_pointer, is_integral, remove_pointer +#include // pair, declval + +// #include + +// #include + + +namespace nlohmann +{ +namespace detail +{ +/// the supported input formats +enum class input_format_t { json, cbor, msgpack, ubjson, bson }; + +//////////////////// +// input adapters // +//////////////////// + +/*! +Input adapter for stdio file access. This adapter read only 1 byte and do not use any + buffer. This adapter is a very low level adapter. +*/ +class file_input_adapter +{ + public: + using char_type = char; + + JSON_HEDLEY_NON_NULL(2) + explicit file_input_adapter(std::FILE* f) noexcept + : m_file(f) + {} + + // make class move-only + file_input_adapter(const file_input_adapter&) = delete; + file_input_adapter(file_input_adapter&&) = default; + file_input_adapter& operator=(const file_input_adapter&) = delete; + file_input_adapter& operator=(file_input_adapter&&) = delete; + + std::char_traits::int_type get_character() noexcept + { + return std::fgetc(m_file); + } + + private: + /// the file pointer to read from + std::FILE* m_file; +}; + + +/*! +Input adapter for a (caching) istream. Ignores a UFT Byte Order Mark at +beginning of input. Does not support changing the underlying std::streambuf +in mid-input. Maintains underlying std::istream and std::streambuf to support +subsequent use of standard std::istream operations to process any input +characters following those used in parsing the JSON input. Clears the +std::istream flags; any input errors (e.g., EOF) will be detected by the first +subsequent call for input from the std::istream. +*/ +class input_stream_adapter +{ + public: + using char_type = char; + + ~input_stream_adapter() + { + // clear stream flags; we use underlying streambuf I/O, do not + // maintain ifstream flags, except eof + if (is != nullptr) + { + is->clear(is->rdstate() & std::ios::eofbit); + } + } + + explicit input_stream_adapter(std::istream& i) + : is(&i), sb(i.rdbuf()) + {} + + // delete because of pointer members + input_stream_adapter(const input_stream_adapter&) = delete; + input_stream_adapter& operator=(input_stream_adapter&) = delete; + input_stream_adapter& operator=(input_stream_adapter&& rhs) = delete; + + input_stream_adapter(input_stream_adapter&& rhs) noexcept : is(rhs.is), sb(rhs.sb) + { + rhs.is = nullptr; + rhs.sb = nullptr; + } + + // std::istream/std::streambuf use std::char_traits::to_int_type, to + // ensure that std::char_traits::eof() and the character 0xFF do not + // end up as the same value, eg. 0xFFFFFFFF. + std::char_traits::int_type get_character() + { + auto res = sb->sbumpc(); + // set eof manually, as we don't use the istream interface. + if (JSON_HEDLEY_UNLIKELY(res == EOF)) + { + is->clear(is->rdstate() | std::ios::eofbit); + } + return res; + } + + private: + /// the associated input stream + std::istream* is = nullptr; + std::streambuf* sb = nullptr; +}; + +// General-purpose iterator-based adapter. It might not be as fast as +// theoretically possible for some containers, but it is extremely versatile. +template +class iterator_input_adapter +{ + public: + using char_type = typename std::iterator_traits::value_type; + + iterator_input_adapter(IteratorType first, IteratorType last) + : current(std::move(first)), end(std::move(last)) {} + + typename std::char_traits::int_type get_character() + { + if (JSON_HEDLEY_LIKELY(current != end)) + { + auto result = std::char_traits::to_int_type(*current); + std::advance(current, 1); + return result; + } + else + { + return std::char_traits::eof(); + } + } + + private: + IteratorType current; + IteratorType end; + + template + friend struct wide_string_input_helper; + + bool empty() const + { + return current == end; + } + +}; + + +template +struct wide_string_input_helper; + +template +struct wide_string_input_helper +{ + // UTF-32 + static void fill_buffer(BaseInputAdapter& input, + std::array::int_type, 4>& utf8_bytes, + size_t& utf8_bytes_index, + size_t& utf8_bytes_filled) + { + utf8_bytes_index = 0; + + if (JSON_HEDLEY_UNLIKELY(input.empty())) + { + utf8_bytes[0] = std::char_traits::eof(); + utf8_bytes_filled = 1; + } + else + { + // get the current character + const auto wc = input.get_character(); + + // UTF-32 to UTF-8 encoding + if (wc < 0x80) + { + utf8_bytes[0] = static_cast::int_type>(wc); + utf8_bytes_filled = 1; + } + else if (wc <= 0x7FF) + { + utf8_bytes[0] = static_cast::int_type>(0xC0u | ((static_cast(wc) >> 6u) & 0x1Fu)); + utf8_bytes[1] = static_cast::int_type>(0x80u | (static_cast(wc) & 0x3Fu)); + utf8_bytes_filled = 2; + } + else if (wc <= 0xFFFF) + { + utf8_bytes[0] = static_cast::int_type>(0xE0u | ((static_cast(wc) >> 12u) & 0x0Fu)); + utf8_bytes[1] = static_cast::int_type>(0x80u | ((static_cast(wc) >> 6u) & 0x3Fu)); + utf8_bytes[2] = static_cast::int_type>(0x80u | (static_cast(wc) & 0x3Fu)); + utf8_bytes_filled = 3; + } + else if (wc <= 0x10FFFF) + { + utf8_bytes[0] = static_cast::int_type>(0xF0u | ((static_cast(wc) >> 18u) & 0x07u)); + utf8_bytes[1] = static_cast::int_type>(0x80u | ((static_cast(wc) >> 12u) & 0x3Fu)); + utf8_bytes[2] = static_cast::int_type>(0x80u | ((static_cast(wc) >> 6u) & 0x3Fu)); + utf8_bytes[3] = static_cast::int_type>(0x80u | (static_cast(wc) & 0x3Fu)); + utf8_bytes_filled = 4; + } + else + { + // unknown character + utf8_bytes[0] = static_cast::int_type>(wc); + utf8_bytes_filled = 1; + } + } + } +}; + +template +struct wide_string_input_helper +{ + // UTF-16 + static void fill_buffer(BaseInputAdapter& input, + std::array::int_type, 4>& utf8_bytes, + size_t& utf8_bytes_index, + size_t& utf8_bytes_filled) + { + utf8_bytes_index = 0; + + if (JSON_HEDLEY_UNLIKELY(input.empty())) + { + utf8_bytes[0] = std::char_traits::eof(); + utf8_bytes_filled = 1; + } + else + { + // get the current character + const auto wc = input.get_character(); + + // UTF-16 to UTF-8 encoding + if (wc < 0x80) + { + utf8_bytes[0] = static_cast::int_type>(wc); + utf8_bytes_filled = 1; + } + else if (wc <= 0x7FF) + { + utf8_bytes[0] = static_cast::int_type>(0xC0u | ((static_cast(wc) >> 6u))); + utf8_bytes[1] = static_cast::int_type>(0x80u | (static_cast(wc) & 0x3Fu)); + utf8_bytes_filled = 2; + } + else if (0xD800 > wc || wc >= 0xE000) + { + utf8_bytes[0] = static_cast::int_type>(0xE0u | ((static_cast(wc) >> 12u))); + utf8_bytes[1] = static_cast::int_type>(0x80u | ((static_cast(wc) >> 6u) & 0x3Fu)); + utf8_bytes[2] = static_cast::int_type>(0x80u | (static_cast(wc) & 0x3Fu)); + utf8_bytes_filled = 3; + } + else + { + if (JSON_HEDLEY_UNLIKELY(!input.empty())) + { + const auto wc2 = static_cast(input.get_character()); + const auto charcode = 0x10000u + (((static_cast(wc) & 0x3FFu) << 10u) | (wc2 & 0x3FFu)); + utf8_bytes[0] = static_cast::int_type>(0xF0u | (charcode >> 18u)); + utf8_bytes[1] = static_cast::int_type>(0x80u | ((charcode >> 12u) & 0x3Fu)); + utf8_bytes[2] = static_cast::int_type>(0x80u | ((charcode >> 6u) & 0x3Fu)); + utf8_bytes[3] = static_cast::int_type>(0x80u | (charcode & 0x3Fu)); + utf8_bytes_filled = 4; + } + else + { + utf8_bytes[0] = static_cast::int_type>(wc); + utf8_bytes_filled = 1; + } + } + } + } +}; + +// Wraps another input apdater to convert wide character types into individual bytes. +template +class wide_string_input_adapter +{ + public: + using char_type = char; + + wide_string_input_adapter(BaseInputAdapter base) + : base_adapter(base) {} + + typename std::char_traits::int_type get_character() noexcept + { + // check if buffer needs to be filled + if (utf8_bytes_index == utf8_bytes_filled) + { + fill_buffer(); + + JSON_ASSERT(utf8_bytes_filled > 0); + JSON_ASSERT(utf8_bytes_index == 0); + } + + // use buffer + JSON_ASSERT(utf8_bytes_filled > 0); + JSON_ASSERT(utf8_bytes_index < utf8_bytes_filled); + return utf8_bytes[utf8_bytes_index++]; + } + + private: + BaseInputAdapter base_adapter; + + template + void fill_buffer() + { + wide_string_input_helper::fill_buffer(base_adapter, utf8_bytes, utf8_bytes_index, utf8_bytes_filled); + } + + /// a buffer for UTF-8 bytes + std::array::int_type, 4> utf8_bytes = {{0, 0, 0, 0}}; + + /// index to the utf8_codes array for the next valid byte + std::size_t utf8_bytes_index = 0; + /// number of valid bytes in the utf8_codes array + std::size_t utf8_bytes_filled = 0; +}; + + +template +struct iterator_input_adapter_factory +{ + using iterator_type = IteratorType; + using char_type = typename std::iterator_traits::value_type; + using adapter_type = iterator_input_adapter; + + static adapter_type create(IteratorType first, IteratorType last) + { + return adapter_type(std::move(first), std::move(last)); + } +}; + +template +struct is_iterator_of_multibyte +{ + using value_type = typename std::iterator_traits::value_type; + enum + { + value = sizeof(value_type) > 1 + }; +}; + +template +struct iterator_input_adapter_factory::value>> +{ + using iterator_type = IteratorType; + using char_type = typename std::iterator_traits::value_type; + using base_adapter_type = iterator_input_adapter; + using adapter_type = wide_string_input_adapter; + + static adapter_type create(IteratorType first, IteratorType last) + { + return adapter_type(base_adapter_type(std::move(first), std::move(last))); + } +}; + +// General purpose iterator-based input +template +typename iterator_input_adapter_factory::adapter_type input_adapter(IteratorType first, IteratorType last) +{ + using factory_type = iterator_input_adapter_factory; + return factory_type::create(first, last); +} + +// Convenience shorthand from container to iterator +template +auto input_adapter(const ContainerType& container) -> decltype(input_adapter(begin(container), end(container))) +{ + // Enable ADL + using std::begin; + using std::end; + + return input_adapter(begin(container), end(container)); +} + +// Special cases with fast paths +inline file_input_adapter input_adapter(std::FILE* file) +{ + return file_input_adapter(file); +} + +inline input_stream_adapter input_adapter(std::istream& stream) +{ + return input_stream_adapter(stream); +} + +inline input_stream_adapter input_adapter(std::istream&& stream) +{ + return input_stream_adapter(stream); +} + +using contiguous_bytes_input_adapter = decltype(input_adapter(std::declval(), std::declval())); + +// Null-delimited strings, and the like. +template < typename CharT, + typename std::enable_if < + std::is_pointer::value&& + !std::is_array::value&& + std::is_integral::type>::value&& + sizeof(typename std::remove_pointer::type) == 1, + int >::type = 0 > +contiguous_bytes_input_adapter input_adapter(CharT b) +{ + auto length = std::strlen(reinterpret_cast(b)); + const auto* ptr = reinterpret_cast(b); + return input_adapter(ptr, ptr + length); +} + +template +auto input_adapter(T (&array)[N]) -> decltype(input_adapter(array, array + N)) +{ + return input_adapter(array, array + N); +} + +// This class only handles inputs of input_buffer_adapter type. +// It's required so that expressions like {ptr, len} can be implicitely casted +// to the correct adapter. +class span_input_adapter +{ + public: + template < typename CharT, + typename std::enable_if < + std::is_pointer::value&& + std::is_integral::type>::value&& + sizeof(typename std::remove_pointer::type) == 1, + int >::type = 0 > + span_input_adapter(CharT b, std::size_t l) + : ia(reinterpret_cast(b), reinterpret_cast(b) + l) {} + + template::iterator_category, std::random_access_iterator_tag>::value, + int>::type = 0> + span_input_adapter(IteratorType first, IteratorType last) + : ia(input_adapter(first, last)) {} + + contiguous_bytes_input_adapter&& get() + { + return std::move(ia); + } + + private: + contiguous_bytes_input_adapter ia; +}; +} // namespace detail +} // namespace nlohmann + +// #include + + +#include +#include // string +#include // move +#include // vector + +// #include + +// #include + + +namespace nlohmann +{ + +/*! +@brief SAX interface + +This class describes the SAX interface used by @ref nlohmann::json::sax_parse. +Each function is called in different situations while the input is parsed. The +boolean return value informs the parser whether to continue processing the +input. +*/ +template +struct json_sax +{ + using number_integer_t = typename BasicJsonType::number_integer_t; + using number_unsigned_t = typename BasicJsonType::number_unsigned_t; + using number_float_t = typename BasicJsonType::number_float_t; + using string_t = typename BasicJsonType::string_t; + using binary_t = typename BasicJsonType::binary_t; + + /*! + @brief a null value was read + @return whether parsing should proceed + */ + virtual bool null() = 0; + + /*! + @brief a boolean value was read + @param[in] val boolean value + @return whether parsing should proceed + */ + virtual bool boolean(bool val) = 0; + + /*! + @brief an integer number was read + @param[in] val integer value + @return whether parsing should proceed + */ + virtual bool number_integer(number_integer_t val) = 0; + + /*! + @brief an unsigned integer number was read + @param[in] val unsigned integer value + @return whether parsing should proceed + */ + virtual bool number_unsigned(number_unsigned_t val) = 0; + + /*! + @brief an floating-point number was read + @param[in] val floating-point value + @param[in] s raw token value + @return whether parsing should proceed + */ + virtual bool number_float(number_float_t val, const string_t& s) = 0; + + /*! + @brief a string was read + @param[in] val string value + @return whether parsing should proceed + @note It is safe to move the passed string. + */ + virtual bool string(string_t& val) = 0; + + /*! + @brief a binary string was read + @param[in] val binary value + @return whether parsing should proceed + @note It is safe to move the passed binary. + */ + virtual bool binary(binary_t& val) = 0; + + /*! + @brief the beginning of an object was read + @param[in] elements number of object elements or -1 if unknown + @return whether parsing should proceed + @note binary formats may report the number of elements + */ + virtual bool start_object(std::size_t elements) = 0; + + /*! + @brief an object key was read + @param[in] val object key + @return whether parsing should proceed + @note It is safe to move the passed string. + */ + virtual bool key(string_t& val) = 0; + + /*! + @brief the end of an object was read + @return whether parsing should proceed + */ + virtual bool end_object() = 0; + + /*! + @brief the beginning of an array was read + @param[in] elements number of array elements or -1 if unknown + @return whether parsing should proceed + @note binary formats may report the number of elements + */ + virtual bool start_array(std::size_t elements) = 0; + + /*! + @brief the end of an array was read + @return whether parsing should proceed + */ + virtual bool end_array() = 0; + + /*! + @brief a parse error occurred + @param[in] position the position in the input where the error occurs + @param[in] last_token the last read token + @param[in] ex an exception object describing the error + @return whether parsing should proceed (must return false) + */ + virtual bool parse_error(std::size_t position, + const std::string& last_token, + const detail::exception& ex) = 0; + + virtual ~json_sax() = default; +}; + + +namespace detail +{ +/*! +@brief SAX implementation to create a JSON value from SAX events + +This class implements the @ref json_sax interface and processes the SAX events +to create a JSON value which makes it basically a DOM parser. The structure or +hierarchy of the JSON value is managed by the stack `ref_stack` which contains +a pointer to the respective array or object for each recursion depth. + +After successful parsing, the value that is passed by reference to the +constructor contains the parsed value. + +@tparam BasicJsonType the JSON type +*/ +template +class json_sax_dom_parser +{ + public: + using number_integer_t = typename BasicJsonType::number_integer_t; + using number_unsigned_t = typename BasicJsonType::number_unsigned_t; + using number_float_t = typename BasicJsonType::number_float_t; + using string_t = typename BasicJsonType::string_t; + using binary_t = typename BasicJsonType::binary_t; + + /*! + @param[in, out] r reference to a JSON value that is manipulated while + parsing + @param[in] allow_exceptions_ whether parse errors yield exceptions + */ + explicit json_sax_dom_parser(BasicJsonType& r, const bool allow_exceptions_ = true) + : root(r), allow_exceptions(allow_exceptions_) + {} + + // make class move-only + json_sax_dom_parser(const json_sax_dom_parser&) = delete; + json_sax_dom_parser(json_sax_dom_parser&&) = default; + json_sax_dom_parser& operator=(const json_sax_dom_parser&) = delete; + json_sax_dom_parser& operator=(json_sax_dom_parser&&) = default; + ~json_sax_dom_parser() = default; + + bool null() + { + handle_value(nullptr); + return true; + } + + bool boolean(bool val) + { + handle_value(val); + return true; + } + + bool number_integer(number_integer_t val) + { + handle_value(val); + return true; + } + + bool number_unsigned(number_unsigned_t val) + { + handle_value(val); + return true; + } + + bool number_float(number_float_t val, const string_t& /*unused*/) + { + handle_value(val); + return true; + } + + bool string(string_t& val) + { + handle_value(val); + return true; + } + + bool binary(binary_t& val) + { + handle_value(std::move(val)); + return true; + } + + bool start_object(std::size_t len) + { + ref_stack.push_back(handle_value(BasicJsonType::value_t::object)); + + if (JSON_HEDLEY_UNLIKELY(len != std::size_t(-1) && len > ref_stack.back()->max_size())) + { + JSON_THROW(out_of_range::create(408, + "excessive object size: " + std::to_string(len))); + } + + return true; + } + + bool key(string_t& val) + { + // add null at given key and store the reference for later + object_element = &(ref_stack.back()->m_value.object->operator[](val)); + return true; + } + + bool end_object() + { + ref_stack.pop_back(); + return true; + } + + bool start_array(std::size_t len) + { + ref_stack.push_back(handle_value(BasicJsonType::value_t::array)); + + if (JSON_HEDLEY_UNLIKELY(len != std::size_t(-1) && len > ref_stack.back()->max_size())) + { + JSON_THROW(out_of_range::create(408, + "excessive array size: " + std::to_string(len))); + } + + return true; + } + + bool end_array() + { + ref_stack.pop_back(); + return true; + } + + template + bool parse_error(std::size_t /*unused*/, const std::string& /*unused*/, + const Exception& ex) + { + errored = true; + static_cast(ex); + if (allow_exceptions) + { + JSON_THROW(ex); + } + return false; + } + + constexpr bool is_errored() const + { + return errored; + } + + private: + /*! + @invariant If the ref stack is empty, then the passed value will be the new + root. + @invariant If the ref stack contains a value, then it is an array or an + object to which we can add elements + */ + template + JSON_HEDLEY_RETURNS_NON_NULL + BasicJsonType* handle_value(Value&& v) + { + if (ref_stack.empty()) + { + root = BasicJsonType(std::forward(v)); + return &root; + } + + JSON_ASSERT(ref_stack.back()->is_array() || ref_stack.back()->is_object()); + + if (ref_stack.back()->is_array()) + { + ref_stack.back()->m_value.array->emplace_back(std::forward(v)); + return &(ref_stack.back()->m_value.array->back()); + } + + JSON_ASSERT(ref_stack.back()->is_object()); + JSON_ASSERT(object_element); + *object_element = BasicJsonType(std::forward(v)); + return object_element; + } + + /// the parsed JSON value + BasicJsonType& root; + /// stack to model hierarchy of values + std::vector ref_stack {}; + /// helper to hold the reference for the next object element + BasicJsonType* object_element = nullptr; + /// whether a syntax error occurred + bool errored = false; + /// whether to throw exceptions in case of errors + const bool allow_exceptions = true; +}; + +template +class json_sax_dom_callback_parser +{ + public: + using number_integer_t = typename BasicJsonType::number_integer_t; + using number_unsigned_t = typename BasicJsonType::number_unsigned_t; + using number_float_t = typename BasicJsonType::number_float_t; + using string_t = typename BasicJsonType::string_t; + using binary_t = typename BasicJsonType::binary_t; + using parser_callback_t = typename BasicJsonType::parser_callback_t; + using parse_event_t = typename BasicJsonType::parse_event_t; + + json_sax_dom_callback_parser(BasicJsonType& r, + const parser_callback_t cb, + const bool allow_exceptions_ = true) + : root(r), callback(cb), allow_exceptions(allow_exceptions_) + { + keep_stack.push_back(true); + } + + // make class move-only + json_sax_dom_callback_parser(const json_sax_dom_callback_parser&) = delete; + json_sax_dom_callback_parser(json_sax_dom_callback_parser&&) = default; + json_sax_dom_callback_parser& operator=(const json_sax_dom_callback_parser&) = delete; + json_sax_dom_callback_parser& operator=(json_sax_dom_callback_parser&&) = default; + ~json_sax_dom_callback_parser() = default; + + bool null() + { + handle_value(nullptr); + return true; + } + + bool boolean(bool val) + { + handle_value(val); + return true; + } + + bool number_integer(number_integer_t val) + { + handle_value(val); + return true; + } + + bool number_unsigned(number_unsigned_t val) + { + handle_value(val); + return true; + } + + bool number_float(number_float_t val, const string_t& /*unused*/) + { + handle_value(val); + return true; + } + + bool string(string_t& val) + { + handle_value(val); + return true; + } + + bool binary(binary_t& val) + { + handle_value(std::move(val)); + return true; + } + + bool start_object(std::size_t len) + { + // check callback for object start + const bool keep = callback(static_cast(ref_stack.size()), parse_event_t::object_start, discarded); + keep_stack.push_back(keep); + + auto val = handle_value(BasicJsonType::value_t::object, true); + ref_stack.push_back(val.second); + + // check object limit + if (ref_stack.back() && JSON_HEDLEY_UNLIKELY(len != std::size_t(-1) && len > ref_stack.back()->max_size())) + { + JSON_THROW(out_of_range::create(408, "excessive object size: " + std::to_string(len))); + } + + return true; + } + + bool key(string_t& val) + { + BasicJsonType k = BasicJsonType(val); + + // check callback for key + const bool keep = callback(static_cast(ref_stack.size()), parse_event_t::key, k); + key_keep_stack.push_back(keep); + + // add discarded value at given key and store the reference for later + if (keep && ref_stack.back()) + { + object_element = &(ref_stack.back()->m_value.object->operator[](val) = discarded); + } + + return true; + } + + bool end_object() + { + if (ref_stack.back() && !callback(static_cast(ref_stack.size()) - 1, parse_event_t::object_end, *ref_stack.back())) + { + // discard object + *ref_stack.back() = discarded; + } + + JSON_ASSERT(!ref_stack.empty()); + JSON_ASSERT(!keep_stack.empty()); + ref_stack.pop_back(); + keep_stack.pop_back(); + + if (!ref_stack.empty() && ref_stack.back() && ref_stack.back()->is_structured()) + { + // remove discarded value + for (auto it = ref_stack.back()->begin(); it != ref_stack.back()->end(); ++it) + { + if (it->is_discarded()) + { + ref_stack.back()->erase(it); + break; + } + } + } + + return true; + } + + bool start_array(std::size_t len) + { + const bool keep = callback(static_cast(ref_stack.size()), parse_event_t::array_start, discarded); + keep_stack.push_back(keep); + + auto val = handle_value(BasicJsonType::value_t::array, true); + ref_stack.push_back(val.second); + + // check array limit + if (ref_stack.back() && JSON_HEDLEY_UNLIKELY(len != std::size_t(-1) && len > ref_stack.back()->max_size())) + { + JSON_THROW(out_of_range::create(408, "excessive array size: " + std::to_string(len))); + } + + return true; + } + + bool end_array() + { + bool keep = true; + + if (ref_stack.back()) + { + keep = callback(static_cast(ref_stack.size()) - 1, parse_event_t::array_end, *ref_stack.back()); + if (!keep) + { + // discard array + *ref_stack.back() = discarded; + } + } + + JSON_ASSERT(!ref_stack.empty()); + JSON_ASSERT(!keep_stack.empty()); + ref_stack.pop_back(); + keep_stack.pop_back(); + + // remove discarded value + if (!keep && !ref_stack.empty() && ref_stack.back()->is_array()) + { + ref_stack.back()->m_value.array->pop_back(); + } + + return true; + } + + template + bool parse_error(std::size_t /*unused*/, const std::string& /*unused*/, + const Exception& ex) + { + errored = true; + static_cast(ex); + if (allow_exceptions) + { + JSON_THROW(ex); + } + return false; + } + + constexpr bool is_errored() const + { + return errored; + } + + private: + /*! + @param[in] v value to add to the JSON value we build during parsing + @param[in] skip_callback whether we should skip calling the callback + function; this is required after start_array() and + start_object() SAX events, because otherwise we would call the + callback function with an empty array or object, respectively. + + @invariant If the ref stack is empty, then the passed value will be the new + root. + @invariant If the ref stack contains a value, then it is an array or an + object to which we can add elements + + @return pair of boolean (whether value should be kept) and pointer (to the + passed value in the ref_stack hierarchy; nullptr if not kept) + */ + template + std::pair handle_value(Value&& v, const bool skip_callback = false) + { + JSON_ASSERT(!keep_stack.empty()); + + // do not handle this value if we know it would be added to a discarded + // container + if (!keep_stack.back()) + { + return {false, nullptr}; + } + + // create value + auto value = BasicJsonType(std::forward(v)); + + // check callback + const bool keep = skip_callback || callback(static_cast(ref_stack.size()), parse_event_t::value, value); + + // do not handle this value if we just learnt it shall be discarded + if (!keep) + { + return {false, nullptr}; + } + + if (ref_stack.empty()) + { + root = std::move(value); + return {true, &root}; + } + + // skip this value if we already decided to skip the parent + // (https://github.com/nlohmann/json/issues/971#issuecomment-413678360) + if (!ref_stack.back()) + { + return {false, nullptr}; + } + + // we now only expect arrays and objects + JSON_ASSERT(ref_stack.back()->is_array() || ref_stack.back()->is_object()); + + // array + if (ref_stack.back()->is_array()) + { + ref_stack.back()->m_value.array->push_back(std::move(value)); + return {true, &(ref_stack.back()->m_value.array->back())}; + } + + // object + JSON_ASSERT(ref_stack.back()->is_object()); + // check if we should store an element for the current key + JSON_ASSERT(!key_keep_stack.empty()); + const bool store_element = key_keep_stack.back(); + key_keep_stack.pop_back(); + + if (!store_element) + { + return {false, nullptr}; + } + + JSON_ASSERT(object_element); + *object_element = std::move(value); + return {true, object_element}; + } + + /// the parsed JSON value + BasicJsonType& root; + /// stack to model hierarchy of values + std::vector ref_stack {}; + /// stack to manage which values to keep + std::vector keep_stack {}; + /// stack to manage which object keys to keep + std::vector key_keep_stack {}; + /// helper to hold the reference for the next object element + BasicJsonType* object_element = nullptr; + /// whether a syntax error occurred + bool errored = false; + /// callback function + const parser_callback_t callback = nullptr; + /// whether to throw exceptions in case of errors + const bool allow_exceptions = true; + /// a discarded value for the callback + BasicJsonType discarded = BasicJsonType::value_t::discarded; +}; + +template +class json_sax_acceptor +{ + public: + using number_integer_t = typename BasicJsonType::number_integer_t; + using number_unsigned_t = typename BasicJsonType::number_unsigned_t; + using number_float_t = typename BasicJsonType::number_float_t; + using string_t = typename BasicJsonType::string_t; + using binary_t = typename BasicJsonType::binary_t; + + bool null() + { + return true; + } + + bool boolean(bool /*unused*/) + { + return true; + } + + bool number_integer(number_integer_t /*unused*/) + { + return true; + } + + bool number_unsigned(number_unsigned_t /*unused*/) + { + return true; + } + + bool number_float(number_float_t /*unused*/, const string_t& /*unused*/) + { + return true; + } + + bool string(string_t& /*unused*/) + { + return true; + } + + bool binary(binary_t& /*unused*/) + { + return true; + } + + bool start_object(std::size_t /*unused*/ = std::size_t(-1)) + { + return true; + } + + bool key(string_t& /*unused*/) + { + return true; + } + + bool end_object() + { + return true; + } + + bool start_array(std::size_t /*unused*/ = std::size_t(-1)) + { + return true; + } + + bool end_array() + { + return true; + } + + bool parse_error(std::size_t /*unused*/, const std::string& /*unused*/, const detail::exception& /*unused*/) + { + return false; + } +}; +} // namespace detail + +} // namespace nlohmann + +// #include + + +#include // array +#include // localeconv +#include // size_t +#include // snprintf +#include // strtof, strtod, strtold, strtoll, strtoull +#include // initializer_list +#include // char_traits, string +#include // move +#include // vector + +// #include + +// #include + +// #include + + +namespace nlohmann +{ +namespace detail +{ +/////////// +// lexer // +/////////// + +template +class lexer_base +{ + public: + /// token types for the parser + enum class token_type + { + uninitialized, ///< indicating the scanner is uninitialized + literal_true, ///< the `true` literal + literal_false, ///< the `false` literal + literal_null, ///< the `null` literal + value_string, ///< a string -- use get_string() for actual value + value_unsigned, ///< an unsigned integer -- use get_number_unsigned() for actual value + value_integer, ///< a signed integer -- use get_number_integer() for actual value + value_float, ///< an floating point number -- use get_number_float() for actual value + begin_array, ///< the character for array begin `[` + begin_object, ///< the character for object begin `{` + end_array, ///< the character for array end `]` + end_object, ///< the character for object end `}` + name_separator, ///< the name separator `:` + value_separator, ///< the value separator `,` + parse_error, ///< indicating a parse error + end_of_input, ///< indicating the end of the input buffer + literal_or_value ///< a literal or the begin of a value (only for diagnostics) + }; + + /// return name of values of type token_type (only used for errors) + JSON_HEDLEY_RETURNS_NON_NULL + JSON_HEDLEY_CONST + static const char* token_type_name(const token_type t) noexcept + { + switch (t) + { + case token_type::uninitialized: + return ""; + case token_type::literal_true: + return "true literal"; + case token_type::literal_false: + return "false literal"; + case token_type::literal_null: + return "null literal"; + case token_type::value_string: + return "string literal"; + case token_type::value_unsigned: + case token_type::value_integer: + case token_type::value_float: + return "number literal"; + case token_type::begin_array: + return "'['"; + case token_type::begin_object: + return "'{'"; + case token_type::end_array: + return "']'"; + case token_type::end_object: + return "'}'"; + case token_type::name_separator: + return "':'"; + case token_type::value_separator: + return "','"; + case token_type::parse_error: + return ""; + case token_type::end_of_input: + return "end of input"; + case token_type::literal_or_value: + return "'[', '{', or a literal"; + // LCOV_EXCL_START + default: // catch non-enum values + return "unknown token"; + // LCOV_EXCL_STOP + } + } +}; +/*! +@brief lexical analysis + +This class organizes the lexical analysis during JSON deserialization. +*/ +template +class lexer : public lexer_base +{ + using number_integer_t = typename BasicJsonType::number_integer_t; + using number_unsigned_t = typename BasicJsonType::number_unsigned_t; + using number_float_t = typename BasicJsonType::number_float_t; + using string_t = typename BasicJsonType::string_t; + using char_type = typename InputAdapterType::char_type; + using char_int_type = typename std::char_traits::int_type; + + public: + using token_type = typename lexer_base::token_type; + + explicit lexer(InputAdapterType&& adapter, bool ignore_comments_ = false) + : ia(std::move(adapter)) + , ignore_comments(ignore_comments_) + , decimal_point_char(static_cast(get_decimal_point())) + {} + + // delete because of pointer members + lexer(const lexer&) = delete; + lexer(lexer&&) = default; + lexer& operator=(lexer&) = delete; + lexer& operator=(lexer&&) = default; + ~lexer() = default; + + private: + ///////////////////// + // locales + ///////////////////// + + /// return the locale-dependent decimal point + JSON_HEDLEY_PURE + static char get_decimal_point() noexcept + { + const auto* loc = localeconv(); + JSON_ASSERT(loc != nullptr); + return (loc->decimal_point == nullptr) ? '.' : *(loc->decimal_point); + } + + ///////////////////// + // scan functions + ///////////////////// + + /*! + @brief get codepoint from 4 hex characters following `\u` + + For input "\u c1 c2 c3 c4" the codepoint is: + (c1 * 0x1000) + (c2 * 0x0100) + (c3 * 0x0010) + c4 + = (c1 << 12) + (c2 << 8) + (c3 << 4) + (c4 << 0) + + Furthermore, the possible characters '0'..'9', 'A'..'F', and 'a'..'f' + must be converted to the integers 0x0..0x9, 0xA..0xF, 0xA..0xF, resp. The + conversion is done by subtracting the offset (0x30, 0x37, and 0x57) + between the ASCII value of the character and the desired integer value. + + @return codepoint (0x0000..0xFFFF) or -1 in case of an error (e.g. EOF or + non-hex character) + */ + int get_codepoint() + { + // this function only makes sense after reading `\u` + JSON_ASSERT(current == 'u'); + int codepoint = 0; + + const auto factors = { 12u, 8u, 4u, 0u }; + for (const auto factor : factors) + { + get(); + + if (current >= '0' && current <= '9') + { + codepoint += static_cast((static_cast(current) - 0x30u) << factor); + } + else if (current >= 'A' && current <= 'F') + { + codepoint += static_cast((static_cast(current) - 0x37u) << factor); + } + else if (current >= 'a' && current <= 'f') + { + codepoint += static_cast((static_cast(current) - 0x57u) << factor); + } + else + { + return -1; + } + } + + JSON_ASSERT(0x0000 <= codepoint && codepoint <= 0xFFFF); + return codepoint; + } + + /*! + @brief check if the next byte(s) are inside a given range + + Adds the current byte and, for each passed range, reads a new byte and + checks if it is inside the range. If a violation was detected, set up an + error message and return false. Otherwise, return true. + + @param[in] ranges list of integers; interpreted as list of pairs of + inclusive lower and upper bound, respectively + + @pre The passed list @a ranges must have 2, 4, or 6 elements; that is, + 1, 2, or 3 pairs. This precondition is enforced by an assertion. + + @return true if and only if no range violation was detected + */ + bool next_byte_in_range(std::initializer_list ranges) + { + JSON_ASSERT(ranges.size() == 2 || ranges.size() == 4 || ranges.size() == 6); + add(current); + + for (auto range = ranges.begin(); range != ranges.end(); ++range) + { + get(); + if (JSON_HEDLEY_LIKELY(*range <= current && current <= *(++range))) + { + add(current); + } + else + { + error_message = "invalid string: ill-formed UTF-8 byte"; + return false; + } + } + + return true; + } + + /*! + @brief scan a string literal + + This function scans a string according to Sect. 7 of RFC 7159. While + scanning, bytes are escaped and copied into buffer token_buffer. Then the + function returns successfully, token_buffer is *not* null-terminated (as it + may contain \0 bytes), and token_buffer.size() is the number of bytes in the + string. + + @return token_type::value_string if string could be successfully scanned, + token_type::parse_error otherwise + + @note In case of errors, variable error_message contains a textual + description. + */ + token_type scan_string() + { + // reset token_buffer (ignore opening quote) + reset(); + + // we entered the function by reading an open quote + JSON_ASSERT(current == '\"'); + + while (true) + { + // get next character + switch (get()) + { + // end of file while parsing string + case std::char_traits::eof(): + { + error_message = "invalid string: missing closing quote"; + return token_type::parse_error; + } + + // closing quote + case '\"': + { + return token_type::value_string; + } + + // escapes + case '\\': + { + switch (get()) + { + // quotation mark + case '\"': + add('\"'); + break; + // reverse solidus + case '\\': + add('\\'); + break; + // solidus + case '/': + add('/'); + break; + // backspace + case 'b': + add('\b'); + break; + // form feed + case 'f': + add('\f'); + break; + // line feed + case 'n': + add('\n'); + break; + // carriage return + case 'r': + add('\r'); + break; + // tab + case 't': + add('\t'); + break; + + // unicode escapes + case 'u': + { + const int codepoint1 = get_codepoint(); + int codepoint = codepoint1; // start with codepoint1 + + if (JSON_HEDLEY_UNLIKELY(codepoint1 == -1)) + { + error_message = "invalid string: '\\u' must be followed by 4 hex digits"; + return token_type::parse_error; + } + + // check if code point is a high surrogate + if (0xD800 <= codepoint1 && codepoint1 <= 0xDBFF) + { + // expect next \uxxxx entry + if (JSON_HEDLEY_LIKELY(get() == '\\' && get() == 'u')) + { + const int codepoint2 = get_codepoint(); + + if (JSON_HEDLEY_UNLIKELY(codepoint2 == -1)) + { + error_message = "invalid string: '\\u' must be followed by 4 hex digits"; + return token_type::parse_error; + } + + // check if codepoint2 is a low surrogate + if (JSON_HEDLEY_LIKELY(0xDC00 <= codepoint2 && codepoint2 <= 0xDFFF)) + { + // overwrite codepoint + codepoint = static_cast( + // high surrogate occupies the most significant 22 bits + (static_cast(codepoint1) << 10u) + // low surrogate occupies the least significant 15 bits + + static_cast(codepoint2) + // there is still the 0xD800, 0xDC00 and 0x10000 noise + // in the result so we have to subtract with: + // (0xD800 << 10) + DC00 - 0x10000 = 0x35FDC00 + - 0x35FDC00u); + } + else + { + error_message = "invalid string: surrogate U+D800..U+DBFF must be followed by U+DC00..U+DFFF"; + return token_type::parse_error; + } + } + else + { + error_message = "invalid string: surrogate U+D800..U+DBFF must be followed by U+DC00..U+DFFF"; + return token_type::parse_error; + } + } + else + { + if (JSON_HEDLEY_UNLIKELY(0xDC00 <= codepoint1 && codepoint1 <= 0xDFFF)) + { + error_message = "invalid string: surrogate U+DC00..U+DFFF must follow U+D800..U+DBFF"; + return token_type::parse_error; + } + } + + // result of the above calculation yields a proper codepoint + JSON_ASSERT(0x00 <= codepoint && codepoint <= 0x10FFFF); + + // translate codepoint into bytes + if (codepoint < 0x80) + { + // 1-byte characters: 0xxxxxxx (ASCII) + add(static_cast(codepoint)); + } + else if (codepoint <= 0x7FF) + { + // 2-byte characters: 110xxxxx 10xxxxxx + add(static_cast(0xC0u | (static_cast(codepoint) >> 6u))); + add(static_cast(0x80u | (static_cast(codepoint) & 0x3Fu))); + } + else if (codepoint <= 0xFFFF) + { + // 3-byte characters: 1110xxxx 10xxxxxx 10xxxxxx + add(static_cast(0xE0u | (static_cast(codepoint) >> 12u))); + add(static_cast(0x80u | ((static_cast(codepoint) >> 6u) & 0x3Fu))); + add(static_cast(0x80u | (static_cast(codepoint) & 0x3Fu))); + } + else + { + // 4-byte characters: 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx + add(static_cast(0xF0u | (static_cast(codepoint) >> 18u))); + add(static_cast(0x80u | ((static_cast(codepoint) >> 12u) & 0x3Fu))); + add(static_cast(0x80u | ((static_cast(codepoint) >> 6u) & 0x3Fu))); + add(static_cast(0x80u | (static_cast(codepoint) & 0x3Fu))); + } + + break; + } + + // other characters after escape + default: + error_message = "invalid string: forbidden character after backslash"; + return token_type::parse_error; + } + + break; + } + + // invalid control characters + case 0x00: + { + error_message = "invalid string: control character U+0000 (NUL) must be escaped to \\u0000"; + return token_type::parse_error; + } + + case 0x01: + { + error_message = "invalid string: control character U+0001 (SOH) must be escaped to \\u0001"; + return token_type::parse_error; + } + + case 0x02: + { + error_message = "invalid string: control character U+0002 (STX) must be escaped to \\u0002"; + return token_type::parse_error; + } + + case 0x03: + { + error_message = "invalid string: control character U+0003 (ETX) must be escaped to \\u0003"; + return token_type::parse_error; + } + + case 0x04: + { + error_message = "invalid string: control character U+0004 (EOT) must be escaped to \\u0004"; + return token_type::parse_error; + } + + case 0x05: + { + error_message = "invalid string: control character U+0005 (ENQ) must be escaped to \\u0005"; + return token_type::parse_error; + } + + case 0x06: + { + error_message = "invalid string: control character U+0006 (ACK) must be escaped to \\u0006"; + return token_type::parse_error; + } + + case 0x07: + { + error_message = "invalid string: control character U+0007 (BEL) must be escaped to \\u0007"; + return token_type::parse_error; + } + + case 0x08: + { + error_message = "invalid string: control character U+0008 (BS) must be escaped to \\u0008 or \\b"; + return token_type::parse_error; + } + + case 0x09: + { + error_message = "invalid string: control character U+0009 (HT) must be escaped to \\u0009 or \\t"; + return token_type::parse_error; + } + + case 0x0A: + { + error_message = "invalid string: control character U+000A (LF) must be escaped to \\u000A or \\n"; + return token_type::parse_error; + } + + case 0x0B: + { + error_message = "invalid string: control character U+000B (VT) must be escaped to \\u000B"; + return token_type::parse_error; + } + + case 0x0C: + { + error_message = "invalid string: control character U+000C (FF) must be escaped to \\u000C or \\f"; + return token_type::parse_error; + } + + case 0x0D: + { + error_message = "invalid string: control character U+000D (CR) must be escaped to \\u000D or \\r"; + return token_type::parse_error; + } + + case 0x0E: + { + error_message = "invalid string: control character U+000E (SO) must be escaped to \\u000E"; + return token_type::parse_error; + } + + case 0x0F: + { + error_message = "invalid string: control character U+000F (SI) must be escaped to \\u000F"; + return token_type::parse_error; + } + + case 0x10: + { + error_message = "invalid string: control character U+0010 (DLE) must be escaped to \\u0010"; + return token_type::parse_error; + } + + case 0x11: + { + error_message = "invalid string: control character U+0011 (DC1) must be escaped to \\u0011"; + return token_type::parse_error; + } + + case 0x12: + { + error_message = "invalid string: control character U+0012 (DC2) must be escaped to \\u0012"; + return token_type::parse_error; + } + + case 0x13: + { + error_message = "invalid string: control character U+0013 (DC3) must be escaped to \\u0013"; + return token_type::parse_error; + } + + case 0x14: + { + error_message = "invalid string: control character U+0014 (DC4) must be escaped to \\u0014"; + return token_type::parse_error; + } + + case 0x15: + { + error_message = "invalid string: control character U+0015 (NAK) must be escaped to \\u0015"; + return token_type::parse_error; + } + + case 0x16: + { + error_message = "invalid string: control character U+0016 (SYN) must be escaped to \\u0016"; + return token_type::parse_error; + } + + case 0x17: + { + error_message = "invalid string: control character U+0017 (ETB) must be escaped to \\u0017"; + return token_type::parse_error; + } + + case 0x18: + { + error_message = "invalid string: control character U+0018 (CAN) must be escaped to \\u0018"; + return token_type::parse_error; + } + + case 0x19: + { + error_message = "invalid string: control character U+0019 (EM) must be escaped to \\u0019"; + return token_type::parse_error; + } + + case 0x1A: + { + error_message = "invalid string: control character U+001A (SUB) must be escaped to \\u001A"; + return token_type::parse_error; + } + + case 0x1B: + { + error_message = "invalid string: control character U+001B (ESC) must be escaped to \\u001B"; + return token_type::parse_error; + } + + case 0x1C: + { + error_message = "invalid string: control character U+001C (FS) must be escaped to \\u001C"; + return token_type::parse_error; + } + + case 0x1D: + { + error_message = "invalid string: control character U+001D (GS) must be escaped to \\u001D"; + return token_type::parse_error; + } + + case 0x1E: + { + error_message = "invalid string: control character U+001E (RS) must be escaped to \\u001E"; + return token_type::parse_error; + } + + case 0x1F: + { + error_message = "invalid string: control character U+001F (US) must be escaped to \\u001F"; + return token_type::parse_error; + } + + // U+0020..U+007F (except U+0022 (quote) and U+005C (backspace)) + case 0x20: + case 0x21: + case 0x23: + case 0x24: + case 0x25: + case 0x26: + case 0x27: + case 0x28: + case 0x29: + case 0x2A: + case 0x2B: + case 0x2C: + case 0x2D: + case 0x2E: + case 0x2F: + case 0x30: + case 0x31: + case 0x32: + case 0x33: + case 0x34: + case 0x35: + case 0x36: + case 0x37: + case 0x38: + case 0x39: + case 0x3A: + case 0x3B: + case 0x3C: + case 0x3D: + case 0x3E: + case 0x3F: + case 0x40: + case 0x41: + case 0x42: + case 0x43: + case 0x44: + case 0x45: + case 0x46: + case 0x47: + case 0x48: + case 0x49: + case 0x4A: + case 0x4B: + case 0x4C: + case 0x4D: + case 0x4E: + case 0x4F: + case 0x50: + case 0x51: + case 0x52: + case 0x53: + case 0x54: + case 0x55: + case 0x56: + case 0x57: + case 0x58: + case 0x59: + case 0x5A: + case 0x5B: + case 0x5D: + case 0x5E: + case 0x5F: + case 0x60: + case 0x61: + case 0x62: + case 0x63: + case 0x64: + case 0x65: + case 0x66: + case 0x67: + case 0x68: + case 0x69: + case 0x6A: + case 0x6B: + case 0x6C: + case 0x6D: + case 0x6E: + case 0x6F: + case 0x70: + case 0x71: + case 0x72: + case 0x73: + case 0x74: + case 0x75: + case 0x76: + case 0x77: + case 0x78: + case 0x79: + case 0x7A: + case 0x7B: + case 0x7C: + case 0x7D: + case 0x7E: + case 0x7F: + { + add(current); + break; + } + + // U+0080..U+07FF: bytes C2..DF 80..BF + case 0xC2: + case 0xC3: + case 0xC4: + case 0xC5: + case 0xC6: + case 0xC7: + case 0xC8: + case 0xC9: + case 0xCA: + case 0xCB: + case 0xCC: + case 0xCD: + case 0xCE: + case 0xCF: + case 0xD0: + case 0xD1: + case 0xD2: + case 0xD3: + case 0xD4: + case 0xD5: + case 0xD6: + case 0xD7: + case 0xD8: + case 0xD9: + case 0xDA: + case 0xDB: + case 0xDC: + case 0xDD: + case 0xDE: + case 0xDF: + { + if (JSON_HEDLEY_UNLIKELY(!next_byte_in_range({0x80, 0xBF}))) + { + return token_type::parse_error; + } + break; + } + + // U+0800..U+0FFF: bytes E0 A0..BF 80..BF + case 0xE0: + { + if (JSON_HEDLEY_UNLIKELY(!(next_byte_in_range({0xA0, 0xBF, 0x80, 0xBF})))) + { + return token_type::parse_error; + } + break; + } + + // U+1000..U+CFFF: bytes E1..EC 80..BF 80..BF + // U+E000..U+FFFF: bytes EE..EF 80..BF 80..BF + case 0xE1: + case 0xE2: + case 0xE3: + case 0xE4: + case 0xE5: + case 0xE6: + case 0xE7: + case 0xE8: + case 0xE9: + case 0xEA: + case 0xEB: + case 0xEC: + case 0xEE: + case 0xEF: + { + if (JSON_HEDLEY_UNLIKELY(!(next_byte_in_range({0x80, 0xBF, 0x80, 0xBF})))) + { + return token_type::parse_error; + } + break; + } + + // U+D000..U+D7FF: bytes ED 80..9F 80..BF + case 0xED: + { + if (JSON_HEDLEY_UNLIKELY(!(next_byte_in_range({0x80, 0x9F, 0x80, 0xBF})))) + { + return token_type::parse_error; + } + break; + } + + // U+10000..U+3FFFF F0 90..BF 80..BF 80..BF + case 0xF0: + { + if (JSON_HEDLEY_UNLIKELY(!(next_byte_in_range({0x90, 0xBF, 0x80, 0xBF, 0x80, 0xBF})))) + { + return token_type::parse_error; + } + break; + } + + // U+40000..U+FFFFF F1..F3 80..BF 80..BF 80..BF + case 0xF1: + case 0xF2: + case 0xF3: + { + if (JSON_HEDLEY_UNLIKELY(!(next_byte_in_range({0x80, 0xBF, 0x80, 0xBF, 0x80, 0xBF})))) + { + return token_type::parse_error; + } + break; + } + + // U+100000..U+10FFFF F4 80..8F 80..BF 80..BF + case 0xF4: + { + if (JSON_HEDLEY_UNLIKELY(!(next_byte_in_range({0x80, 0x8F, 0x80, 0xBF, 0x80, 0xBF})))) + { + return token_type::parse_error; + } + break; + } + + // remaining bytes (80..C1 and F5..FF) are ill-formed + default: + { + error_message = "invalid string: ill-formed UTF-8 byte"; + return token_type::parse_error; + } + } + } + } + + /*! + * @brief scan a comment + * @return whether comment could be scanned successfully + */ + bool scan_comment() + { + switch (get()) + { + // single-line comments skip input until a newline or EOF is read + case '/': + { + while (true) + { + switch (get()) + { + case '\n': + case '\r': + case std::char_traits::eof(): + case '\0': + return true; + + default: + break; + } + } + } + + // multi-line comments skip input until */ is read + case '*': + { + while (true) + { + switch (get()) + { + case std::char_traits::eof(): + case '\0': + { + error_message = "invalid comment; missing closing '*/'"; + return false; + } + + case '*': + { + switch (get()) + { + case '/': + return true; + + default: + { + unget(); + continue; + } + } + } + + default: + continue; + } + } + } + + // unexpected character after reading '/' + default: + { + error_message = "invalid comment; expecting '/' or '*' after '/'"; + return false; + } + } + } + + JSON_HEDLEY_NON_NULL(2) + static void strtof(float& f, const char* str, char** endptr) noexcept + { + f = std::strtof(str, endptr); + } + + JSON_HEDLEY_NON_NULL(2) + static void strtof(double& f, const char* str, char** endptr) noexcept + { + f = std::strtod(str, endptr); + } + + JSON_HEDLEY_NON_NULL(2) + static void strtof(long double& f, const char* str, char** endptr) noexcept + { + f = std::strtold(str, endptr); + } + + /*! + @brief scan a number literal + + This function scans a string according to Sect. 6 of RFC 7159. + + The function is realized with a deterministic finite state machine derived + from the grammar described in RFC 7159. Starting in state "init", the + input is read and used to determined the next state. Only state "done" + accepts the number. State "error" is a trap state to model errors. In the + table below, "anything" means any character but the ones listed before. + + state | 0 | 1-9 | e E | + | - | . | anything + ---------|----------|----------|----------|---------|---------|----------|----------- + init | zero | any1 | [error] | [error] | minus | [error] | [error] + minus | zero | any1 | [error] | [error] | [error] | [error] | [error] + zero | done | done | exponent | done | done | decimal1 | done + any1 | any1 | any1 | exponent | done | done | decimal1 | done + decimal1 | decimal2 | decimal2 | [error] | [error] | [error] | [error] | [error] + decimal2 | decimal2 | decimal2 | exponent | done | done | done | done + exponent | any2 | any2 | [error] | sign | sign | [error] | [error] + sign | any2 | any2 | [error] | [error] | [error] | [error] | [error] + any2 | any2 | any2 | done | done | done | done | done + + The state machine is realized with one label per state (prefixed with + "scan_number_") and `goto` statements between them. The state machine + contains cycles, but any cycle can be left when EOF is read. Therefore, + the function is guaranteed to terminate. + + During scanning, the read bytes are stored in token_buffer. This string is + then converted to a signed integer, an unsigned integer, or a + floating-point number. + + @return token_type::value_unsigned, token_type::value_integer, or + token_type::value_float if number could be successfully scanned, + token_type::parse_error otherwise + + @note The scanner is independent of the current locale. Internally, the + locale's decimal point is used instead of `.` to work with the + locale-dependent converters. + */ + token_type scan_number() // lgtm [cpp/use-of-goto] + { + // reset token_buffer to store the number's bytes + reset(); + + // the type of the parsed number; initially set to unsigned; will be + // changed if minus sign, decimal point or exponent is read + token_type number_type = token_type::value_unsigned; + + // state (init): we just found out we need to scan a number + switch (current) + { + case '-': + { + add(current); + goto scan_number_minus; + } + + case '0': + { + add(current); + goto scan_number_zero; + } + + case '1': + case '2': + case '3': + case '4': + case '5': + case '6': + case '7': + case '8': + case '9': + { + add(current); + goto scan_number_any1; + } + + // all other characters are rejected outside scan_number() + default: // LCOV_EXCL_LINE + JSON_ASSERT(false); // LCOV_EXCL_LINE + } + +scan_number_minus: + // state: we just parsed a leading minus sign + number_type = token_type::value_integer; + switch (get()) + { + case '0': + { + add(current); + goto scan_number_zero; + } + + case '1': + case '2': + case '3': + case '4': + case '5': + case '6': + case '7': + case '8': + case '9': + { + add(current); + goto scan_number_any1; + } + + default: + { + error_message = "invalid number; expected digit after '-'"; + return token_type::parse_error; + } + } + +scan_number_zero: + // state: we just parse a zero (maybe with a leading minus sign) + switch (get()) + { + case '.': + { + add(decimal_point_char); + goto scan_number_decimal1; + } + + case 'e': + case 'E': + { + add(current); + goto scan_number_exponent; + } + + default: + goto scan_number_done; + } + +scan_number_any1: + // state: we just parsed a number 0-9 (maybe with a leading minus sign) + switch (get()) + { + case '0': + case '1': + case '2': + case '3': + case '4': + case '5': + case '6': + case '7': + case '8': + case '9': + { + add(current); + goto scan_number_any1; + } + + case '.': + { + add(decimal_point_char); + goto scan_number_decimal1; + } + + case 'e': + case 'E': + { + add(current); + goto scan_number_exponent; + } + + default: + goto scan_number_done; + } + +scan_number_decimal1: + // state: we just parsed a decimal point + number_type = token_type::value_float; + switch (get()) + { + case '0': + case '1': + case '2': + case '3': + case '4': + case '5': + case '6': + case '7': + case '8': + case '9': + { + add(current); + goto scan_number_decimal2; + } + + default: + { + error_message = "invalid number; expected digit after '.'"; + return token_type::parse_error; + } + } + +scan_number_decimal2: + // we just parsed at least one number after a decimal point + switch (get()) + { + case '0': + case '1': + case '2': + case '3': + case '4': + case '5': + case '6': + case '7': + case '8': + case '9': + { + add(current); + goto scan_number_decimal2; + } + + case 'e': + case 'E': + { + add(current); + goto scan_number_exponent; + } + + default: + goto scan_number_done; + } + +scan_number_exponent: + // we just parsed an exponent + number_type = token_type::value_float; + switch (get()) + { + case '+': + case '-': + { + add(current); + goto scan_number_sign; + } + + case '0': + case '1': + case '2': + case '3': + case '4': + case '5': + case '6': + case '7': + case '8': + case '9': + { + add(current); + goto scan_number_any2; + } + + default: + { + error_message = + "invalid number; expected '+', '-', or digit after exponent"; + return token_type::parse_error; + } + } + +scan_number_sign: + // we just parsed an exponent sign + switch (get()) + { + case '0': + case '1': + case '2': + case '3': + case '4': + case '5': + case '6': + case '7': + case '8': + case '9': + { + add(current); + goto scan_number_any2; + } + + default: + { + error_message = "invalid number; expected digit after exponent sign"; + return token_type::parse_error; + } + } + +scan_number_any2: + // we just parsed a number after the exponent or exponent sign + switch (get()) + { + case '0': + case '1': + case '2': + case '3': + case '4': + case '5': + case '6': + case '7': + case '8': + case '9': + { + add(current); + goto scan_number_any2; + } + + default: + goto scan_number_done; + } + +scan_number_done: + // unget the character after the number (we only read it to know that + // we are done scanning a number) + unget(); + + char* endptr = nullptr; + errno = 0; + + // try to parse integers first and fall back to floats + if (number_type == token_type::value_unsigned) + { + const auto x = std::strtoull(token_buffer.data(), &endptr, 10); + + // we checked the number format before + JSON_ASSERT(endptr == token_buffer.data() + token_buffer.size()); + + if (errno == 0) + { + value_unsigned = static_cast(x); + if (value_unsigned == x) + { + return token_type::value_unsigned; + } + } + } + else if (number_type == token_type::value_integer) + { + const auto x = std::strtoll(token_buffer.data(), &endptr, 10); + + // we checked the number format before + JSON_ASSERT(endptr == token_buffer.data() + token_buffer.size()); + + if (errno == 0) + { + value_integer = static_cast(x); + if (value_integer == x) + { + return token_type::value_integer; + } + } + } + + // this code is reached if we parse a floating-point number or if an + // integer conversion above failed + strtof(value_float, token_buffer.data(), &endptr); + + // we checked the number format before + JSON_ASSERT(endptr == token_buffer.data() + token_buffer.size()); + + return token_type::value_float; + } + + /*! + @param[in] literal_text the literal text to expect + @param[in] length the length of the passed literal text + @param[in] return_type the token type to return on success + */ + JSON_HEDLEY_NON_NULL(2) + token_type scan_literal(const char_type* literal_text, const std::size_t length, + token_type return_type) + { + JSON_ASSERT(std::char_traits::to_char_type(current) == literal_text[0]); + for (std::size_t i = 1; i < length; ++i) + { + if (JSON_HEDLEY_UNLIKELY(std::char_traits::to_char_type(get()) != literal_text[i])) + { + error_message = "invalid literal"; + return token_type::parse_error; + } + } + return return_type; + } + + ///////////////////// + // input management + ///////////////////// + + /// reset token_buffer; current character is beginning of token + void reset() noexcept + { + token_buffer.clear(); + token_string.clear(); + token_string.push_back(std::char_traits::to_char_type(current)); + } + + /* + @brief get next character from the input + + This function provides the interface to the used input adapter. It does + not throw in case the input reached EOF, but returns a + `std::char_traits::eof()` in that case. Stores the scanned characters + for use in error messages. + + @return character read from the input + */ + char_int_type get() + { + ++position.chars_read_total; + ++position.chars_read_current_line; + + if (next_unget) + { + // just reset the next_unget variable and work with current + next_unget = false; + } + else + { + current = ia.get_character(); + } + + if (JSON_HEDLEY_LIKELY(current != std::char_traits::eof())) + { + token_string.push_back(std::char_traits::to_char_type(current)); + } + + if (current == '\n') + { + ++position.lines_read; + position.chars_read_current_line = 0; + } + + return current; + } + + /*! + @brief unget current character (read it again on next get) + + We implement unget by setting variable next_unget to true. The input is not + changed - we just simulate ungetting by modifying chars_read_total, + chars_read_current_line, and token_string. The next call to get() will + behave as if the unget character is read again. + */ + void unget() + { + next_unget = true; + + --position.chars_read_total; + + // in case we "unget" a newline, we have to also decrement the lines_read + if (position.chars_read_current_line == 0) + { + if (position.lines_read > 0) + { + --position.lines_read; + } + } + else + { + --position.chars_read_current_line; + } + + if (JSON_HEDLEY_LIKELY(current != std::char_traits::eof())) + { + JSON_ASSERT(!token_string.empty()); + token_string.pop_back(); + } + } + + /// add a character to token_buffer + void add(char_int_type c) + { + token_buffer.push_back(static_cast(c)); + } + + public: + ///////////////////// + // value getters + ///////////////////// + + /// return integer value + constexpr number_integer_t get_number_integer() const noexcept + { + return value_integer; + } + + /// return unsigned integer value + constexpr number_unsigned_t get_number_unsigned() const noexcept + { + return value_unsigned; + } + + /// return floating-point value + constexpr number_float_t get_number_float() const noexcept + { + return value_float; + } + + /// return current string value (implicitly resets the token; useful only once) + string_t& get_string() + { + return token_buffer; + } + + ///////////////////// + // diagnostics + ///////////////////// + + /// return position of last read token + constexpr position_t get_position() const noexcept + { + return position; + } + + /// return the last read token (for errors only). Will never contain EOF + /// (an arbitrary value that is not a valid char value, often -1), because + /// 255 may legitimately occur. May contain NUL, which should be escaped. + std::string get_token_string() const + { + // escape control characters + std::string result; + for (const auto c : token_string) + { + if (static_cast(c) <= '\x1F') + { + // escape control characters + std::array cs{{}}; + (std::snprintf)(cs.data(), cs.size(), "", static_cast(c)); + result += cs.data(); + } + else + { + // add character as is + result.push_back(static_cast(c)); + } + } + + return result; + } + + /// return syntax error message + JSON_HEDLEY_RETURNS_NON_NULL + constexpr const char* get_error_message() const noexcept + { + return error_message; + } + + ///////////////////// + // actual scanner + ///////////////////// + + /*! + @brief skip the UTF-8 byte order mark + @return true iff there is no BOM or the correct BOM has been skipped + */ + bool skip_bom() + { + if (get() == 0xEF) + { + // check if we completely parse the BOM + return get() == 0xBB && get() == 0xBF; + } + + // the first character is not the beginning of the BOM; unget it to + // process is later + unget(); + return true; + } + + void skip_whitespace() + { + do + { + get(); + } + while (current == ' ' || current == '\t' || current == '\n' || current == '\r'); + } + + token_type scan() + { + // initially, skip the BOM + if (position.chars_read_total == 0 && !skip_bom()) + { + error_message = "invalid BOM; must be 0xEF 0xBB 0xBF if given"; + return token_type::parse_error; + } + + // read next character and ignore whitespace + skip_whitespace(); + + // ignore comments + while (ignore_comments && current == '/') + { + if (!scan_comment()) + { + return token_type::parse_error; + } + + // skip following whitespace + skip_whitespace(); + } + + switch (current) + { + // structural characters + case '[': + return token_type::begin_array; + case ']': + return token_type::end_array; + case '{': + return token_type::begin_object; + case '}': + return token_type::end_object; + case ':': + return token_type::name_separator; + case ',': + return token_type::value_separator; + + // literals + case 't': + { + std::array true_literal = {{'t', 'r', 'u', 'e'}}; + return scan_literal(true_literal.data(), true_literal.size(), token_type::literal_true); + } + case 'f': + { + std::array false_literal = {{'f', 'a', 'l', 's', 'e'}}; + return scan_literal(false_literal.data(), false_literal.size(), token_type::literal_false); + } + case 'n': + { + std::array null_literal = {{'n', 'u', 'l', 'l'}}; + return scan_literal(null_literal.data(), null_literal.size(), token_type::literal_null); + } + + // string + case '\"': + return scan_string(); + + // number + case '-': + case '0': + case '1': + case '2': + case '3': + case '4': + case '5': + case '6': + case '7': + case '8': + case '9': + return scan_number(); + + // end of input (the null byte is needed when parsing from + // string literals) + case '\0': + case std::char_traits::eof(): + return token_type::end_of_input; + + // error + default: + error_message = "invalid literal"; + return token_type::parse_error; + } + } + + private: + /// input adapter + InputAdapterType ia; + + /// whether comments should be ignored (true) or signaled as errors (false) + const bool ignore_comments = false; + + /// the current character + char_int_type current = std::char_traits::eof(); + + /// whether the next get() call should just return current + bool next_unget = false; + + /// the start position of the current token + position_t position {}; + + /// raw input token string (for error messages) + std::vector token_string {}; + + /// buffer for variable-length tokens (numbers, strings) + string_t token_buffer {}; + + /// a description of occurred lexer errors + const char* error_message = ""; + + // number values + number_integer_t value_integer = 0; + number_unsigned_t value_unsigned = 0; + number_float_t value_float = 0; + + /// the decimal point + const char_int_type decimal_point_char = '.'; +}; +} // namespace detail +} // namespace nlohmann + +// #include + +// #include + + +#include // size_t +#include // declval +#include // string + +// #include + +// #include + + +namespace nlohmann +{ +namespace detail +{ +template +using null_function_t = decltype(std::declval().null()); + +template +using boolean_function_t = + decltype(std::declval().boolean(std::declval())); + +template +using number_integer_function_t = + decltype(std::declval().number_integer(std::declval())); + +template +using number_unsigned_function_t = + decltype(std::declval().number_unsigned(std::declval())); + +template +using number_float_function_t = decltype(std::declval().number_float( + std::declval(), std::declval())); + +template +using string_function_t = + decltype(std::declval().string(std::declval())); + +template +using binary_function_t = + decltype(std::declval().binary(std::declval())); + +template +using start_object_function_t = + decltype(std::declval().start_object(std::declval())); + +template +using key_function_t = + decltype(std::declval().key(std::declval())); + +template +using end_object_function_t = decltype(std::declval().end_object()); + +template +using start_array_function_t = + decltype(std::declval().start_array(std::declval())); + +template +using end_array_function_t = decltype(std::declval().end_array()); + +template +using parse_error_function_t = decltype(std::declval().parse_error( + std::declval(), std::declval(), + std::declval())); + +template +struct is_sax +{ + private: + static_assert(is_basic_json::value, + "BasicJsonType must be of type basic_json<...>"); + + using number_integer_t = typename BasicJsonType::number_integer_t; + using number_unsigned_t = typename BasicJsonType::number_unsigned_t; + using number_float_t = typename BasicJsonType::number_float_t; + using string_t = typename BasicJsonType::string_t; + using binary_t = typename BasicJsonType::binary_t; + using exception_t = typename BasicJsonType::exception; + + public: + static constexpr bool value = + is_detected_exact::value && + is_detected_exact::value && + is_detected_exact::value && + is_detected_exact::value && + is_detected_exact::value && + is_detected_exact::value && + is_detected_exact::value && + is_detected_exact::value && + is_detected_exact::value && + is_detected_exact::value && + is_detected_exact::value && + is_detected_exact::value && + is_detected_exact::value; +}; + +template +struct is_sax_static_asserts +{ + private: + static_assert(is_basic_json::value, + "BasicJsonType must be of type basic_json<...>"); + + using number_integer_t = typename BasicJsonType::number_integer_t; + using number_unsigned_t = typename BasicJsonType::number_unsigned_t; + using number_float_t = typename BasicJsonType::number_float_t; + using string_t = typename BasicJsonType::string_t; + using binary_t = typename BasicJsonType::binary_t; + using exception_t = typename BasicJsonType::exception; + + public: + static_assert(is_detected_exact::value, + "Missing/invalid function: bool null()"); + static_assert(is_detected_exact::value, + "Missing/invalid function: bool boolean(bool)"); + static_assert(is_detected_exact::value, + "Missing/invalid function: bool boolean(bool)"); + static_assert( + is_detected_exact::value, + "Missing/invalid function: bool number_integer(number_integer_t)"); + static_assert( + is_detected_exact::value, + "Missing/invalid function: bool number_unsigned(number_unsigned_t)"); + static_assert(is_detected_exact::value, + "Missing/invalid function: bool number_float(number_float_t, const string_t&)"); + static_assert( + is_detected_exact::value, + "Missing/invalid function: bool string(string_t&)"); + static_assert( + is_detected_exact::value, + "Missing/invalid function: bool binary(binary_t&)"); + static_assert(is_detected_exact::value, + "Missing/invalid function: bool start_object(std::size_t)"); + static_assert(is_detected_exact::value, + "Missing/invalid function: bool key(string_t&)"); + static_assert(is_detected_exact::value, + "Missing/invalid function: bool end_object()"); + static_assert(is_detected_exact::value, + "Missing/invalid function: bool start_array(std::size_t)"); + static_assert(is_detected_exact::value, + "Missing/invalid function: bool end_array()"); + static_assert( + is_detected_exact::value, + "Missing/invalid function: bool parse_error(std::size_t, const " + "std::string&, const exception&)"); +}; +} // namespace detail +} // namespace nlohmann + +// #include + + +namespace nlohmann +{ +namespace detail +{ + +/// how to treat CBOR tags +enum class cbor_tag_handler_t +{ + error, ///< throw a parse_error exception in case of a tag + ignore ///< ignore tags +}; + +/*! +@brief determine system byte order + +@return true if and only if system's byte order is little endian + +@note from https://stackoverflow.com/a/1001328/266378 +*/ +static inline bool little_endianess(int num = 1) noexcept +{ + return *reinterpret_cast(&num) == 1; +} + + +/////////////////// +// binary reader // +/////////////////// + +/*! +@brief deserialization of CBOR, MessagePack, and UBJSON values +*/ +template> +class binary_reader +{ + using number_integer_t = typename BasicJsonType::number_integer_t; + using number_unsigned_t = typename BasicJsonType::number_unsigned_t; + using number_float_t = typename BasicJsonType::number_float_t; + using string_t = typename BasicJsonType::string_t; + using binary_t = typename BasicJsonType::binary_t; + using json_sax_t = SAX; + using char_type = typename InputAdapterType::char_type; + using char_int_type = typename std::char_traits::int_type; + + public: + /*! + @brief create a binary reader + + @param[in] adapter input adapter to read from + */ + explicit binary_reader(InputAdapterType&& adapter) : ia(std::move(adapter)) + { + (void)detail::is_sax_static_asserts {}; + } + + // make class move-only + binary_reader(const binary_reader&) = delete; + binary_reader(binary_reader&&) = default; + binary_reader& operator=(const binary_reader&) = delete; + binary_reader& operator=(binary_reader&&) = default; + ~binary_reader() = default; + + /*! + @param[in] format the binary format to parse + @param[in] sax_ a SAX event processor + @param[in] strict whether to expect the input to be consumed completed + @param[in] tag_handler how to treat CBOR tags + + @return + */ + JSON_HEDLEY_NON_NULL(3) + bool sax_parse(const input_format_t format, + json_sax_t* sax_, + const bool strict = true, + const cbor_tag_handler_t tag_handler = cbor_tag_handler_t::error) + { + sax = sax_; + bool result = false; + + switch (format) + { + case input_format_t::bson: + result = parse_bson_internal(); + break; + + case input_format_t::cbor: + result = parse_cbor_internal(true, tag_handler); + break; + + case input_format_t::msgpack: + result = parse_msgpack_internal(); + break; + + case input_format_t::ubjson: + result = parse_ubjson_internal(); + break; + + default: // LCOV_EXCL_LINE + JSON_ASSERT(false); // LCOV_EXCL_LINE + } + + // strict mode: next byte must be EOF + if (result && strict) + { + if (format == input_format_t::ubjson) + { + get_ignore_noop(); + } + else + { + get(); + } + + if (JSON_HEDLEY_UNLIKELY(current != std::char_traits::eof())) + { + return sax->parse_error(chars_read, get_token_string(), + parse_error::create(110, chars_read, exception_message(format, "expected end of input; last byte: 0x" + get_token_string(), "value"))); + } + } + + return result; + } + + private: + ////////// + // BSON // + ////////// + + /*! + @brief Reads in a BSON-object and passes it to the SAX-parser. + @return whether a valid BSON-value was passed to the SAX parser + */ + bool parse_bson_internal() + { + std::int32_t document_size{}; + get_number(input_format_t::bson, document_size); + + if (JSON_HEDLEY_UNLIKELY(!sax->start_object(std::size_t(-1)))) + { + return false; + } + + if (JSON_HEDLEY_UNLIKELY(!parse_bson_element_list(/*is_array*/false))) + { + return false; + } + + return sax->end_object(); + } + + /*! + @brief Parses a C-style string from the BSON input. + @param[in, out] result A reference to the string variable where the read + string is to be stored. + @return `true` if the \x00-byte indicating the end of the string was + encountered before the EOF; false` indicates an unexpected EOF. + */ + bool get_bson_cstr(string_t& result) + { + auto out = std::back_inserter(result); + while (true) + { + get(); + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::bson, "cstring"))) + { + return false; + } + if (current == 0x00) + { + return true; + } + *out++ = static_cast(current); + } + } + + /*! + @brief Parses a zero-terminated string of length @a len from the BSON + input. + @param[in] len The length (including the zero-byte at the end) of the + string to be read. + @param[in, out] result A reference to the string variable where the read + string is to be stored. + @tparam NumberType The type of the length @a len + @pre len >= 1 + @return `true` if the string was successfully parsed + */ + template + bool get_bson_string(const NumberType len, string_t& result) + { + if (JSON_HEDLEY_UNLIKELY(len < 1)) + { + auto last_token = get_token_string(); + return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, exception_message(input_format_t::bson, "string length must be at least 1, is " + std::to_string(len), "string"))); + } + + return get_string(input_format_t::bson, len - static_cast(1), result) && get() != std::char_traits::eof(); + } + + /*! + @brief Parses a byte array input of length @a len from the BSON input. + @param[in] len The length of the byte array to be read. + @param[in, out] result A reference to the binary variable where the read + array is to be stored. + @tparam NumberType The type of the length @a len + @pre len >= 0 + @return `true` if the byte array was successfully parsed + */ + template + bool get_bson_binary(const NumberType len, binary_t& result) + { + if (JSON_HEDLEY_UNLIKELY(len < 0)) + { + auto last_token = get_token_string(); + return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, exception_message(input_format_t::bson, "byte array length cannot be negative, is " + std::to_string(len), "binary"))); + } + + // All BSON binary values have a subtype + std::uint8_t subtype{}; + get_number(input_format_t::bson, subtype); + result.set_subtype(subtype); + + return get_binary(input_format_t::bson, len, result); + } + + /*! + @brief Read a BSON document element of the given @a element_type. + @param[in] element_type The BSON element type, c.f. http://bsonspec.org/spec.html + @param[in] element_type_parse_position The position in the input stream, + where the `element_type` was read. + @warning Not all BSON element types are supported yet. An unsupported + @a element_type will give rise to a parse_error.114: + Unsupported BSON record type 0x... + @return whether a valid BSON-object/array was passed to the SAX parser + */ + bool parse_bson_element_internal(const char_int_type element_type, + const std::size_t element_type_parse_position) + { + switch (element_type) + { + case 0x01: // double + { + double number{}; + return get_number(input_format_t::bson, number) && sax->number_float(static_cast(number), ""); + } + + case 0x02: // string + { + std::int32_t len{}; + string_t value; + return get_number(input_format_t::bson, len) && get_bson_string(len, value) && sax->string(value); + } + + case 0x03: // object + { + return parse_bson_internal(); + } + + case 0x04: // array + { + return parse_bson_array(); + } + + case 0x05: // binary + { + std::int32_t len{}; + binary_t value; + return get_number(input_format_t::bson, len) && get_bson_binary(len, value) && sax->binary(value); + } + + case 0x08: // boolean + { + return sax->boolean(get() != 0); + } + + case 0x0A: // null + { + return sax->null(); + } + + case 0x10: // int32 + { + std::int32_t value{}; + return get_number(input_format_t::bson, value) && sax->number_integer(value); + } + + case 0x12: // int64 + { + std::int64_t value{}; + return get_number(input_format_t::bson, value) && sax->number_integer(value); + } + + default: // anything else not supported (yet) + { + std::array cr{{}}; + (std::snprintf)(cr.data(), cr.size(), "%.2hhX", static_cast(element_type)); + return sax->parse_error(element_type_parse_position, std::string(cr.data()), parse_error::create(114, element_type_parse_position, "Unsupported BSON record type 0x" + std::string(cr.data()))); + } + } + } + + /*! + @brief Read a BSON element list (as specified in the BSON-spec) + + The same binary layout is used for objects and arrays, hence it must be + indicated with the argument @a is_array which one is expected + (true --> array, false --> object). + + @param[in] is_array Determines if the element list being read is to be + treated as an object (@a is_array == false), or as an + array (@a is_array == true). + @return whether a valid BSON-object/array was passed to the SAX parser + */ + bool parse_bson_element_list(const bool is_array) + { + string_t key; + + while (auto element_type = get()) + { + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::bson, "element list"))) + { + return false; + } + + const std::size_t element_type_parse_position = chars_read; + if (JSON_HEDLEY_UNLIKELY(!get_bson_cstr(key))) + { + return false; + } + + if (!is_array && !sax->key(key)) + { + return false; + } + + if (JSON_HEDLEY_UNLIKELY(!parse_bson_element_internal(element_type, element_type_parse_position))) + { + return false; + } + + // get_bson_cstr only appends + key.clear(); + } + + return true; + } + + /*! + @brief Reads an array from the BSON input and passes it to the SAX-parser. + @return whether a valid BSON-array was passed to the SAX parser + */ + bool parse_bson_array() + { + std::int32_t document_size{}; + get_number(input_format_t::bson, document_size); + + if (JSON_HEDLEY_UNLIKELY(!sax->start_array(std::size_t(-1)))) + { + return false; + } + + if (JSON_HEDLEY_UNLIKELY(!parse_bson_element_list(/*is_array*/true))) + { + return false; + } + + return sax->end_array(); + } + + ////////// + // CBOR // + ////////// + + /*! + @param[in] get_char whether a new character should be retrieved from the + input (true) or whether the last read character should + be considered instead (false) + @param[in] tag_handler how CBOR tags should be treated + + @return whether a valid CBOR value was passed to the SAX parser + */ + bool parse_cbor_internal(const bool get_char, + const cbor_tag_handler_t tag_handler) + { + switch (get_char ? get() : current) + { + // EOF + case std::char_traits::eof(): + return unexpect_eof(input_format_t::cbor, "value"); + + // Integer 0x00..0x17 (0..23) + case 0x00: + case 0x01: + case 0x02: + case 0x03: + case 0x04: + case 0x05: + case 0x06: + case 0x07: + case 0x08: + case 0x09: + case 0x0A: + case 0x0B: + case 0x0C: + case 0x0D: + case 0x0E: + case 0x0F: + case 0x10: + case 0x11: + case 0x12: + case 0x13: + case 0x14: + case 0x15: + case 0x16: + case 0x17: + return sax->number_unsigned(static_cast(current)); + + case 0x18: // Unsigned integer (one-byte uint8_t follows) + { + std::uint8_t number{}; + return get_number(input_format_t::cbor, number) && sax->number_unsigned(number); + } + + case 0x19: // Unsigned integer (two-byte uint16_t follows) + { + std::uint16_t number{}; + return get_number(input_format_t::cbor, number) && sax->number_unsigned(number); + } + + case 0x1A: // Unsigned integer (four-byte uint32_t follows) + { + std::uint32_t number{}; + return get_number(input_format_t::cbor, number) && sax->number_unsigned(number); + } + + case 0x1B: // Unsigned integer (eight-byte uint64_t follows) + { + std::uint64_t number{}; + return get_number(input_format_t::cbor, number) && sax->number_unsigned(number); + } + + // Negative integer -1-0x00..-1-0x17 (-1..-24) + case 0x20: + case 0x21: + case 0x22: + case 0x23: + case 0x24: + case 0x25: + case 0x26: + case 0x27: + case 0x28: + case 0x29: + case 0x2A: + case 0x2B: + case 0x2C: + case 0x2D: + case 0x2E: + case 0x2F: + case 0x30: + case 0x31: + case 0x32: + case 0x33: + case 0x34: + case 0x35: + case 0x36: + case 0x37: + return sax->number_integer(static_cast(0x20 - 1 - current)); + + case 0x38: // Negative integer (one-byte uint8_t follows) + { + std::uint8_t number{}; + return get_number(input_format_t::cbor, number) && sax->number_integer(static_cast(-1) - number); + } + + case 0x39: // Negative integer -1-n (two-byte uint16_t follows) + { + std::uint16_t number{}; + return get_number(input_format_t::cbor, number) && sax->number_integer(static_cast(-1) - number); + } + + case 0x3A: // Negative integer -1-n (four-byte uint32_t follows) + { + std::uint32_t number{}; + return get_number(input_format_t::cbor, number) && sax->number_integer(static_cast(-1) - number); + } + + case 0x3B: // Negative integer -1-n (eight-byte uint64_t follows) + { + std::uint64_t number{}; + return get_number(input_format_t::cbor, number) && sax->number_integer(static_cast(-1) + - static_cast(number)); + } + + // Binary data (0x00..0x17 bytes follow) + case 0x40: + case 0x41: + case 0x42: + case 0x43: + case 0x44: + case 0x45: + case 0x46: + case 0x47: + case 0x48: + case 0x49: + case 0x4A: + case 0x4B: + case 0x4C: + case 0x4D: + case 0x4E: + case 0x4F: + case 0x50: + case 0x51: + case 0x52: + case 0x53: + case 0x54: + case 0x55: + case 0x56: + case 0x57: + case 0x58: // Binary data (one-byte uint8_t for n follows) + case 0x59: // Binary data (two-byte uint16_t for n follow) + case 0x5A: // Binary data (four-byte uint32_t for n follow) + case 0x5B: // Binary data (eight-byte uint64_t for n follow) + case 0x5F: // Binary data (indefinite length) + { + binary_t b; + return get_cbor_binary(b) && sax->binary(b); + } + + // UTF-8 string (0x00..0x17 bytes follow) + case 0x60: + case 0x61: + case 0x62: + case 0x63: + case 0x64: + case 0x65: + case 0x66: + case 0x67: + case 0x68: + case 0x69: + case 0x6A: + case 0x6B: + case 0x6C: + case 0x6D: + case 0x6E: + case 0x6F: + case 0x70: + case 0x71: + case 0x72: + case 0x73: + case 0x74: + case 0x75: + case 0x76: + case 0x77: + case 0x78: // UTF-8 string (one-byte uint8_t for n follows) + case 0x79: // UTF-8 string (two-byte uint16_t for n follow) + case 0x7A: // UTF-8 string (four-byte uint32_t for n follow) + case 0x7B: // UTF-8 string (eight-byte uint64_t for n follow) + case 0x7F: // UTF-8 string (indefinite length) + { + string_t s; + return get_cbor_string(s) && sax->string(s); + } + + // array (0x00..0x17 data items follow) + case 0x80: + case 0x81: + case 0x82: + case 0x83: + case 0x84: + case 0x85: + case 0x86: + case 0x87: + case 0x88: + case 0x89: + case 0x8A: + case 0x8B: + case 0x8C: + case 0x8D: + case 0x8E: + case 0x8F: + case 0x90: + case 0x91: + case 0x92: + case 0x93: + case 0x94: + case 0x95: + case 0x96: + case 0x97: + return get_cbor_array(static_cast(static_cast(current) & 0x1Fu), tag_handler); + + case 0x98: // array (one-byte uint8_t for n follows) + { + std::uint8_t len{}; + return get_number(input_format_t::cbor, len) && get_cbor_array(static_cast(len), tag_handler); + } + + case 0x99: // array (two-byte uint16_t for n follow) + { + std::uint16_t len{}; + return get_number(input_format_t::cbor, len) && get_cbor_array(static_cast(len), tag_handler); + } + + case 0x9A: // array (four-byte uint32_t for n follow) + { + std::uint32_t len{}; + return get_number(input_format_t::cbor, len) && get_cbor_array(static_cast(len), tag_handler); + } + + case 0x9B: // array (eight-byte uint64_t for n follow) + { + std::uint64_t len{}; + return get_number(input_format_t::cbor, len) && get_cbor_array(static_cast(len), tag_handler); + } + + case 0x9F: // array (indefinite length) + return get_cbor_array(std::size_t(-1), tag_handler); + + // map (0x00..0x17 pairs of data items follow) + case 0xA0: + case 0xA1: + case 0xA2: + case 0xA3: + case 0xA4: + case 0xA5: + case 0xA6: + case 0xA7: + case 0xA8: + case 0xA9: + case 0xAA: + case 0xAB: + case 0xAC: + case 0xAD: + case 0xAE: + case 0xAF: + case 0xB0: + case 0xB1: + case 0xB2: + case 0xB3: + case 0xB4: + case 0xB5: + case 0xB6: + case 0xB7: + return get_cbor_object(static_cast(static_cast(current) & 0x1Fu), tag_handler); + + case 0xB8: // map (one-byte uint8_t for n follows) + { + std::uint8_t len{}; + return get_number(input_format_t::cbor, len) && get_cbor_object(static_cast(len), tag_handler); + } + + case 0xB9: // map (two-byte uint16_t for n follow) + { + std::uint16_t len{}; + return get_number(input_format_t::cbor, len) && get_cbor_object(static_cast(len), tag_handler); + } + + case 0xBA: // map (four-byte uint32_t for n follow) + { + std::uint32_t len{}; + return get_number(input_format_t::cbor, len) && get_cbor_object(static_cast(len), tag_handler); + } + + case 0xBB: // map (eight-byte uint64_t for n follow) + { + std::uint64_t len{}; + return get_number(input_format_t::cbor, len) && get_cbor_object(static_cast(len), tag_handler); + } + + case 0xBF: // map (indefinite length) + return get_cbor_object(std::size_t(-1), tag_handler); + + case 0xC6: // tagged item + case 0xC7: + case 0xC8: + case 0xC9: + case 0xCA: + case 0xCB: + case 0xCC: + case 0xCD: + case 0xCE: + case 0xCF: + case 0xD0: + case 0xD1: + case 0xD2: + case 0xD3: + case 0xD4: + case 0xD8: // tagged item (1 bytes follow) + case 0xD9: // tagged item (2 bytes follow) + case 0xDA: // tagged item (4 bytes follow) + case 0xDB: // tagged item (8 bytes follow) + { + switch (tag_handler) + { + case cbor_tag_handler_t::error: + { + auto last_token = get_token_string(); + return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, exception_message(input_format_t::cbor, "invalid byte: 0x" + last_token, "value"))); + } + + case cbor_tag_handler_t::ignore: + { + switch (current) + { + case 0xD8: + { + std::uint8_t len{}; + get_number(input_format_t::cbor, len); + break; + } + case 0xD9: + { + std::uint16_t len{}; + get_number(input_format_t::cbor, len); + break; + } + case 0xDA: + { + std::uint32_t len{}; + get_number(input_format_t::cbor, len); + break; + } + case 0xDB: + { + std::uint64_t len{}; + get_number(input_format_t::cbor, len); + break; + } + default: + break; + } + return parse_cbor_internal(true, tag_handler); + } + + default: // LCOV_EXCL_LINE + JSON_ASSERT(false); // LCOV_EXCL_LINE + } + } + + case 0xF4: // false + return sax->boolean(false); + + case 0xF5: // true + return sax->boolean(true); + + case 0xF6: // null + return sax->null(); + + case 0xF9: // Half-Precision Float (two-byte IEEE 754) + { + const auto byte1_raw = get(); + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::cbor, "number"))) + { + return false; + } + const auto byte2_raw = get(); + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::cbor, "number"))) + { + return false; + } + + const auto byte1 = static_cast(byte1_raw); + const auto byte2 = static_cast(byte2_raw); + + // code from RFC 7049, Appendix D, Figure 3: + // As half-precision floating-point numbers were only added + // to IEEE 754 in 2008, today's programming platforms often + // still only have limited support for them. It is very + // easy to include at least decoding support for them even + // without such support. An example of a small decoder for + // half-precision floating-point numbers in the C language + // is shown in Fig. 3. + const auto half = static_cast((byte1 << 8u) + byte2); + const double val = [&half] + { + const int exp = (half >> 10u) & 0x1Fu; + const unsigned int mant = half & 0x3FFu; + JSON_ASSERT(0 <= exp&& exp <= 32); + JSON_ASSERT(mant <= 1024); + switch (exp) + { + case 0: + return std::ldexp(mant, -24); + case 31: + return (mant == 0) + ? std::numeric_limits::infinity() + : std::numeric_limits::quiet_NaN(); + default: + return std::ldexp(mant + 1024, exp - 25); + } + }(); + return sax->number_float((half & 0x8000u) != 0 + ? static_cast(-val) + : static_cast(val), ""); + } + + case 0xFA: // Single-Precision Float (four-byte IEEE 754) + { + float number{}; + return get_number(input_format_t::cbor, number) && sax->number_float(static_cast(number), ""); + } + + case 0xFB: // Double-Precision Float (eight-byte IEEE 754) + { + double number{}; + return get_number(input_format_t::cbor, number) && sax->number_float(static_cast(number), ""); + } + + default: // anything else (0xFF is handled inside the other types) + { + auto last_token = get_token_string(); + return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, exception_message(input_format_t::cbor, "invalid byte: 0x" + last_token, "value"))); + } + } + } + + /*! + @brief reads a CBOR string + + This function first reads starting bytes to determine the expected + string length and then copies this number of bytes into a string. + Additionally, CBOR's strings with indefinite lengths are supported. + + @param[out] result created string + + @return whether string creation completed + */ + bool get_cbor_string(string_t& result) + { + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::cbor, "string"))) + { + return false; + } + + switch (current) + { + // UTF-8 string (0x00..0x17 bytes follow) + case 0x60: + case 0x61: + case 0x62: + case 0x63: + case 0x64: + case 0x65: + case 0x66: + case 0x67: + case 0x68: + case 0x69: + case 0x6A: + case 0x6B: + case 0x6C: + case 0x6D: + case 0x6E: + case 0x6F: + case 0x70: + case 0x71: + case 0x72: + case 0x73: + case 0x74: + case 0x75: + case 0x76: + case 0x77: + { + return get_string(input_format_t::cbor, static_cast(current) & 0x1Fu, result); + } + + case 0x78: // UTF-8 string (one-byte uint8_t for n follows) + { + std::uint8_t len{}; + return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result); + } + + case 0x79: // UTF-8 string (two-byte uint16_t for n follow) + { + std::uint16_t len{}; + return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result); + } + + case 0x7A: // UTF-8 string (four-byte uint32_t for n follow) + { + std::uint32_t len{}; + return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result); + } + + case 0x7B: // UTF-8 string (eight-byte uint64_t for n follow) + { + std::uint64_t len{}; + return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result); + } + + case 0x7F: // UTF-8 string (indefinite length) + { + while (get() != 0xFF) + { + string_t chunk; + if (!get_cbor_string(chunk)) + { + return false; + } + result.append(chunk); + } + return true; + } + + default: + { + auto last_token = get_token_string(); + return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(input_format_t::cbor, "expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x" + last_token, "string"))); + } + } + } + + /*! + @brief reads a CBOR byte array + + This function first reads starting bytes to determine the expected + byte array length and then copies this number of bytes into the byte array. + Additionally, CBOR's byte arrays with indefinite lengths are supported. + + @param[out] result created byte array + + @return whether byte array creation completed + */ + bool get_cbor_binary(binary_t& result) + { + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::cbor, "binary"))) + { + return false; + } + + switch (current) + { + // Binary data (0x00..0x17 bytes follow) + case 0x40: + case 0x41: + case 0x42: + case 0x43: + case 0x44: + case 0x45: + case 0x46: + case 0x47: + case 0x48: + case 0x49: + case 0x4A: + case 0x4B: + case 0x4C: + case 0x4D: + case 0x4E: + case 0x4F: + case 0x50: + case 0x51: + case 0x52: + case 0x53: + case 0x54: + case 0x55: + case 0x56: + case 0x57: + { + return get_binary(input_format_t::cbor, static_cast(current) & 0x1Fu, result); + } + + case 0x58: // Binary data (one-byte uint8_t for n follows) + { + std::uint8_t len{}; + return get_number(input_format_t::cbor, len) && + get_binary(input_format_t::cbor, len, result); + } + + case 0x59: // Binary data (two-byte uint16_t for n follow) + { + std::uint16_t len{}; + return get_number(input_format_t::cbor, len) && + get_binary(input_format_t::cbor, len, result); + } + + case 0x5A: // Binary data (four-byte uint32_t for n follow) + { + std::uint32_t len{}; + return get_number(input_format_t::cbor, len) && + get_binary(input_format_t::cbor, len, result); + } + + case 0x5B: // Binary data (eight-byte uint64_t for n follow) + { + std::uint64_t len{}; + return get_number(input_format_t::cbor, len) && + get_binary(input_format_t::cbor, len, result); + } + + case 0x5F: // Binary data (indefinite length) + { + while (get() != 0xFF) + { + binary_t chunk; + if (!get_cbor_binary(chunk)) + { + return false; + } + result.insert(result.end(), chunk.begin(), chunk.end()); + } + return true; + } + + default: + { + auto last_token = get_token_string(); + return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(input_format_t::cbor, "expected length specification (0x40-0x5B) or indefinite binary array type (0x5F); last byte: 0x" + last_token, "binary"))); + } + } + } + + /*! + @param[in] len the length of the array or std::size_t(-1) for an + array of indefinite size + @param[in] tag_handler how CBOR tags should be treated + @return whether array creation completed + */ + bool get_cbor_array(const std::size_t len, + const cbor_tag_handler_t tag_handler) + { + if (JSON_HEDLEY_UNLIKELY(!sax->start_array(len))) + { + return false; + } + + if (len != std::size_t(-1)) + { + for (std::size_t i = 0; i < len; ++i) + { + if (JSON_HEDLEY_UNLIKELY(!parse_cbor_internal(true, tag_handler))) + { + return false; + } + } + } + else + { + while (get() != 0xFF) + { + if (JSON_HEDLEY_UNLIKELY(!parse_cbor_internal(false, tag_handler))) + { + return false; + } + } + } + + return sax->end_array(); + } + + /*! + @param[in] len the length of the object or std::size_t(-1) for an + object of indefinite size + @param[in] tag_handler how CBOR tags should be treated + @return whether object creation completed + */ + bool get_cbor_object(const std::size_t len, + const cbor_tag_handler_t tag_handler) + { + if (JSON_HEDLEY_UNLIKELY(!sax->start_object(len))) + { + return false; + } + + string_t key; + if (len != std::size_t(-1)) + { + for (std::size_t i = 0; i < len; ++i) + { + get(); + if (JSON_HEDLEY_UNLIKELY(!get_cbor_string(key) || !sax->key(key))) + { + return false; + } + + if (JSON_HEDLEY_UNLIKELY(!parse_cbor_internal(true, tag_handler))) + { + return false; + } + key.clear(); + } + } + else + { + while (get() != 0xFF) + { + if (JSON_HEDLEY_UNLIKELY(!get_cbor_string(key) || !sax->key(key))) + { + return false; + } + + if (JSON_HEDLEY_UNLIKELY(!parse_cbor_internal(true, tag_handler))) + { + return false; + } + key.clear(); + } + } + + return sax->end_object(); + } + + ///////////// + // MsgPack // + ///////////// + + /*! + @return whether a valid MessagePack value was passed to the SAX parser + */ + bool parse_msgpack_internal() + { + switch (get()) + { + // EOF + case std::char_traits::eof(): + return unexpect_eof(input_format_t::msgpack, "value"); + + // positive fixint + case 0x00: + case 0x01: + case 0x02: + case 0x03: + case 0x04: + case 0x05: + case 0x06: + case 0x07: + case 0x08: + case 0x09: + case 0x0A: + case 0x0B: + case 0x0C: + case 0x0D: + case 0x0E: + case 0x0F: + case 0x10: + case 0x11: + case 0x12: + case 0x13: + case 0x14: + case 0x15: + case 0x16: + case 0x17: + case 0x18: + case 0x19: + case 0x1A: + case 0x1B: + case 0x1C: + case 0x1D: + case 0x1E: + case 0x1F: + case 0x20: + case 0x21: + case 0x22: + case 0x23: + case 0x24: + case 0x25: + case 0x26: + case 0x27: + case 0x28: + case 0x29: + case 0x2A: + case 0x2B: + case 0x2C: + case 0x2D: + case 0x2E: + case 0x2F: + case 0x30: + case 0x31: + case 0x32: + case 0x33: + case 0x34: + case 0x35: + case 0x36: + case 0x37: + case 0x38: + case 0x39: + case 0x3A: + case 0x3B: + case 0x3C: + case 0x3D: + case 0x3E: + case 0x3F: + case 0x40: + case 0x41: + case 0x42: + case 0x43: + case 0x44: + case 0x45: + case 0x46: + case 0x47: + case 0x48: + case 0x49: + case 0x4A: + case 0x4B: + case 0x4C: + case 0x4D: + case 0x4E: + case 0x4F: + case 0x50: + case 0x51: + case 0x52: + case 0x53: + case 0x54: + case 0x55: + case 0x56: + case 0x57: + case 0x58: + case 0x59: + case 0x5A: + case 0x5B: + case 0x5C: + case 0x5D: + case 0x5E: + case 0x5F: + case 0x60: + case 0x61: + case 0x62: + case 0x63: + case 0x64: + case 0x65: + case 0x66: + case 0x67: + case 0x68: + case 0x69: + case 0x6A: + case 0x6B: + case 0x6C: + case 0x6D: + case 0x6E: + case 0x6F: + case 0x70: + case 0x71: + case 0x72: + case 0x73: + case 0x74: + case 0x75: + case 0x76: + case 0x77: + case 0x78: + case 0x79: + case 0x7A: + case 0x7B: + case 0x7C: + case 0x7D: + case 0x7E: + case 0x7F: + return sax->number_unsigned(static_cast(current)); + + // fixmap + case 0x80: + case 0x81: + case 0x82: + case 0x83: + case 0x84: + case 0x85: + case 0x86: + case 0x87: + case 0x88: + case 0x89: + case 0x8A: + case 0x8B: + case 0x8C: + case 0x8D: + case 0x8E: + case 0x8F: + return get_msgpack_object(static_cast(static_cast(current) & 0x0Fu)); + + // fixarray + case 0x90: + case 0x91: + case 0x92: + case 0x93: + case 0x94: + case 0x95: + case 0x96: + case 0x97: + case 0x98: + case 0x99: + case 0x9A: + case 0x9B: + case 0x9C: + case 0x9D: + case 0x9E: + case 0x9F: + return get_msgpack_array(static_cast(static_cast(current) & 0x0Fu)); + + // fixstr + case 0xA0: + case 0xA1: + case 0xA2: + case 0xA3: + case 0xA4: + case 0xA5: + case 0xA6: + case 0xA7: + case 0xA8: + case 0xA9: + case 0xAA: + case 0xAB: + case 0xAC: + case 0xAD: + case 0xAE: + case 0xAF: + case 0xB0: + case 0xB1: + case 0xB2: + case 0xB3: + case 0xB4: + case 0xB5: + case 0xB6: + case 0xB7: + case 0xB8: + case 0xB9: + case 0xBA: + case 0xBB: + case 0xBC: + case 0xBD: + case 0xBE: + case 0xBF: + case 0xD9: // str 8 + case 0xDA: // str 16 + case 0xDB: // str 32 + { + string_t s; + return get_msgpack_string(s) && sax->string(s); + } + + case 0xC0: // nil + return sax->null(); + + case 0xC2: // false + return sax->boolean(false); + + case 0xC3: // true + return sax->boolean(true); + + case 0xC4: // bin 8 + case 0xC5: // bin 16 + case 0xC6: // bin 32 + case 0xC7: // ext 8 + case 0xC8: // ext 16 + case 0xC9: // ext 32 + case 0xD4: // fixext 1 + case 0xD5: // fixext 2 + case 0xD6: // fixext 4 + case 0xD7: // fixext 8 + case 0xD8: // fixext 16 + { + binary_t b; + return get_msgpack_binary(b) && sax->binary(b); + } + + case 0xCA: // float 32 + { + float number{}; + return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast(number), ""); + } + + case 0xCB: // float 64 + { + double number{}; + return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast(number), ""); + } + + case 0xCC: // uint 8 + { + std::uint8_t number{}; + return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number); + } + + case 0xCD: // uint 16 + { + std::uint16_t number{}; + return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number); + } + + case 0xCE: // uint 32 + { + std::uint32_t number{}; + return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number); + } + + case 0xCF: // uint 64 + { + std::uint64_t number{}; + return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number); + } + + case 0xD0: // int 8 + { + std::int8_t number{}; + return get_number(input_format_t::msgpack, number) && sax->number_integer(number); + } + + case 0xD1: // int 16 + { + std::int16_t number{}; + return get_number(input_format_t::msgpack, number) && sax->number_integer(number); + } + + case 0xD2: // int 32 + { + std::int32_t number{}; + return get_number(input_format_t::msgpack, number) && sax->number_integer(number); + } + + case 0xD3: // int 64 + { + std::int64_t number{}; + return get_number(input_format_t::msgpack, number) && sax->number_integer(number); + } + + case 0xDC: // array 16 + { + std::uint16_t len{}; + return get_number(input_format_t::msgpack, len) && get_msgpack_array(static_cast(len)); + } + + case 0xDD: // array 32 + { + std::uint32_t len{}; + return get_number(input_format_t::msgpack, len) && get_msgpack_array(static_cast(len)); + } + + case 0xDE: // map 16 + { + std::uint16_t len{}; + return get_number(input_format_t::msgpack, len) && get_msgpack_object(static_cast(len)); + } + + case 0xDF: // map 32 + { + std::uint32_t len{}; + return get_number(input_format_t::msgpack, len) && get_msgpack_object(static_cast(len)); + } + + // negative fixint + case 0xE0: + case 0xE1: + case 0xE2: + case 0xE3: + case 0xE4: + case 0xE5: + case 0xE6: + case 0xE7: + case 0xE8: + case 0xE9: + case 0xEA: + case 0xEB: + case 0xEC: + case 0xED: + case 0xEE: + case 0xEF: + case 0xF0: + case 0xF1: + case 0xF2: + case 0xF3: + case 0xF4: + case 0xF5: + case 0xF6: + case 0xF7: + case 0xF8: + case 0xF9: + case 0xFA: + case 0xFB: + case 0xFC: + case 0xFD: + case 0xFE: + case 0xFF: + return sax->number_integer(static_cast(current)); + + default: // anything else + { + auto last_token = get_token_string(); + return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, exception_message(input_format_t::msgpack, "invalid byte: 0x" + last_token, "value"))); + } + } + } + + /*! + @brief reads a MessagePack string + + This function first reads starting bytes to determine the expected + string length and then copies this number of bytes into a string. + + @param[out] result created string + + @return whether string creation completed + */ + bool get_msgpack_string(string_t& result) + { + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::msgpack, "string"))) + { + return false; + } + + switch (current) + { + // fixstr + case 0xA0: + case 0xA1: + case 0xA2: + case 0xA3: + case 0xA4: + case 0xA5: + case 0xA6: + case 0xA7: + case 0xA8: + case 0xA9: + case 0xAA: + case 0xAB: + case 0xAC: + case 0xAD: + case 0xAE: + case 0xAF: + case 0xB0: + case 0xB1: + case 0xB2: + case 0xB3: + case 0xB4: + case 0xB5: + case 0xB6: + case 0xB7: + case 0xB8: + case 0xB9: + case 0xBA: + case 0xBB: + case 0xBC: + case 0xBD: + case 0xBE: + case 0xBF: + { + return get_string(input_format_t::msgpack, static_cast(current) & 0x1Fu, result); + } + + case 0xD9: // str 8 + { + std::uint8_t len{}; + return get_number(input_format_t::msgpack, len) && get_string(input_format_t::msgpack, len, result); + } + + case 0xDA: // str 16 + { + std::uint16_t len{}; + return get_number(input_format_t::msgpack, len) && get_string(input_format_t::msgpack, len, result); + } + + case 0xDB: // str 32 + { + std::uint32_t len{}; + return get_number(input_format_t::msgpack, len) && get_string(input_format_t::msgpack, len, result); + } + + default: + { + auto last_token = get_token_string(); + return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(input_format_t::msgpack, "expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0x" + last_token, "string"))); + } + } + } + + /*! + @brief reads a MessagePack byte array + + This function first reads starting bytes to determine the expected + byte array length and then copies this number of bytes into a byte array. + + @param[out] result created byte array + + @return whether byte array creation completed + */ + bool get_msgpack_binary(binary_t& result) + { + // helper function to set the subtype + auto assign_and_return_true = [&result](std::int8_t subtype) + { + result.set_subtype(static_cast(subtype)); + return true; + }; + + switch (current) + { + case 0xC4: // bin 8 + { + std::uint8_t len{}; + return get_number(input_format_t::msgpack, len) && + get_binary(input_format_t::msgpack, len, result); + } + + case 0xC5: // bin 16 + { + std::uint16_t len{}; + return get_number(input_format_t::msgpack, len) && + get_binary(input_format_t::msgpack, len, result); + } + + case 0xC6: // bin 32 + { + std::uint32_t len{}; + return get_number(input_format_t::msgpack, len) && + get_binary(input_format_t::msgpack, len, result); + } + + case 0xC7: // ext 8 + { + std::uint8_t len{}; + std::int8_t subtype{}; + return get_number(input_format_t::msgpack, len) && + get_number(input_format_t::msgpack, subtype) && + get_binary(input_format_t::msgpack, len, result) && + assign_and_return_true(subtype); + } + + case 0xC8: // ext 16 + { + std::uint16_t len{}; + std::int8_t subtype{}; + return get_number(input_format_t::msgpack, len) && + get_number(input_format_t::msgpack, subtype) && + get_binary(input_format_t::msgpack, len, result) && + assign_and_return_true(subtype); + } + + case 0xC9: // ext 32 + { + std::uint32_t len{}; + std::int8_t subtype{}; + return get_number(input_format_t::msgpack, len) && + get_number(input_format_t::msgpack, subtype) && + get_binary(input_format_t::msgpack, len, result) && + assign_and_return_true(subtype); + } + + case 0xD4: // fixext 1 + { + std::int8_t subtype{}; + return get_number(input_format_t::msgpack, subtype) && + get_binary(input_format_t::msgpack, 1, result) && + assign_and_return_true(subtype); + } + + case 0xD5: // fixext 2 + { + std::int8_t subtype{}; + return get_number(input_format_t::msgpack, subtype) && + get_binary(input_format_t::msgpack, 2, result) && + assign_and_return_true(subtype); + } + + case 0xD6: // fixext 4 + { + std::int8_t subtype{}; + return get_number(input_format_t::msgpack, subtype) && + get_binary(input_format_t::msgpack, 4, result) && + assign_and_return_true(subtype); + } + + case 0xD7: // fixext 8 + { + std::int8_t subtype{}; + return get_number(input_format_t::msgpack, subtype) && + get_binary(input_format_t::msgpack, 8, result) && + assign_and_return_true(subtype); + } + + case 0xD8: // fixext 16 + { + std::int8_t subtype{}; + return get_number(input_format_t::msgpack, subtype) && + get_binary(input_format_t::msgpack, 16, result) && + assign_and_return_true(subtype); + } + + default: // LCOV_EXCL_LINE + return false; // LCOV_EXCL_LINE + } + } + + /*! + @param[in] len the length of the array + @return whether array creation completed + */ + bool get_msgpack_array(const std::size_t len) + { + if (JSON_HEDLEY_UNLIKELY(!sax->start_array(len))) + { + return false; + } + + for (std::size_t i = 0; i < len; ++i) + { + if (JSON_HEDLEY_UNLIKELY(!parse_msgpack_internal())) + { + return false; + } + } + + return sax->end_array(); + } + + /*! + @param[in] len the length of the object + @return whether object creation completed + */ + bool get_msgpack_object(const std::size_t len) + { + if (JSON_HEDLEY_UNLIKELY(!sax->start_object(len))) + { + return false; + } + + string_t key; + for (std::size_t i = 0; i < len; ++i) + { + get(); + if (JSON_HEDLEY_UNLIKELY(!get_msgpack_string(key) || !sax->key(key))) + { + return false; + } + + if (JSON_HEDLEY_UNLIKELY(!parse_msgpack_internal())) + { + return false; + } + key.clear(); + } + + return sax->end_object(); + } + + //////////// + // UBJSON // + //////////// + + /*! + @param[in] get_char whether a new character should be retrieved from the + input (true, default) or whether the last read + character should be considered instead + + @return whether a valid UBJSON value was passed to the SAX parser + */ + bool parse_ubjson_internal(const bool get_char = true) + { + return get_ubjson_value(get_char ? get_ignore_noop() : current); + } + + /*! + @brief reads a UBJSON string + + This function is either called after reading the 'S' byte explicitly + indicating a string, or in case of an object key where the 'S' byte can be + left out. + + @param[out] result created string + @param[in] get_char whether a new character should be retrieved from the + input (true, default) or whether the last read + character should be considered instead + + @return whether string creation completed + */ + bool get_ubjson_string(string_t& result, const bool get_char = true) + { + if (get_char) + { + get(); // TODO(niels): may we ignore N here? + } + + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::ubjson, "value"))) + { + return false; + } + + switch (current) + { + case 'U': + { + std::uint8_t len{}; + return get_number(input_format_t::ubjson, len) && get_string(input_format_t::ubjson, len, result); + } + + case 'i': + { + std::int8_t len{}; + return get_number(input_format_t::ubjson, len) && get_string(input_format_t::ubjson, len, result); + } + + case 'I': + { + std::int16_t len{}; + return get_number(input_format_t::ubjson, len) && get_string(input_format_t::ubjson, len, result); + } + + case 'l': + { + std::int32_t len{}; + return get_number(input_format_t::ubjson, len) && get_string(input_format_t::ubjson, len, result); + } + + case 'L': + { + std::int64_t len{}; + return get_number(input_format_t::ubjson, len) && get_string(input_format_t::ubjson, len, result); + } + + default: + auto last_token = get_token_string(); + return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(input_format_t::ubjson, "expected length type specification (U, i, I, l, L); last byte: 0x" + last_token, "string"))); + } + } + + /*! + @param[out] result determined size + @return whether size determination completed + */ + bool get_ubjson_size_value(std::size_t& result) + { + switch (get_ignore_noop()) + { + case 'U': + { + std::uint8_t number{}; + if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::ubjson, number))) + { + return false; + } + result = static_cast(number); + return true; + } + + case 'i': + { + std::int8_t number{}; + if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::ubjson, number))) + { + return false; + } + result = static_cast(number); + return true; + } + + case 'I': + { + std::int16_t number{}; + if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::ubjson, number))) + { + return false; + } + result = static_cast(number); + return true; + } + + case 'l': + { + std::int32_t number{}; + if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::ubjson, number))) + { + return false; + } + result = static_cast(number); + return true; + } + + case 'L': + { + std::int64_t number{}; + if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::ubjson, number))) + { + return false; + } + result = static_cast(number); + return true; + } + + default: + { + auto last_token = get_token_string(); + return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(input_format_t::ubjson, "expected length type specification (U, i, I, l, L) after '#'; last byte: 0x" + last_token, "size"))); + } + } + } + + /*! + @brief determine the type and size for a container + + In the optimized UBJSON format, a type and a size can be provided to allow + for a more compact representation. + + @param[out] result pair of the size and the type + + @return whether pair creation completed + */ + bool get_ubjson_size_type(std::pair& result) + { + result.first = string_t::npos; // size + result.second = 0; // type + + get_ignore_noop(); + + if (current == '$') + { + result.second = get(); // must not ignore 'N', because 'N' maybe the type + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::ubjson, "type"))) + { + return false; + } + + get_ignore_noop(); + if (JSON_HEDLEY_UNLIKELY(current != '#')) + { + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::ubjson, "value"))) + { + return false; + } + auto last_token = get_token_string(); + return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, exception_message(input_format_t::ubjson, "expected '#' after type information; last byte: 0x" + last_token, "size"))); + } + + return get_ubjson_size_value(result.first); + } + + if (current == '#') + { + return get_ubjson_size_value(result.first); + } + + return true; + } + + /*! + @param prefix the previously read or set type prefix + @return whether value creation completed + */ + bool get_ubjson_value(const char_int_type prefix) + { + switch (prefix) + { + case std::char_traits::eof(): // EOF + return unexpect_eof(input_format_t::ubjson, "value"); + + case 'T': // true + return sax->boolean(true); + case 'F': // false + return sax->boolean(false); + + case 'Z': // null + return sax->null(); + + case 'U': + { + std::uint8_t number{}; + return get_number(input_format_t::ubjson, number) && sax->number_unsigned(number); + } + + case 'i': + { + std::int8_t number{}; + return get_number(input_format_t::ubjson, number) && sax->number_integer(number); + } + + case 'I': + { + std::int16_t number{}; + return get_number(input_format_t::ubjson, number) && sax->number_integer(number); + } + + case 'l': + { + std::int32_t number{}; + return get_number(input_format_t::ubjson, number) && sax->number_integer(number); + } + + case 'L': + { + std::int64_t number{}; + return get_number(input_format_t::ubjson, number) && sax->number_integer(number); + } + + case 'd': + { + float number{}; + return get_number(input_format_t::ubjson, number) && sax->number_float(static_cast(number), ""); + } + + case 'D': + { + double number{}; + return get_number(input_format_t::ubjson, number) && sax->number_float(static_cast(number), ""); + } + + case 'H': + { + return get_ubjson_high_precision_number(); + } + + case 'C': // char + { + get(); + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::ubjson, "char"))) + { + return false; + } + if (JSON_HEDLEY_UNLIKELY(current > 127)) + { + auto last_token = get_token_string(); + return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(input_format_t::ubjson, "byte after 'C' must be in range 0x00..0x7F; last byte: 0x" + last_token, "char"))); + } + string_t s(1, static_cast(current)); + return sax->string(s); + } + + case 'S': // string + { + string_t s; + return get_ubjson_string(s) && sax->string(s); + } + + case '[': // array + return get_ubjson_array(); + + case '{': // object + return get_ubjson_object(); + + default: // anything else + { + auto last_token = get_token_string(); + return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, exception_message(input_format_t::ubjson, "invalid byte: 0x" + last_token, "value"))); + } + } + } + + /*! + @return whether array creation completed + */ + bool get_ubjson_array() + { + std::pair size_and_type; + if (JSON_HEDLEY_UNLIKELY(!get_ubjson_size_type(size_and_type))) + { + return false; + } + + if (size_and_type.first != string_t::npos) + { + if (JSON_HEDLEY_UNLIKELY(!sax->start_array(size_and_type.first))) + { + return false; + } + + if (size_and_type.second != 0) + { + if (size_and_type.second != 'N') + { + for (std::size_t i = 0; i < size_and_type.first; ++i) + { + if (JSON_HEDLEY_UNLIKELY(!get_ubjson_value(size_and_type.second))) + { + return false; + } + } + } + } + else + { + for (std::size_t i = 0; i < size_and_type.first; ++i) + { + if (JSON_HEDLEY_UNLIKELY(!parse_ubjson_internal())) + { + return false; + } + } + } + } + else + { + if (JSON_HEDLEY_UNLIKELY(!sax->start_array(std::size_t(-1)))) + { + return false; + } + + while (current != ']') + { + if (JSON_HEDLEY_UNLIKELY(!parse_ubjson_internal(false))) + { + return false; + } + get_ignore_noop(); + } + } + + return sax->end_array(); + } + + /*! + @return whether object creation completed + */ + bool get_ubjson_object() + { + std::pair size_and_type; + if (JSON_HEDLEY_UNLIKELY(!get_ubjson_size_type(size_and_type))) + { + return false; + } + + string_t key; + if (size_and_type.first != string_t::npos) + { + if (JSON_HEDLEY_UNLIKELY(!sax->start_object(size_and_type.first))) + { + return false; + } + + if (size_and_type.second != 0) + { + for (std::size_t i = 0; i < size_and_type.first; ++i) + { + if (JSON_HEDLEY_UNLIKELY(!get_ubjson_string(key) || !sax->key(key))) + { + return false; + } + if (JSON_HEDLEY_UNLIKELY(!get_ubjson_value(size_and_type.second))) + { + return false; + } + key.clear(); + } + } + else + { + for (std::size_t i = 0; i < size_and_type.first; ++i) + { + if (JSON_HEDLEY_UNLIKELY(!get_ubjson_string(key) || !sax->key(key))) + { + return false; + } + if (JSON_HEDLEY_UNLIKELY(!parse_ubjson_internal())) + { + return false; + } + key.clear(); + } + } + } + else + { + if (JSON_HEDLEY_UNLIKELY(!sax->start_object(std::size_t(-1)))) + { + return false; + } + + while (current != '}') + { + if (JSON_HEDLEY_UNLIKELY(!get_ubjson_string(key, false) || !sax->key(key))) + { + return false; + } + if (JSON_HEDLEY_UNLIKELY(!parse_ubjson_internal())) + { + return false; + } + get_ignore_noop(); + key.clear(); + } + } + + return sax->end_object(); + } + + // Note, no reader for UBJSON binary types is implemented because they do + // not exist + + bool get_ubjson_high_precision_number() + { + // get size of following number string + std::size_t size{}; + auto res = get_ubjson_size_value(size); + if (JSON_HEDLEY_UNLIKELY(!res)) + { + return res; + } + + // get number string + std::vector number_vector; + for (std::size_t i = 0; i < size; ++i) + { + get(); + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::ubjson, "number"))) + { + return false; + } + number_vector.push_back(static_cast(current)); + } + + // parse number string + auto number_ia = detail::input_adapter(std::forward(number_vector)); + auto number_lexer = detail::lexer(std::move(number_ia), false); + const auto result_number = number_lexer.scan(); + const auto number_string = number_lexer.get_token_string(); + const auto result_remainder = number_lexer.scan(); + + using token_type = typename detail::lexer_base::token_type; + + if (JSON_HEDLEY_UNLIKELY(result_remainder != token_type::end_of_input)) + { + return sax->parse_error(chars_read, number_string, parse_error::create(115, chars_read, exception_message(input_format_t::ubjson, "invalid number text: " + number_lexer.get_token_string(), "high-precision number"))); + } + + switch (result_number) + { + case token_type::value_integer: + return sax->number_integer(number_lexer.get_number_integer()); + case token_type::value_unsigned: + return sax->number_unsigned(number_lexer.get_number_unsigned()); + case token_type::value_float: + return sax->number_float(number_lexer.get_number_float(), std::move(number_string)); + default: + return sax->parse_error(chars_read, number_string, parse_error::create(115, chars_read, exception_message(input_format_t::ubjson, "invalid number text: " + number_lexer.get_token_string(), "high-precision number"))); + } + } + + /////////////////////// + // Utility functions // + /////////////////////// + + /*! + @brief get next character from the input + + This function provides the interface to the used input adapter. It does + not throw in case the input reached EOF, but returns a -'ve valued + `std::char_traits::eof()` in that case. + + @return character read from the input + */ + char_int_type get() + { + ++chars_read; + return current = ia.get_character(); + } + + /*! + @return character read from the input after ignoring all 'N' entries + */ + char_int_type get_ignore_noop() + { + do + { + get(); + } + while (current == 'N'); + + return current; + } + + /* + @brief read a number from the input + + @tparam NumberType the type of the number + @param[in] format the current format (for diagnostics) + @param[out] result number of type @a NumberType + + @return whether conversion completed + + @note This function needs to respect the system's endianess, because + bytes in CBOR, MessagePack, and UBJSON are stored in network order + (big endian) and therefore need reordering on little endian systems. + */ + template + bool get_number(const input_format_t format, NumberType& result) + { + // step 1: read input into array with system's byte order + std::array vec; + for (std::size_t i = 0; i < sizeof(NumberType); ++i) + { + get(); + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(format, "number"))) + { + return false; + } + + // reverse byte order prior to conversion if necessary + if (is_little_endian != InputIsLittleEndian) + { + vec[sizeof(NumberType) - i - 1] = static_cast(current); + } + else + { + vec[i] = static_cast(current); // LCOV_EXCL_LINE + } + } + + // step 2: convert array into number of type T and return + std::memcpy(&result, vec.data(), sizeof(NumberType)); + return true; + } + + /*! + @brief create a string by reading characters from the input + + @tparam NumberType the type of the number + @param[in] format the current format (for diagnostics) + @param[in] len number of characters to read + @param[out] result string created by reading @a len bytes + + @return whether string creation completed + + @note We can not reserve @a len bytes for the result, because @a len + may be too large. Usually, @ref unexpect_eof() detects the end of + the input before we run out of string memory. + */ + template + bool get_string(const input_format_t format, + const NumberType len, + string_t& result) + { + bool success = true; + for (NumberType i = 0; i < len; i++) + { + get(); + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(format, "string"))) + { + success = false; + break; + } + result.push_back(static_cast(current)); + }; + return success; + } + + /*! + @brief create a byte array by reading bytes from the input + + @tparam NumberType the type of the number + @param[in] format the current format (for diagnostics) + @param[in] len number of bytes to read + @param[out] result byte array created by reading @a len bytes + + @return whether byte array creation completed + + @note We can not reserve @a len bytes for the result, because @a len + may be too large. Usually, @ref unexpect_eof() detects the end of + the input before we run out of memory. + */ + template + bool get_binary(const input_format_t format, + const NumberType len, + binary_t& result) + { + bool success = true; + for (NumberType i = 0; i < len; i++) + { + get(); + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(format, "binary"))) + { + success = false; + break; + } + result.push_back(static_cast(current)); + } + return success; + } + + /*! + @param[in] format the current format (for diagnostics) + @param[in] context further context information (for diagnostics) + @return whether the last read character is not EOF + */ + JSON_HEDLEY_NON_NULL(3) + bool unexpect_eof(const input_format_t format, const char* context) const + { + if (JSON_HEDLEY_UNLIKELY(current == std::char_traits::eof())) + { + return sax->parse_error(chars_read, "", + parse_error::create(110, chars_read, exception_message(format, "unexpected end of input", context))); + } + return true; + } + + /*! + @return a string representation of the last read byte + */ + std::string get_token_string() const + { + std::array cr{{}}; + (std::snprintf)(cr.data(), cr.size(), "%.2hhX", static_cast(current)); + return std::string{cr.data()}; + } + + /*! + @param[in] format the current format + @param[in] detail a detailed error message + @param[in] context further context information + @return a message string to use in the parse_error exceptions + */ + std::string exception_message(const input_format_t format, + const std::string& detail, + const std::string& context) const + { + std::string error_msg = "syntax error while parsing "; + + switch (format) + { + case input_format_t::cbor: + error_msg += "CBOR"; + break; + + case input_format_t::msgpack: + error_msg += "MessagePack"; + break; + + case input_format_t::ubjson: + error_msg += "UBJSON"; + break; + + case input_format_t::bson: + error_msg += "BSON"; + break; + + default: // LCOV_EXCL_LINE + JSON_ASSERT(false); // LCOV_EXCL_LINE + } + + return error_msg + " " + context + ": " + detail; + } + + private: + /// input adapter + InputAdapterType ia; + + /// the current character + char_int_type current = std::char_traits::eof(); + + /// the number of characters read + std::size_t chars_read = 0; + + /// whether we can assume little endianess + const bool is_little_endian = little_endianess(); + + /// the SAX parser + json_sax_t* sax = nullptr; +}; +} // namespace detail +} // namespace nlohmann + +// #include + +// #include + +// #include + + +#include // isfinite +#include // uint8_t +#include // function +#include // string +#include // move +#include // vector + +// #include + +// #include + +// #include + +// #include + +// #include + +// #include + +// #include + + +namespace nlohmann +{ +namespace detail +{ +//////////// +// parser // +//////////// + +enum class parse_event_t : uint8_t +{ + /// the parser read `{` and started to process a JSON object + object_start, + /// the parser read `}` and finished processing a JSON object + object_end, + /// the parser read `[` and started to process a JSON array + array_start, + /// the parser read `]` and finished processing a JSON array + array_end, + /// the parser read a key of a value in an object + key, + /// the parser finished reading a JSON value + value +}; + +template +using parser_callback_t = + std::function; + +/*! +@brief syntax analysis + +This class implements a recursive descent parser. +*/ +template +class parser +{ + using number_integer_t = typename BasicJsonType::number_integer_t; + using number_unsigned_t = typename BasicJsonType::number_unsigned_t; + using number_float_t = typename BasicJsonType::number_float_t; + using string_t = typename BasicJsonType::string_t; + using lexer_t = lexer; + using token_type = typename lexer_t::token_type; + + public: + /// a parser reading from an input adapter + explicit parser(InputAdapterType&& adapter, + const parser_callback_t cb = nullptr, + const bool allow_exceptions_ = true, + const bool skip_comments = false) + : callback(cb) + , m_lexer(std::move(adapter), skip_comments) + , allow_exceptions(allow_exceptions_) + { + // read first token + get_token(); + } + + /*! + @brief public parser interface + + @param[in] strict whether to expect the last token to be EOF + @param[in,out] result parsed JSON value + + @throw parse_error.101 in case of an unexpected token + @throw parse_error.102 if to_unicode fails or surrogate error + @throw parse_error.103 if to_unicode fails + */ + void parse(const bool strict, BasicJsonType& result) + { + if (callback) + { + json_sax_dom_callback_parser sdp(result, callback, allow_exceptions); + sax_parse_internal(&sdp); + result.assert_invariant(); + + // in strict mode, input must be completely read + if (strict && (get_token() != token_type::end_of_input)) + { + sdp.parse_error(m_lexer.get_position(), + m_lexer.get_token_string(), + parse_error::create(101, m_lexer.get_position(), + exception_message(token_type::end_of_input, "value"))); + } + + // in case of an error, return discarded value + if (sdp.is_errored()) + { + result = value_t::discarded; + return; + } + + // set top-level value to null if it was discarded by the callback + // function + if (result.is_discarded()) + { + result = nullptr; + } + } + else + { + json_sax_dom_parser sdp(result, allow_exceptions); + sax_parse_internal(&sdp); + result.assert_invariant(); + + // in strict mode, input must be completely read + if (strict && (get_token() != token_type::end_of_input)) + { + sdp.parse_error(m_lexer.get_position(), + m_lexer.get_token_string(), + parse_error::create(101, m_lexer.get_position(), + exception_message(token_type::end_of_input, "value"))); + } + + // in case of an error, return discarded value + if (sdp.is_errored()) + { + result = value_t::discarded; + return; + } + } + } + + /*! + @brief public accept interface + + @param[in] strict whether to expect the last token to be EOF + @return whether the input is a proper JSON text + */ + bool accept(const bool strict = true) + { + json_sax_acceptor sax_acceptor; + return sax_parse(&sax_acceptor, strict); + } + + template + JSON_HEDLEY_NON_NULL(2) + bool sax_parse(SAX* sax, const bool strict = true) + { + (void)detail::is_sax_static_asserts {}; + const bool result = sax_parse_internal(sax); + + // strict mode: next byte must be EOF + if (result && strict && (get_token() != token_type::end_of_input)) + { + return sax->parse_error(m_lexer.get_position(), + m_lexer.get_token_string(), + parse_error::create(101, m_lexer.get_position(), + exception_message(token_type::end_of_input, "value"))); + } + + return result; + } + + private: + template + JSON_HEDLEY_NON_NULL(2) + bool sax_parse_internal(SAX* sax) + { + // stack to remember the hierarchy of structured values we are parsing + // true = array; false = object + std::vector states; + // value to avoid a goto (see comment where set to true) + bool skip_to_state_evaluation = false; + + while (true) + { + if (!skip_to_state_evaluation) + { + // invariant: get_token() was called before each iteration + switch (last_token) + { + case token_type::begin_object: + { + if (JSON_HEDLEY_UNLIKELY(!sax->start_object(std::size_t(-1)))) + { + return false; + } + + // closing } -> we are done + if (get_token() == token_type::end_object) + { + if (JSON_HEDLEY_UNLIKELY(!sax->end_object())) + { + return false; + } + break; + } + + // parse key + if (JSON_HEDLEY_UNLIKELY(last_token != token_type::value_string)) + { + return sax->parse_error(m_lexer.get_position(), + m_lexer.get_token_string(), + parse_error::create(101, m_lexer.get_position(), + exception_message(token_type::value_string, "object key"))); + } + if (JSON_HEDLEY_UNLIKELY(!sax->key(m_lexer.get_string()))) + { + return false; + } + + // parse separator (:) + if (JSON_HEDLEY_UNLIKELY(get_token() != token_type::name_separator)) + { + return sax->parse_error(m_lexer.get_position(), + m_lexer.get_token_string(), + parse_error::create(101, m_lexer.get_position(), + exception_message(token_type::name_separator, "object separator"))); + } + + // remember we are now inside an object + states.push_back(false); + + // parse values + get_token(); + continue; + } + + case token_type::begin_array: + { + if (JSON_HEDLEY_UNLIKELY(!sax->start_array(std::size_t(-1)))) + { + return false; + } + + // closing ] -> we are done + if (get_token() == token_type::end_array) + { + if (JSON_HEDLEY_UNLIKELY(!sax->end_array())) + { + return false; + } + break; + } + + // remember we are now inside an array + states.push_back(true); + + // parse values (no need to call get_token) + continue; + } + + case token_type::value_float: + { + const auto res = m_lexer.get_number_float(); + + if (JSON_HEDLEY_UNLIKELY(!std::isfinite(res))) + { + return sax->parse_error(m_lexer.get_position(), + m_lexer.get_token_string(), + out_of_range::create(406, "number overflow parsing '" + m_lexer.get_token_string() + "'")); + } + + if (JSON_HEDLEY_UNLIKELY(!sax->number_float(res, m_lexer.get_string()))) + { + return false; + } + + break; + } + + case token_type::literal_false: + { + if (JSON_HEDLEY_UNLIKELY(!sax->boolean(false))) + { + return false; + } + break; + } + + case token_type::literal_null: + { + if (JSON_HEDLEY_UNLIKELY(!sax->null())) + { + return false; + } + break; + } + + case token_type::literal_true: + { + if (JSON_HEDLEY_UNLIKELY(!sax->boolean(true))) + { + return false; + } + break; + } + + case token_type::value_integer: + { + if (JSON_HEDLEY_UNLIKELY(!sax->number_integer(m_lexer.get_number_integer()))) + { + return false; + } + break; + } + + case token_type::value_string: + { + if (JSON_HEDLEY_UNLIKELY(!sax->string(m_lexer.get_string()))) + { + return false; + } + break; + } + + case token_type::value_unsigned: + { + if (JSON_HEDLEY_UNLIKELY(!sax->number_unsigned(m_lexer.get_number_unsigned()))) + { + return false; + } + break; + } + + case token_type::parse_error: + { + // using "uninitialized" to avoid "expected" message + return sax->parse_error(m_lexer.get_position(), + m_lexer.get_token_string(), + parse_error::create(101, m_lexer.get_position(), + exception_message(token_type::uninitialized, "value"))); + } + + default: // the last token was unexpected + { + return sax->parse_error(m_lexer.get_position(), + m_lexer.get_token_string(), + parse_error::create(101, m_lexer.get_position(), + exception_message(token_type::literal_or_value, "value"))); + } + } + } + else + { + skip_to_state_evaluation = false; + } + + // we reached this line after we successfully parsed a value + if (states.empty()) + { + // empty stack: we reached the end of the hierarchy: done + return true; + } + + if (states.back()) // array + { + // comma -> next value + if (get_token() == token_type::value_separator) + { + // parse a new value + get_token(); + continue; + } + + // closing ] + if (JSON_HEDLEY_LIKELY(last_token == token_type::end_array)) + { + if (JSON_HEDLEY_UNLIKELY(!sax->end_array())) + { + return false; + } + + // We are done with this array. Before we can parse a + // new value, we need to evaluate the new state first. + // By setting skip_to_state_evaluation to false, we + // are effectively jumping to the beginning of this if. + JSON_ASSERT(!states.empty()); + states.pop_back(); + skip_to_state_evaluation = true; + continue; + } + + return sax->parse_error(m_lexer.get_position(), + m_lexer.get_token_string(), + parse_error::create(101, m_lexer.get_position(), + exception_message(token_type::end_array, "array"))); + } + else // object + { + // comma -> next value + if (get_token() == token_type::value_separator) + { + // parse key + if (JSON_HEDLEY_UNLIKELY(get_token() != token_type::value_string)) + { + return sax->parse_error(m_lexer.get_position(), + m_lexer.get_token_string(), + parse_error::create(101, m_lexer.get_position(), + exception_message(token_type::value_string, "object key"))); + } + + if (JSON_HEDLEY_UNLIKELY(!sax->key(m_lexer.get_string()))) + { + return false; + } + + // parse separator (:) + if (JSON_HEDLEY_UNLIKELY(get_token() != token_type::name_separator)) + { + return sax->parse_error(m_lexer.get_position(), + m_lexer.get_token_string(), + parse_error::create(101, m_lexer.get_position(), + exception_message(token_type::name_separator, "object separator"))); + } + + // parse values + get_token(); + continue; + } + + // closing } + if (JSON_HEDLEY_LIKELY(last_token == token_type::end_object)) + { + if (JSON_HEDLEY_UNLIKELY(!sax->end_object())) + { + return false; + } + + // We are done with this object. Before we can parse a + // new value, we need to evaluate the new state first. + // By setting skip_to_state_evaluation to false, we + // are effectively jumping to the beginning of this if. + JSON_ASSERT(!states.empty()); + states.pop_back(); + skip_to_state_evaluation = true; + continue; + } + + return sax->parse_error(m_lexer.get_position(), + m_lexer.get_token_string(), + parse_error::create(101, m_lexer.get_position(), + exception_message(token_type::end_object, "object"))); + } + } + } + + /// get next token from lexer + token_type get_token() + { + return last_token = m_lexer.scan(); + } + + std::string exception_message(const token_type expected, const std::string& context) + { + std::string error_msg = "syntax error "; + + if (!context.empty()) + { + error_msg += "while parsing " + context + " "; + } + + error_msg += "- "; + + if (last_token == token_type::parse_error) + { + error_msg += std::string(m_lexer.get_error_message()) + "; last read: '" + + m_lexer.get_token_string() + "'"; + } + else + { + error_msg += "unexpected " + std::string(lexer_t::token_type_name(last_token)); + } + + if (expected != token_type::uninitialized) + { + error_msg += "; expected " + std::string(lexer_t::token_type_name(expected)); + } + + return error_msg; + } + + private: + /// callback function + const parser_callback_t callback = nullptr; + /// the type of the last read token + token_type last_token = token_type::uninitialized; + /// the lexer + lexer_t m_lexer; + /// whether to throw exceptions in case of errors + const bool allow_exceptions = true; +}; +} // namespace detail +} // namespace nlohmann + +// #include + + +// #include + + +#include // ptrdiff_t +#include // numeric_limits + +namespace nlohmann +{ +namespace detail +{ +/* +@brief an iterator for primitive JSON types + +This class models an iterator for primitive JSON types (boolean, number, +string). It's only purpose is to allow the iterator/const_iterator classes +to "iterate" over primitive values. Internally, the iterator is modeled by +a `difference_type` variable. Value begin_value (`0`) models the begin, +end_value (`1`) models past the end. +*/ +class primitive_iterator_t +{ + private: + using difference_type = std::ptrdiff_t; + static constexpr difference_type begin_value = 0; + static constexpr difference_type end_value = begin_value + 1; + + /// iterator as signed integer type + difference_type m_it = (std::numeric_limits::min)(); + + public: + constexpr difference_type get_value() const noexcept + { + return m_it; + } + + /// set iterator to a defined beginning + void set_begin() noexcept + { + m_it = begin_value; + } + + /// set iterator to a defined past the end + void set_end() noexcept + { + m_it = end_value; + } + + /// return whether the iterator can be dereferenced + constexpr bool is_begin() const noexcept + { + return m_it == begin_value; + } + + /// return whether the iterator is at end + constexpr bool is_end() const noexcept + { + return m_it == end_value; + } + + friend constexpr bool operator==(primitive_iterator_t lhs, primitive_iterator_t rhs) noexcept + { + return lhs.m_it == rhs.m_it; + } + + friend constexpr bool operator<(primitive_iterator_t lhs, primitive_iterator_t rhs) noexcept + { + return lhs.m_it < rhs.m_it; + } + + primitive_iterator_t operator+(difference_type n) noexcept + { + auto result = *this; + result += n; + return result; + } + + friend constexpr difference_type operator-(primitive_iterator_t lhs, primitive_iterator_t rhs) noexcept + { + return lhs.m_it - rhs.m_it; + } + + primitive_iterator_t& operator++() noexcept + { + ++m_it; + return *this; + } + + primitive_iterator_t const operator++(int) noexcept + { + auto result = *this; + ++m_it; + return result; + } + + primitive_iterator_t& operator--() noexcept + { + --m_it; + return *this; + } + + primitive_iterator_t const operator--(int) noexcept + { + auto result = *this; + --m_it; + return result; + } + + primitive_iterator_t& operator+=(difference_type n) noexcept + { + m_it += n; + return *this; + } + + primitive_iterator_t& operator-=(difference_type n) noexcept + { + m_it -= n; + return *this; + } +}; +} // namespace detail +} // namespace nlohmann + + +namespace nlohmann +{ +namespace detail +{ +/*! +@brief an iterator value + +@note This structure could easily be a union, but MSVC currently does not allow +unions members with complex constructors, see https://github.com/nlohmann/json/pull/105. +*/ +template struct internal_iterator +{ + /// iterator for JSON objects + typename BasicJsonType::object_t::iterator object_iterator {}; + /// iterator for JSON arrays + typename BasicJsonType::array_t::iterator array_iterator {}; + /// generic iterator for all other types + primitive_iterator_t primitive_iterator {}; +}; +} // namespace detail +} // namespace nlohmann + +// #include + + +#include // iterator, random_access_iterator_tag, bidirectional_iterator_tag, advance, next +#include // conditional, is_const, remove_const + +// #include + +// #include + +// #include + +// #include + +// #include + +// #include + +// #include + + +namespace nlohmann +{ +namespace detail +{ +// forward declare, to be able to friend it later on +template class iteration_proxy; +template class iteration_proxy_value; + +/*! +@brief a template for a bidirectional iterator for the @ref basic_json class +This class implements a both iterators (iterator and const_iterator) for the +@ref basic_json class. +@note An iterator is called *initialized* when a pointer to a JSON value has + been set (e.g., by a constructor or a copy assignment). If the iterator is + default-constructed, it is *uninitialized* and most methods are undefined. + **The library uses assertions to detect calls on uninitialized iterators.** +@requirement The class satisfies the following concept requirements: +- +[BidirectionalIterator](https://en.cppreference.com/w/cpp/named_req/BidirectionalIterator): + The iterator that can be moved can be moved in both directions (i.e. + incremented and decremented). +@since version 1.0.0, simplified in version 2.0.9, change to bidirectional + iterators in version 3.0.0 (see https://github.com/nlohmann/json/issues/593) +*/ +template +class iter_impl +{ + /// allow basic_json to access private members + friend iter_impl::value, typename std::remove_const::type, const BasicJsonType>::type>; + friend BasicJsonType; + friend iteration_proxy; + friend iteration_proxy_value; + + using object_t = typename BasicJsonType::object_t; + using array_t = typename BasicJsonType::array_t; + // make sure BasicJsonType is basic_json or const basic_json + static_assert(is_basic_json::type>::value, + "iter_impl only accepts (const) basic_json"); + + public: + + /// The std::iterator class template (used as a base class to provide typedefs) is deprecated in C++17. + /// The C++ Standard has never required user-defined iterators to derive from std::iterator. + /// A user-defined iterator should provide publicly accessible typedefs named + /// iterator_category, value_type, difference_type, pointer, and reference. + /// Note that value_type is required to be non-const, even for constant iterators. + using iterator_category = std::bidirectional_iterator_tag; + + /// the type of the values when the iterator is dereferenced + using value_type = typename BasicJsonType::value_type; + /// a type to represent differences between iterators + using difference_type = typename BasicJsonType::difference_type; + /// defines a pointer to the type iterated over (value_type) + using pointer = typename std::conditional::value, + typename BasicJsonType::const_pointer, + typename BasicJsonType::pointer>::type; + /// defines a reference to the type iterated over (value_type) + using reference = + typename std::conditional::value, + typename BasicJsonType::const_reference, + typename BasicJsonType::reference>::type; + + /// default constructor + iter_impl() = default; + + /*! + @brief constructor for a given JSON instance + @param[in] object pointer to a JSON object for this iterator + @pre object != nullptr + @post The iterator is initialized; i.e. `m_object != nullptr`. + */ + explicit iter_impl(pointer object) noexcept : m_object(object) + { + JSON_ASSERT(m_object != nullptr); + + switch (m_object->m_type) + { + case value_t::object: + { + m_it.object_iterator = typename object_t::iterator(); + break; + } + + case value_t::array: + { + m_it.array_iterator = typename array_t::iterator(); + break; + } + + default: + { + m_it.primitive_iterator = primitive_iterator_t(); + break; + } + } + } + + /*! + @note The conventional copy constructor and copy assignment are implicitly + defined. Combined with the following converting constructor and + assignment, they support: (1) copy from iterator to iterator, (2) + copy from const iterator to const iterator, and (3) conversion from + iterator to const iterator. However conversion from const iterator + to iterator is not defined. + */ + + /*! + @brief const copy constructor + @param[in] other const iterator to copy from + @note This copy constructor had to be defined explicitly to circumvent a bug + occurring on msvc v19.0 compiler (VS 2015) debug build. For more + information refer to: https://github.com/nlohmann/json/issues/1608 + */ + iter_impl(const iter_impl& other) noexcept + : m_object(other.m_object), m_it(other.m_it) + {} + + /*! + @brief converting assignment + @param[in] other const iterator to copy from + @return const/non-const iterator + @note It is not checked whether @a other is initialized. + */ + iter_impl& operator=(const iter_impl& other) noexcept + { + m_object = other.m_object; + m_it = other.m_it; + return *this; + } + + /*! + @brief converting constructor + @param[in] other non-const iterator to copy from + @note It is not checked whether @a other is initialized. + */ + iter_impl(const iter_impl::type>& other) noexcept + : m_object(other.m_object), m_it(other.m_it) + {} + + /*! + @brief converting assignment + @param[in] other non-const iterator to copy from + @return const/non-const iterator + @note It is not checked whether @a other is initialized. + */ + iter_impl& operator=(const iter_impl::type>& other) noexcept + { + m_object = other.m_object; + m_it = other.m_it; + return *this; + } + + private: + /*! + @brief set the iterator to the first value + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + void set_begin() noexcept + { + JSON_ASSERT(m_object != nullptr); + + switch (m_object->m_type) + { + case value_t::object: + { + m_it.object_iterator = m_object->m_value.object->begin(); + break; + } + + case value_t::array: + { + m_it.array_iterator = m_object->m_value.array->begin(); + break; + } + + case value_t::null: + { + // set to end so begin()==end() is true: null is empty + m_it.primitive_iterator.set_end(); + break; + } + + default: + { + m_it.primitive_iterator.set_begin(); + break; + } + } + } + + /*! + @brief set the iterator past the last value + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + void set_end() noexcept + { + JSON_ASSERT(m_object != nullptr); + + switch (m_object->m_type) + { + case value_t::object: + { + m_it.object_iterator = m_object->m_value.object->end(); + break; + } + + case value_t::array: + { + m_it.array_iterator = m_object->m_value.array->end(); + break; + } + + default: + { + m_it.primitive_iterator.set_end(); + break; + } + } + } + + public: + /*! + @brief return a reference to the value pointed to by the iterator + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + reference operator*() const + { + JSON_ASSERT(m_object != nullptr); + + switch (m_object->m_type) + { + case value_t::object: + { + JSON_ASSERT(m_it.object_iterator != m_object->m_value.object->end()); + return m_it.object_iterator->second; + } + + case value_t::array: + { + JSON_ASSERT(m_it.array_iterator != m_object->m_value.array->end()); + return *m_it.array_iterator; + } + + case value_t::null: + JSON_THROW(invalid_iterator::create(214, "cannot get value")); + + default: + { + if (JSON_HEDLEY_LIKELY(m_it.primitive_iterator.is_begin())) + { + return *m_object; + } + + JSON_THROW(invalid_iterator::create(214, "cannot get value")); + } + } + } + + /*! + @brief dereference the iterator + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + pointer operator->() const + { + JSON_ASSERT(m_object != nullptr); + + switch (m_object->m_type) + { + case value_t::object: + { + JSON_ASSERT(m_it.object_iterator != m_object->m_value.object->end()); + return &(m_it.object_iterator->second); + } + + case value_t::array: + { + JSON_ASSERT(m_it.array_iterator != m_object->m_value.array->end()); + return &*m_it.array_iterator; + } + + default: + { + if (JSON_HEDLEY_LIKELY(m_it.primitive_iterator.is_begin())) + { + return m_object; + } + + JSON_THROW(invalid_iterator::create(214, "cannot get value")); + } + } + } + + /*! + @brief post-increment (it++) + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + iter_impl const operator++(int) + { + auto result = *this; + ++(*this); + return result; + } + + /*! + @brief pre-increment (++it) + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + iter_impl& operator++() + { + JSON_ASSERT(m_object != nullptr); + + switch (m_object->m_type) + { + case value_t::object: + { + std::advance(m_it.object_iterator, 1); + break; + } + + case value_t::array: + { + std::advance(m_it.array_iterator, 1); + break; + } + + default: + { + ++m_it.primitive_iterator; + break; + } + } + + return *this; + } + + /*! + @brief post-decrement (it--) + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + iter_impl const operator--(int) + { + auto result = *this; + --(*this); + return result; + } + + /*! + @brief pre-decrement (--it) + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + iter_impl& operator--() + { + JSON_ASSERT(m_object != nullptr); + + switch (m_object->m_type) + { + case value_t::object: + { + std::advance(m_it.object_iterator, -1); + break; + } + + case value_t::array: + { + std::advance(m_it.array_iterator, -1); + break; + } + + default: + { + --m_it.primitive_iterator; + break; + } + } + + return *this; + } + + /*! + @brief comparison: equal + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + bool operator==(const iter_impl& other) const + { + // if objects are not the same, the comparison is undefined + if (JSON_HEDLEY_UNLIKELY(m_object != other.m_object)) + { + JSON_THROW(invalid_iterator::create(212, "cannot compare iterators of different containers")); + } + + JSON_ASSERT(m_object != nullptr); + + switch (m_object->m_type) + { + case value_t::object: + return (m_it.object_iterator == other.m_it.object_iterator); + + case value_t::array: + return (m_it.array_iterator == other.m_it.array_iterator); + + default: + return (m_it.primitive_iterator == other.m_it.primitive_iterator); + } + } + + /*! + @brief comparison: not equal + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + bool operator!=(const iter_impl& other) const + { + return !operator==(other); + } + + /*! + @brief comparison: smaller + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + bool operator<(const iter_impl& other) const + { + // if objects are not the same, the comparison is undefined + if (JSON_HEDLEY_UNLIKELY(m_object != other.m_object)) + { + JSON_THROW(invalid_iterator::create(212, "cannot compare iterators of different containers")); + } + + JSON_ASSERT(m_object != nullptr); + + switch (m_object->m_type) + { + case value_t::object: + JSON_THROW(invalid_iterator::create(213, "cannot compare order of object iterators")); + + case value_t::array: + return (m_it.array_iterator < other.m_it.array_iterator); + + default: + return (m_it.primitive_iterator < other.m_it.primitive_iterator); + } + } + + /*! + @brief comparison: less than or equal + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + bool operator<=(const iter_impl& other) const + { + return !other.operator < (*this); + } + + /*! + @brief comparison: greater than + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + bool operator>(const iter_impl& other) const + { + return !operator<=(other); + } + + /*! + @brief comparison: greater than or equal + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + bool operator>=(const iter_impl& other) const + { + return !operator<(other); + } + + /*! + @brief add to iterator + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + iter_impl& operator+=(difference_type i) + { + JSON_ASSERT(m_object != nullptr); + + switch (m_object->m_type) + { + case value_t::object: + JSON_THROW(invalid_iterator::create(209, "cannot use offsets with object iterators")); + + case value_t::array: + { + std::advance(m_it.array_iterator, i); + break; + } + + default: + { + m_it.primitive_iterator += i; + break; + } + } + + return *this; + } + + /*! + @brief subtract from iterator + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + iter_impl& operator-=(difference_type i) + { + return operator+=(-i); + } + + /*! + @brief add to iterator + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + iter_impl operator+(difference_type i) const + { + auto result = *this; + result += i; + return result; + } + + /*! + @brief addition of distance and iterator + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + friend iter_impl operator+(difference_type i, const iter_impl& it) + { + auto result = it; + result += i; + return result; + } + + /*! + @brief subtract from iterator + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + iter_impl operator-(difference_type i) const + { + auto result = *this; + result -= i; + return result; + } + + /*! + @brief return difference + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + difference_type operator-(const iter_impl& other) const + { + JSON_ASSERT(m_object != nullptr); + + switch (m_object->m_type) + { + case value_t::object: + JSON_THROW(invalid_iterator::create(209, "cannot use offsets with object iterators")); + + case value_t::array: + return m_it.array_iterator - other.m_it.array_iterator; + + default: + return m_it.primitive_iterator - other.m_it.primitive_iterator; + } + } + + /*! + @brief access to successor + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + reference operator[](difference_type n) const + { + JSON_ASSERT(m_object != nullptr); + + switch (m_object->m_type) + { + case value_t::object: + JSON_THROW(invalid_iterator::create(208, "cannot use operator[] for object iterators")); + + case value_t::array: + return *std::next(m_it.array_iterator, n); + + case value_t::null: + JSON_THROW(invalid_iterator::create(214, "cannot get value")); + + default: + { + if (JSON_HEDLEY_LIKELY(m_it.primitive_iterator.get_value() == -n)) + { + return *m_object; + } + + JSON_THROW(invalid_iterator::create(214, "cannot get value")); + } + } + } + + /*! + @brief return the key of an object iterator + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + const typename object_t::key_type& key() const + { + JSON_ASSERT(m_object != nullptr); + + if (JSON_HEDLEY_LIKELY(m_object->is_object())) + { + return m_it.object_iterator->first; + } + + JSON_THROW(invalid_iterator::create(207, "cannot use key() for non-object iterators")); + } + + /*! + @brief return the value of an iterator + @pre The iterator is initialized; i.e. `m_object != nullptr`. + */ + reference value() const + { + return operator*(); + } + + private: + /// associated JSON instance + pointer m_object = nullptr; + /// the actual iterator of the associated instance + internal_iterator::type> m_it {}; +}; +} // namespace detail +} // namespace nlohmann + +// #include + +// #include + + +#include // ptrdiff_t +#include // reverse_iterator +#include // declval + +namespace nlohmann +{ +namespace detail +{ +////////////////////// +// reverse_iterator // +////////////////////// + +/*! +@brief a template for a reverse iterator class + +@tparam Base the base iterator type to reverse. Valid types are @ref +iterator (to create @ref reverse_iterator) and @ref const_iterator (to +create @ref const_reverse_iterator). + +@requirement The class satisfies the following concept requirements: +- +[BidirectionalIterator](https://en.cppreference.com/w/cpp/named_req/BidirectionalIterator): + The iterator that can be moved can be moved in both directions (i.e. + incremented and decremented). +- [OutputIterator](https://en.cppreference.com/w/cpp/named_req/OutputIterator): + It is possible to write to the pointed-to element (only if @a Base is + @ref iterator). + +@since version 1.0.0 +*/ +template +class json_reverse_iterator : public std::reverse_iterator +{ + public: + using difference_type = std::ptrdiff_t; + /// shortcut to the reverse iterator adapter + using base_iterator = std::reverse_iterator; + /// the reference type for the pointed-to element + using reference = typename Base::reference; + + /// create reverse iterator from iterator + explicit json_reverse_iterator(const typename base_iterator::iterator_type& it) noexcept + : base_iterator(it) {} + + /// create reverse iterator from base class + explicit json_reverse_iterator(const base_iterator& it) noexcept : base_iterator(it) {} + + /// post-increment (it++) + json_reverse_iterator const operator++(int) + { + return static_cast(base_iterator::operator++(1)); + } + + /// pre-increment (++it) + json_reverse_iterator& operator++() + { + return static_cast(base_iterator::operator++()); + } + + /// post-decrement (it--) + json_reverse_iterator const operator--(int) + { + return static_cast(base_iterator::operator--(1)); + } + + /// pre-decrement (--it) + json_reverse_iterator& operator--() + { + return static_cast(base_iterator::operator--()); + } + + /// add to iterator + json_reverse_iterator& operator+=(difference_type i) + { + return static_cast(base_iterator::operator+=(i)); + } + + /// add to iterator + json_reverse_iterator operator+(difference_type i) const + { + return static_cast(base_iterator::operator+(i)); + } + + /// subtract from iterator + json_reverse_iterator operator-(difference_type i) const + { + return static_cast(base_iterator::operator-(i)); + } + + /// return difference + difference_type operator-(const json_reverse_iterator& other) const + { + return base_iterator(*this) - base_iterator(other); + } + + /// access to successor + reference operator[](difference_type n) const + { + return *(this->operator+(n)); + } + + /// return the key of an object iterator + auto key() const -> decltype(std::declval().key()) + { + auto it = --this->base(); + return it.key(); + } + + /// return the value of an iterator + reference value() const + { + auto it = --this->base(); + return it.operator * (); + } +}; +} // namespace detail +} // namespace nlohmann + +// #include + +// #include + + +#include // all_of +#include // isdigit +#include // max +#include // accumulate +#include // string +#include // move +#include // vector + +// #include + +// #include + +// #include + + +namespace nlohmann +{ +template +class json_pointer +{ + // allow basic_json to access private members + NLOHMANN_BASIC_JSON_TPL_DECLARATION + friend class basic_json; + + public: + /*! + @brief create JSON pointer + + Create a JSON pointer according to the syntax described in + [Section 3 of RFC6901](https://tools.ietf.org/html/rfc6901#section-3). + + @param[in] s string representing the JSON pointer; if omitted, the empty + string is assumed which references the whole JSON value + + @throw parse_error.107 if the given JSON pointer @a s is nonempty and does + not begin with a slash (`/`); see example below + + @throw parse_error.108 if a tilde (`~`) in the given JSON pointer @a s is + not followed by `0` (representing `~`) or `1` (representing `/`); see + example below + + @liveexample{The example shows the construction several valid JSON pointers + as well as the exceptional behavior.,json_pointer} + + @since version 2.0.0 + */ + explicit json_pointer(const std::string& s = "") + : reference_tokens(split(s)) + {} + + /*! + @brief return a string representation of the JSON pointer + + @invariant For each JSON pointer `ptr`, it holds: + @code {.cpp} + ptr == json_pointer(ptr.to_string()); + @endcode + + @return a string representation of the JSON pointer + + @liveexample{The example shows the result of `to_string`.,json_pointer__to_string} + + @since version 2.0.0 + */ + std::string to_string() const + { + return std::accumulate(reference_tokens.begin(), reference_tokens.end(), + std::string{}, + [](const std::string & a, const std::string & b) + { + return a + "/" + escape(b); + }); + } + + /// @copydoc to_string() + operator std::string() const + { + return to_string(); + } + + /*! + @brief append another JSON pointer at the end of this JSON pointer + + @param[in] ptr JSON pointer to append + @return JSON pointer with @a ptr appended + + @liveexample{The example shows the usage of `operator/=`.,json_pointer__operator_add} + + @complexity Linear in the length of @a ptr. + + @sa @ref operator/=(std::string) to append a reference token + @sa @ref operator/=(std::size_t) to append an array index + @sa @ref operator/(const json_pointer&, const json_pointer&) for a binary operator + + @since version 3.6.0 + */ + json_pointer& operator/=(const json_pointer& ptr) + { + reference_tokens.insert(reference_tokens.end(), + ptr.reference_tokens.begin(), + ptr.reference_tokens.end()); + return *this; + } + + /*! + @brief append an unescaped reference token at the end of this JSON pointer + + @param[in] token reference token to append + @return JSON pointer with @a token appended without escaping @a token + + @liveexample{The example shows the usage of `operator/=`.,json_pointer__operator_add} + + @complexity Amortized constant. + + @sa @ref operator/=(const json_pointer&) to append a JSON pointer + @sa @ref operator/=(std::size_t) to append an array index + @sa @ref operator/(const json_pointer&, std::size_t) for a binary operator + + @since version 3.6.0 + */ + json_pointer& operator/=(std::string token) + { + push_back(std::move(token)); + return *this; + } + + /*! + @brief append an array index at the end of this JSON pointer + + @param[in] array_idx array index to append + @return JSON pointer with @a array_idx appended + + @liveexample{The example shows the usage of `operator/=`.,json_pointer__operator_add} + + @complexity Amortized constant. + + @sa @ref operator/=(const json_pointer&) to append a JSON pointer + @sa @ref operator/=(std::string) to append a reference token + @sa @ref operator/(const json_pointer&, std::string) for a binary operator + + @since version 3.6.0 + */ + json_pointer& operator/=(std::size_t array_idx) + { + return *this /= std::to_string(array_idx); + } + + /*! + @brief create a new JSON pointer by appending the right JSON pointer at the end of the left JSON pointer + + @param[in] lhs JSON pointer + @param[in] rhs JSON pointer + @return a new JSON pointer with @a rhs appended to @a lhs + + @liveexample{The example shows the usage of `operator/`.,json_pointer__operator_add_binary} + + @complexity Linear in the length of @a lhs and @a rhs. + + @sa @ref operator/=(const json_pointer&) to append a JSON pointer + + @since version 3.6.0 + */ + friend json_pointer operator/(const json_pointer& lhs, + const json_pointer& rhs) + { + return json_pointer(lhs) /= rhs; + } + + /*! + @brief create a new JSON pointer by appending the unescaped token at the end of the JSON pointer + + @param[in] ptr JSON pointer + @param[in] token reference token + @return a new JSON pointer with unescaped @a token appended to @a ptr + + @liveexample{The example shows the usage of `operator/`.,json_pointer__operator_add_binary} + + @complexity Linear in the length of @a ptr. + + @sa @ref operator/=(std::string) to append a reference token + + @since version 3.6.0 + */ + friend json_pointer operator/(const json_pointer& ptr, std::string token) + { + return json_pointer(ptr) /= std::move(token); + } + + /*! + @brief create a new JSON pointer by appending the array-index-token at the end of the JSON pointer + + @param[in] ptr JSON pointer + @param[in] array_idx array index + @return a new JSON pointer with @a array_idx appended to @a ptr + + @liveexample{The example shows the usage of `operator/`.,json_pointer__operator_add_binary} + + @complexity Linear in the length of @a ptr. + + @sa @ref operator/=(std::size_t) to append an array index + + @since version 3.6.0 + */ + friend json_pointer operator/(const json_pointer& ptr, std::size_t array_idx) + { + return json_pointer(ptr) /= array_idx; + } + + /*! + @brief returns the parent of this JSON pointer + + @return parent of this JSON pointer; in case this JSON pointer is the root, + the root itself is returned + + @complexity Linear in the length of the JSON pointer. + + @liveexample{The example shows the result of `parent_pointer` for different + JSON Pointers.,json_pointer__parent_pointer} + + @since version 3.6.0 + */ + json_pointer parent_pointer() const + { + if (empty()) + { + return *this; + } + + json_pointer res = *this; + res.pop_back(); + return res; + } + + /*! + @brief remove last reference token + + @pre not `empty()` + + @liveexample{The example shows the usage of `pop_back`.,json_pointer__pop_back} + + @complexity Constant. + + @throw out_of_range.405 if JSON pointer has no parent + + @since version 3.6.0 + */ + void pop_back() + { + if (JSON_HEDLEY_UNLIKELY(empty())) + { + JSON_THROW(detail::out_of_range::create(405, "JSON pointer has no parent")); + } + + reference_tokens.pop_back(); + } + + /*! + @brief return last reference token + + @pre not `empty()` + @return last reference token + + @liveexample{The example shows the usage of `back`.,json_pointer__back} + + @complexity Constant. + + @throw out_of_range.405 if JSON pointer has no parent + + @since version 3.6.0 + */ + const std::string& back() const + { + if (JSON_HEDLEY_UNLIKELY(empty())) + { + JSON_THROW(detail::out_of_range::create(405, "JSON pointer has no parent")); + } + + return reference_tokens.back(); + } + + /*! + @brief append an unescaped token at the end of the reference pointer + + @param[in] token token to add + + @complexity Amortized constant. + + @liveexample{The example shows the result of `push_back` for different + JSON Pointers.,json_pointer__push_back} + + @since version 3.6.0 + */ + void push_back(const std::string& token) + { + reference_tokens.push_back(token); + } + + /// @copydoc push_back(const std::string&) + void push_back(std::string&& token) + { + reference_tokens.push_back(std::move(token)); + } + + /*! + @brief return whether pointer points to the root document + + @return true iff the JSON pointer points to the root document + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this function never throws exceptions. + + @liveexample{The example shows the result of `empty` for different JSON + Pointers.,json_pointer__empty} + + @since version 3.6.0 + */ + bool empty() const noexcept + { + return reference_tokens.empty(); + } + + private: + /*! + @param[in] s reference token to be converted into an array index + + @return integer representation of @a s + + @throw parse_error.106 if an array index begins with '0' + @throw parse_error.109 if an array index begins not with a digit + @throw out_of_range.404 if string @a s could not be converted to an integer + @throw out_of_range.410 if an array index exceeds size_type + */ + static typename BasicJsonType::size_type array_index(const std::string& s) + { + using size_type = typename BasicJsonType::size_type; + + // error condition (cf. RFC 6901, Sect. 4) + if (JSON_HEDLEY_UNLIKELY(s.size() > 1 && s[0] == '0')) + { + JSON_THROW(detail::parse_error::create(106, 0, + "array index '" + s + + "' must not begin with '0'")); + } + + // error condition (cf. RFC 6901, Sect. 4) + if (JSON_HEDLEY_UNLIKELY(s.size() > 1 && !(s[0] >= '1' && s[0] <= '9'))) + { + JSON_THROW(detail::parse_error::create(109, 0, "array index '" + s + "' is not a number")); + } + + std::size_t processed_chars = 0; + unsigned long long res = 0; + JSON_TRY + { + res = std::stoull(s, &processed_chars); + } + JSON_CATCH(std::out_of_range&) + { + JSON_THROW(detail::out_of_range::create(404, "unresolved reference token '" + s + "'")); + } + + // check if the string was completely read + if (JSON_HEDLEY_UNLIKELY(processed_chars != s.size())) + { + JSON_THROW(detail::out_of_range::create(404, "unresolved reference token '" + s + "'")); + } + + // only triggered on special platforms (like 32bit), see also + // https://github.com/nlohmann/json/pull/2203 + if (res >= static_cast((std::numeric_limits::max)())) + { + JSON_THROW(detail::out_of_range::create(410, "array index " + s + " exceeds size_type")); // LCOV_EXCL_LINE + } + + return static_cast(res); + } + + json_pointer top() const + { + if (JSON_HEDLEY_UNLIKELY(empty())) + { + JSON_THROW(detail::out_of_range::create(405, "JSON pointer has no parent")); + } + + json_pointer result = *this; + result.reference_tokens = {reference_tokens[0]}; + return result; + } + + /*! + @brief create and return a reference to the pointed to value + + @complexity Linear in the number of reference tokens. + + @throw parse_error.109 if array index is not a number + @throw type_error.313 if value cannot be unflattened + */ + BasicJsonType& get_and_create(BasicJsonType& j) const + { + auto result = &j; + + // in case no reference tokens exist, return a reference to the JSON value + // j which will be overwritten by a primitive value + for (const auto& reference_token : reference_tokens) + { + switch (result->type()) + { + case detail::value_t::null: + { + if (reference_token == "0") + { + // start a new array if reference token is 0 + result = &result->operator[](0); + } + else + { + // start a new object otherwise + result = &result->operator[](reference_token); + } + break; + } + + case detail::value_t::object: + { + // create an entry in the object + result = &result->operator[](reference_token); + break; + } + + case detail::value_t::array: + { + // create an entry in the array + result = &result->operator[](array_index(reference_token)); + break; + } + + /* + The following code is only reached if there exists a reference + token _and_ the current value is primitive. In this case, we have + an error situation, because primitive values may only occur as + single value; that is, with an empty list of reference tokens. + */ + default: + JSON_THROW(detail::type_error::create(313, "invalid value to unflatten")); + } + } + + return *result; + } + + /*! + @brief return a reference to the pointed to value + + @note This version does not throw if a value is not present, but tries to + create nested values instead. For instance, calling this function + with pointer `"/this/that"` on a null value is equivalent to calling + `operator[]("this").operator[]("that")` on that value, effectively + changing the null value to an object. + + @param[in] ptr a JSON value + + @return reference to the JSON value pointed to by the JSON pointer + + @complexity Linear in the length of the JSON pointer. + + @throw parse_error.106 if an array index begins with '0' + @throw parse_error.109 if an array index was not a number + @throw out_of_range.404 if the JSON pointer can not be resolved + */ + BasicJsonType& get_unchecked(BasicJsonType* ptr) const + { + for (const auto& reference_token : reference_tokens) + { + // convert null values to arrays or objects before continuing + if (ptr->is_null()) + { + // check if reference token is a number + const bool nums = + std::all_of(reference_token.begin(), reference_token.end(), + [](const unsigned char x) + { + return std::isdigit(x); + }); + + // change value to array for numbers or "-" or to object otherwise + *ptr = (nums || reference_token == "-") + ? detail::value_t::array + : detail::value_t::object; + } + + switch (ptr->type()) + { + case detail::value_t::object: + { + // use unchecked object access + ptr = &ptr->operator[](reference_token); + break; + } + + case detail::value_t::array: + { + if (reference_token == "-") + { + // explicitly treat "-" as index beyond the end + ptr = &ptr->operator[](ptr->m_value.array->size()); + } + else + { + // convert array index to number; unchecked access + ptr = &ptr->operator[](array_index(reference_token)); + } + break; + } + + default: + JSON_THROW(detail::out_of_range::create(404, "unresolved reference token '" + reference_token + "'")); + } + } + + return *ptr; + } + + /*! + @throw parse_error.106 if an array index begins with '0' + @throw parse_error.109 if an array index was not a number + @throw out_of_range.402 if the array index '-' is used + @throw out_of_range.404 if the JSON pointer can not be resolved + */ + BasicJsonType& get_checked(BasicJsonType* ptr) const + { + for (const auto& reference_token : reference_tokens) + { + switch (ptr->type()) + { + case detail::value_t::object: + { + // note: at performs range check + ptr = &ptr->at(reference_token); + break; + } + + case detail::value_t::array: + { + if (JSON_HEDLEY_UNLIKELY(reference_token == "-")) + { + // "-" always fails the range check + JSON_THROW(detail::out_of_range::create(402, + "array index '-' (" + std::to_string(ptr->m_value.array->size()) + + ") is out of range")); + } + + // note: at performs range check + ptr = &ptr->at(array_index(reference_token)); + break; + } + + default: + JSON_THROW(detail::out_of_range::create(404, "unresolved reference token '" + reference_token + "'")); + } + } + + return *ptr; + } + + /*! + @brief return a const reference to the pointed to value + + @param[in] ptr a JSON value + + @return const reference to the JSON value pointed to by the JSON + pointer + + @throw parse_error.106 if an array index begins with '0' + @throw parse_error.109 if an array index was not a number + @throw out_of_range.402 if the array index '-' is used + @throw out_of_range.404 if the JSON pointer can not be resolved + */ + const BasicJsonType& get_unchecked(const BasicJsonType* ptr) const + { + for (const auto& reference_token : reference_tokens) + { + switch (ptr->type()) + { + case detail::value_t::object: + { + // use unchecked object access + ptr = &ptr->operator[](reference_token); + break; + } + + case detail::value_t::array: + { + if (JSON_HEDLEY_UNLIKELY(reference_token == "-")) + { + // "-" cannot be used for const access + JSON_THROW(detail::out_of_range::create(402, + "array index '-' (" + std::to_string(ptr->m_value.array->size()) + + ") is out of range")); + } + + // use unchecked array access + ptr = &ptr->operator[](array_index(reference_token)); + break; + } + + default: + JSON_THROW(detail::out_of_range::create(404, "unresolved reference token '" + reference_token + "'")); + } + } + + return *ptr; + } + + /*! + @throw parse_error.106 if an array index begins with '0' + @throw parse_error.109 if an array index was not a number + @throw out_of_range.402 if the array index '-' is used + @throw out_of_range.404 if the JSON pointer can not be resolved + */ + const BasicJsonType& get_checked(const BasicJsonType* ptr) const + { + for (const auto& reference_token : reference_tokens) + { + switch (ptr->type()) + { + case detail::value_t::object: + { + // note: at performs range check + ptr = &ptr->at(reference_token); + break; + } + + case detail::value_t::array: + { + if (JSON_HEDLEY_UNLIKELY(reference_token == "-")) + { + // "-" always fails the range check + JSON_THROW(detail::out_of_range::create(402, + "array index '-' (" + std::to_string(ptr->m_value.array->size()) + + ") is out of range")); + } + + // note: at performs range check + ptr = &ptr->at(array_index(reference_token)); + break; + } + + default: + JSON_THROW(detail::out_of_range::create(404, "unresolved reference token '" + reference_token + "'")); + } + } + + return *ptr; + } + + /*! + @throw parse_error.106 if an array index begins with '0' + @throw parse_error.109 if an array index was not a number + */ + bool contains(const BasicJsonType* ptr) const + { + for (const auto& reference_token : reference_tokens) + { + switch (ptr->type()) + { + case detail::value_t::object: + { + if (!ptr->contains(reference_token)) + { + // we did not find the key in the object + return false; + } + + ptr = &ptr->operator[](reference_token); + break; + } + + case detail::value_t::array: + { + if (JSON_HEDLEY_UNLIKELY(reference_token == "-")) + { + // "-" always fails the range check + return false; + } + if (JSON_HEDLEY_UNLIKELY(reference_token.size() == 1 && !("0" <= reference_token && reference_token <= "9"))) + { + // invalid char + return false; + } + if (JSON_HEDLEY_UNLIKELY(reference_token.size() > 1)) + { + if (JSON_HEDLEY_UNLIKELY(!('1' <= reference_token[0] && reference_token[0] <= '9'))) + { + // first char should be between '1' and '9' + return false; + } + for (std::size_t i = 1; i < reference_token.size(); i++) + { + if (JSON_HEDLEY_UNLIKELY(!('0' <= reference_token[i] && reference_token[i] <= '9'))) + { + // other char should be between '0' and '9' + return false; + } + } + } + + const auto idx = array_index(reference_token); + if (idx >= ptr->size()) + { + // index out of range + return false; + } + + ptr = &ptr->operator[](idx); + break; + } + + default: + { + // we do not expect primitive values if there is still a + // reference token to process + return false; + } + } + } + + // no reference token left means we found a primitive value + return true; + } + + /*! + @brief split the string input to reference tokens + + @note This function is only called by the json_pointer constructor. + All exceptions below are documented there. + + @throw parse_error.107 if the pointer is not empty or begins with '/' + @throw parse_error.108 if character '~' is not followed by '0' or '1' + */ + static std::vector split(const std::string& reference_string) + { + std::vector result; + + // special case: empty reference string -> no reference tokens + if (reference_string.empty()) + { + return result; + } + + // check if nonempty reference string begins with slash + if (JSON_HEDLEY_UNLIKELY(reference_string[0] != '/')) + { + JSON_THROW(detail::parse_error::create(107, 1, + "JSON pointer must be empty or begin with '/' - was: '" + + reference_string + "'")); + } + + // extract the reference tokens: + // - slash: position of the last read slash (or end of string) + // - start: position after the previous slash + for ( + // search for the first slash after the first character + std::size_t slash = reference_string.find_first_of('/', 1), + // set the beginning of the first reference token + start = 1; + // we can stop if start == 0 (if slash == std::string::npos) + start != 0; + // set the beginning of the next reference token + // (will eventually be 0 if slash == std::string::npos) + start = (slash == std::string::npos) ? 0 : slash + 1, + // find next slash + slash = reference_string.find_first_of('/', start)) + { + // use the text between the beginning of the reference token + // (start) and the last slash (slash). + auto reference_token = reference_string.substr(start, slash - start); + + // check reference tokens are properly escaped + for (std::size_t pos = reference_token.find_first_of('~'); + pos != std::string::npos; + pos = reference_token.find_first_of('~', pos + 1)) + { + JSON_ASSERT(reference_token[pos] == '~'); + + // ~ must be followed by 0 or 1 + if (JSON_HEDLEY_UNLIKELY(pos == reference_token.size() - 1 || + (reference_token[pos + 1] != '0' && + reference_token[pos + 1] != '1'))) + { + JSON_THROW(detail::parse_error::create(108, 0, "escape character '~' must be followed with '0' or '1'")); + } + } + + // finally, store the reference token + unescape(reference_token); + result.push_back(reference_token); + } + + return result; + } + + /*! + @brief replace all occurrences of a substring by another string + + @param[in,out] s the string to manipulate; changed so that all + occurrences of @a f are replaced with @a t + @param[in] f the substring to replace with @a t + @param[in] t the string to replace @a f + + @pre The search string @a f must not be empty. **This precondition is + enforced with an assertion.** + + @since version 2.0.0 + */ + static void replace_substring(std::string& s, const std::string& f, + const std::string& t) + { + JSON_ASSERT(!f.empty()); + for (auto pos = s.find(f); // find first occurrence of f + pos != std::string::npos; // make sure f was found + s.replace(pos, f.size(), t), // replace with t, and + pos = s.find(f, pos + t.size())) // find next occurrence of f + {} + } + + /// escape "~" to "~0" and "/" to "~1" + static std::string escape(std::string s) + { + replace_substring(s, "~", "~0"); + replace_substring(s, "/", "~1"); + return s; + } + + /// unescape "~1" to tilde and "~0" to slash (order is important!) + static void unescape(std::string& s) + { + replace_substring(s, "~1", "/"); + replace_substring(s, "~0", "~"); + } + + /*! + @param[in] reference_string the reference string to the current value + @param[in] value the value to consider + @param[in,out] result the result object to insert values to + + @note Empty objects or arrays are flattened to `null`. + */ + static void flatten(const std::string& reference_string, + const BasicJsonType& value, + BasicJsonType& result) + { + switch (value.type()) + { + case detail::value_t::array: + { + if (value.m_value.array->empty()) + { + // flatten empty array as null + result[reference_string] = nullptr; + } + else + { + // iterate array and use index as reference string + for (std::size_t i = 0; i < value.m_value.array->size(); ++i) + { + flatten(reference_string + "/" + std::to_string(i), + value.m_value.array->operator[](i), result); + } + } + break; + } + + case detail::value_t::object: + { + if (value.m_value.object->empty()) + { + // flatten empty object as null + result[reference_string] = nullptr; + } + else + { + // iterate object and use keys as reference string + for (const auto& element : *value.m_value.object) + { + flatten(reference_string + "/" + escape(element.first), element.second, result); + } + } + break; + } + + default: + { + // add primitive value with its reference string + result[reference_string] = value; + break; + } + } + } + + /*! + @param[in] value flattened JSON + + @return unflattened JSON + + @throw parse_error.109 if array index is not a number + @throw type_error.314 if value is not an object + @throw type_error.315 if object values are not primitive + @throw type_error.313 if value cannot be unflattened + */ + static BasicJsonType + unflatten(const BasicJsonType& value) + { + if (JSON_HEDLEY_UNLIKELY(!value.is_object())) + { + JSON_THROW(detail::type_error::create(314, "only objects can be unflattened")); + } + + BasicJsonType result; + + // iterate the JSON object values + for (const auto& element : *value.m_value.object) + { + if (JSON_HEDLEY_UNLIKELY(!element.second.is_primitive())) + { + JSON_THROW(detail::type_error::create(315, "values in object must be primitive")); + } + + // assign value to reference pointed to by JSON pointer; Note that if + // the JSON pointer is "" (i.e., points to the whole value), function + // get_and_create returns a reference to result itself. An assignment + // will then create a primitive value. + json_pointer(element.first).get_and_create(result) = element.second; + } + + return result; + } + + /*! + @brief compares two JSON pointers for equality + + @param[in] lhs JSON pointer to compare + @param[in] rhs JSON pointer to compare + @return whether @a lhs is equal to @a rhs + + @complexity Linear in the length of the JSON pointer + + @exceptionsafety No-throw guarantee: this function never throws exceptions. + */ + friend bool operator==(json_pointer const& lhs, + json_pointer const& rhs) noexcept + { + return lhs.reference_tokens == rhs.reference_tokens; + } + + /*! + @brief compares two JSON pointers for inequality + + @param[in] lhs JSON pointer to compare + @param[in] rhs JSON pointer to compare + @return whether @a lhs is not equal @a rhs + + @complexity Linear in the length of the JSON pointer + + @exceptionsafety No-throw guarantee: this function never throws exceptions. + */ + friend bool operator!=(json_pointer const& lhs, + json_pointer const& rhs) noexcept + { + return !(lhs == rhs); + } + + /// the reference tokens + std::vector reference_tokens; +}; +} // namespace nlohmann + +// #include + + +#include +#include + +// #include + + +namespace nlohmann +{ +namespace detail +{ +template +class json_ref +{ + public: + using value_type = BasicJsonType; + + json_ref(value_type&& value) + : owned_value(std::move(value)) + , value_ref(&owned_value) + , is_rvalue(true) + {} + + json_ref(const value_type& value) + : value_ref(const_cast(&value)) + , is_rvalue(false) + {} + + json_ref(std::initializer_list init) + : owned_value(init) + , value_ref(&owned_value) + , is_rvalue(true) + {} + + template < + class... Args, + enable_if_t::value, int> = 0 > + json_ref(Args && ... args) + : owned_value(std::forward(args)...) + , value_ref(&owned_value) + , is_rvalue(true) + {} + + // class should be movable only + json_ref(json_ref&&) = default; + json_ref(const json_ref&) = delete; + json_ref& operator=(const json_ref&) = delete; + json_ref& operator=(json_ref&&) = delete; + ~json_ref() = default; + + value_type moved_or_copied() const + { + if (is_rvalue) + { + return std::move(*value_ref); + } + return *value_ref; + } + + value_type const& operator*() const + { + return *static_cast(value_ref); + } + + value_type const* operator->() const + { + return static_cast(value_ref); + } + + private: + mutable value_type owned_value = nullptr; + value_type* value_ref = nullptr; + const bool is_rvalue = true; +}; +} // namespace detail +} // namespace nlohmann + +// #include + +// #include + +// #include + +// #include + + +#include // reverse +#include // array +#include // uint8_t, uint16_t, uint32_t, uint64_t +#include // memcpy +#include // numeric_limits +#include // string +#include // isnan, isinf + +// #include + +// #include + +// #include + + +#include // copy +#include // size_t +#include // streamsize +#include // back_inserter +#include // shared_ptr, make_shared +#include // basic_ostream +#include // basic_string +#include // vector +// #include + + +namespace nlohmann +{ +namespace detail +{ +/// abstract output adapter interface +template struct output_adapter_protocol +{ + virtual void write_character(CharType c) = 0; + virtual void write_characters(const CharType* s, std::size_t length) = 0; + virtual ~output_adapter_protocol() = default; +}; + +/// a type to simplify interfaces +template +using output_adapter_t = std::shared_ptr>; + +/// output adapter for byte vectors +template +class output_vector_adapter : public output_adapter_protocol +{ + public: + explicit output_vector_adapter(std::vector& vec) noexcept + : v(vec) + {} + + void write_character(CharType c) override + { + v.push_back(c); + } + + JSON_HEDLEY_NON_NULL(2) + void write_characters(const CharType* s, std::size_t length) override + { + std::copy(s, s + length, std::back_inserter(v)); + } + + private: + std::vector& v; +}; + +/// output adapter for output streams +template +class output_stream_adapter : public output_adapter_protocol +{ + public: + explicit output_stream_adapter(std::basic_ostream& s) noexcept + : stream(s) + {} + + void write_character(CharType c) override + { + stream.put(c); + } + + JSON_HEDLEY_NON_NULL(2) + void write_characters(const CharType* s, std::size_t length) override + { + stream.write(s, static_cast(length)); + } + + private: + std::basic_ostream& stream; +}; + +/// output adapter for basic_string +template> +class output_string_adapter : public output_adapter_protocol +{ + public: + explicit output_string_adapter(StringType& s) noexcept + : str(s) + {} + + void write_character(CharType c) override + { + str.push_back(c); + } + + JSON_HEDLEY_NON_NULL(2) + void write_characters(const CharType* s, std::size_t length) override + { + str.append(s, length); + } + + private: + StringType& str; +}; + +template> +class output_adapter +{ + public: + output_adapter(std::vector& vec) + : oa(std::make_shared>(vec)) {} + + output_adapter(std::basic_ostream& s) + : oa(std::make_shared>(s)) {} + + output_adapter(StringType& s) + : oa(std::make_shared>(s)) {} + + operator output_adapter_t() + { + return oa; + } + + private: + output_adapter_t oa = nullptr; +}; +} // namespace detail +} // namespace nlohmann + + +namespace nlohmann +{ +namespace detail +{ +/////////////////// +// binary writer // +/////////////////// + +/*! +@brief serialization to CBOR and MessagePack values +*/ +template +class binary_writer +{ + using string_t = typename BasicJsonType::string_t; + using binary_t = typename BasicJsonType::binary_t; + using number_float_t = typename BasicJsonType::number_float_t; + + public: + /*! + @brief create a binary writer + + @param[in] adapter output adapter to write to + */ + explicit binary_writer(output_adapter_t adapter) : oa(adapter) + { + JSON_ASSERT(oa); + } + + /*! + @param[in] j JSON value to serialize + @pre j.type() == value_t::object + */ + void write_bson(const BasicJsonType& j) + { + switch (j.type()) + { + case value_t::object: + { + write_bson_object(*j.m_value.object); + break; + } + + default: + { + JSON_THROW(type_error::create(317, "to serialize to BSON, top-level type must be object, but is " + std::string(j.type_name()))); + } + } + } + + /*! + @param[in] j JSON value to serialize + */ + void write_cbor(const BasicJsonType& j) + { + switch (j.type()) + { + case value_t::null: + { + oa->write_character(to_char_type(0xF6)); + break; + } + + case value_t::boolean: + { + oa->write_character(j.m_value.boolean + ? to_char_type(0xF5) + : to_char_type(0xF4)); + break; + } + + case value_t::number_integer: + { + if (j.m_value.number_integer >= 0) + { + // CBOR does not differentiate between positive signed + // integers and unsigned integers. Therefore, we used the + // code from the value_t::number_unsigned case here. + if (j.m_value.number_integer <= 0x17) + { + write_number(static_cast(j.m_value.number_integer)); + } + else if (j.m_value.number_integer <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0x18)); + write_number(static_cast(j.m_value.number_integer)); + } + else if (j.m_value.number_integer <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0x19)); + write_number(static_cast(j.m_value.number_integer)); + } + else if (j.m_value.number_integer <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0x1A)); + write_number(static_cast(j.m_value.number_integer)); + } + else + { + oa->write_character(to_char_type(0x1B)); + write_number(static_cast(j.m_value.number_integer)); + } + } + else + { + // The conversions below encode the sign in the first + // byte, and the value is converted to a positive number. + const auto positive_number = -1 - j.m_value.number_integer; + if (j.m_value.number_integer >= -24) + { + write_number(static_cast(0x20 + positive_number)); + } + else if (positive_number <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0x38)); + write_number(static_cast(positive_number)); + } + else if (positive_number <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0x39)); + write_number(static_cast(positive_number)); + } + else if (positive_number <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0x3A)); + write_number(static_cast(positive_number)); + } + else + { + oa->write_character(to_char_type(0x3B)); + write_number(static_cast(positive_number)); + } + } + break; + } + + case value_t::number_unsigned: + { + if (j.m_value.number_unsigned <= 0x17) + { + write_number(static_cast(j.m_value.number_unsigned)); + } + else if (j.m_value.number_unsigned <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0x18)); + write_number(static_cast(j.m_value.number_unsigned)); + } + else if (j.m_value.number_unsigned <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0x19)); + write_number(static_cast(j.m_value.number_unsigned)); + } + else if (j.m_value.number_unsigned <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0x1A)); + write_number(static_cast(j.m_value.number_unsigned)); + } + else + { + oa->write_character(to_char_type(0x1B)); + write_number(static_cast(j.m_value.number_unsigned)); + } + break; + } + + case value_t::number_float: + { + if (std::isnan(j.m_value.number_float)) + { + // NaN is 0xf97e00 in CBOR + oa->write_character(to_char_type(0xF9)); + oa->write_character(to_char_type(0x7E)); + oa->write_character(to_char_type(0x00)); + } + else if (std::isinf(j.m_value.number_float)) + { + // Infinity is 0xf97c00, -Infinity is 0xf9fc00 + oa->write_character(to_char_type(0xf9)); + oa->write_character(j.m_value.number_float > 0 ? to_char_type(0x7C) : to_char_type(0xFC)); + oa->write_character(to_char_type(0x00)); + } + else + { + write_compact_float(j.m_value.number_float, detail::input_format_t::cbor); + } + break; + } + + case value_t::string: + { + // step 1: write control byte and the string length + const auto N = j.m_value.string->size(); + if (N <= 0x17) + { + write_number(static_cast(0x60 + N)); + } + else if (N <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0x78)); + write_number(static_cast(N)); + } + else if (N <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0x79)); + write_number(static_cast(N)); + } + else if (N <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0x7A)); + write_number(static_cast(N)); + } + // LCOV_EXCL_START + else if (N <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0x7B)); + write_number(static_cast(N)); + } + // LCOV_EXCL_STOP + + // step 2: write the string + oa->write_characters( + reinterpret_cast(j.m_value.string->c_str()), + j.m_value.string->size()); + break; + } + + case value_t::array: + { + // step 1: write control byte and the array size + const auto N = j.m_value.array->size(); + if (N <= 0x17) + { + write_number(static_cast(0x80 + N)); + } + else if (N <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0x98)); + write_number(static_cast(N)); + } + else if (N <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0x99)); + write_number(static_cast(N)); + } + else if (N <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0x9A)); + write_number(static_cast(N)); + } + // LCOV_EXCL_START + else if (N <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0x9B)); + write_number(static_cast(N)); + } + // LCOV_EXCL_STOP + + // step 2: write each element + for (const auto& el : *j.m_value.array) + { + write_cbor(el); + } + break; + } + + case value_t::binary: + { + if (j.m_value.binary->has_subtype()) + { + write_number(static_cast(0xd8)); + write_number(j.m_value.binary->subtype()); + } + + // step 1: write control byte and the binary array size + const auto N = j.m_value.binary->size(); + if (N <= 0x17) + { + write_number(static_cast(0x40 + N)); + } + else if (N <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0x58)); + write_number(static_cast(N)); + } + else if (N <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0x59)); + write_number(static_cast(N)); + } + else if (N <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0x5A)); + write_number(static_cast(N)); + } + // LCOV_EXCL_START + else if (N <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0x5B)); + write_number(static_cast(N)); + } + // LCOV_EXCL_STOP + + // step 2: write each element + oa->write_characters( + reinterpret_cast(j.m_value.binary->data()), + N); + + break; + } + + case value_t::object: + { + // step 1: write control byte and the object size + const auto N = j.m_value.object->size(); + if (N <= 0x17) + { + write_number(static_cast(0xA0 + N)); + } + else if (N <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0xB8)); + write_number(static_cast(N)); + } + else if (N <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0xB9)); + write_number(static_cast(N)); + } + else if (N <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0xBA)); + write_number(static_cast(N)); + } + // LCOV_EXCL_START + else if (N <= (std::numeric_limits::max)()) + { + oa->write_character(to_char_type(0xBB)); + write_number(static_cast(N)); + } + // LCOV_EXCL_STOP + + // step 2: write each element + for (const auto& el : *j.m_value.object) + { + write_cbor(el.first); + write_cbor(el.second); + } + break; + } + + default: + break; + } + } + + /*! + @param[in] j JSON value to serialize + */ + void write_msgpack(const BasicJsonType& j) + { + switch (j.type()) + { + case value_t::null: // nil + { + oa->write_character(to_char_type(0xC0)); + break; + } + + case value_t::boolean: // true and false + { + oa->write_character(j.m_value.boolean + ? to_char_type(0xC3) + : to_char_type(0xC2)); + break; + } + + case value_t::number_integer: + { + if (j.m_value.number_integer >= 0) + { + // MessagePack does not differentiate between positive + // signed integers and unsigned integers. Therefore, we used + // the code from the value_t::number_unsigned case here. + if (j.m_value.number_unsigned < 128) + { + // positive fixnum + write_number(static_cast(j.m_value.number_integer)); + } + else if (j.m_value.number_unsigned <= (std::numeric_limits::max)()) + { + // uint 8 + oa->write_character(to_char_type(0xCC)); + write_number(static_cast(j.m_value.number_integer)); + } + else if (j.m_value.number_unsigned <= (std::numeric_limits::max)()) + { + // uint 16 + oa->write_character(to_char_type(0xCD)); + write_number(static_cast(j.m_value.number_integer)); + } + else if (j.m_value.number_unsigned <= (std::numeric_limits::max)()) + { + // uint 32 + oa->write_character(to_char_type(0xCE)); + write_number(static_cast(j.m_value.number_integer)); + } + else if (j.m_value.number_unsigned <= (std::numeric_limits::max)()) + { + // uint 64 + oa->write_character(to_char_type(0xCF)); + write_number(static_cast(j.m_value.number_integer)); + } + } + else + { + if (j.m_value.number_integer >= -32) + { + // negative fixnum + write_number(static_cast(j.m_value.number_integer)); + } + else if (j.m_value.number_integer >= (std::numeric_limits::min)() && + j.m_value.number_integer <= (std::numeric_limits::max)()) + { + // int 8 + oa->write_character(to_char_type(0xD0)); + write_number(static_cast(j.m_value.number_integer)); + } + else if (j.m_value.number_integer >= (std::numeric_limits::min)() && + j.m_value.number_integer <= (std::numeric_limits::max)()) + { + // int 16 + oa->write_character(to_char_type(0xD1)); + write_number(static_cast(j.m_value.number_integer)); + } + else if (j.m_value.number_integer >= (std::numeric_limits::min)() && + j.m_value.number_integer <= (std::numeric_limits::max)()) + { + // int 32 + oa->write_character(to_char_type(0xD2)); + write_number(static_cast(j.m_value.number_integer)); + } + else if (j.m_value.number_integer >= (std::numeric_limits::min)() && + j.m_value.number_integer <= (std::numeric_limits::max)()) + { + // int 64 + oa->write_character(to_char_type(0xD3)); + write_number(static_cast(j.m_value.number_integer)); + } + } + break; + } + + case value_t::number_unsigned: + { + if (j.m_value.number_unsigned < 128) + { + // positive fixnum + write_number(static_cast(j.m_value.number_integer)); + } + else if (j.m_value.number_unsigned <= (std::numeric_limits::max)()) + { + // uint 8 + oa->write_character(to_char_type(0xCC)); + write_number(static_cast(j.m_value.number_integer)); + } + else if (j.m_value.number_unsigned <= (std::numeric_limits::max)()) + { + // uint 16 + oa->write_character(to_char_type(0xCD)); + write_number(static_cast(j.m_value.number_integer)); + } + else if (j.m_value.number_unsigned <= (std::numeric_limits::max)()) + { + // uint 32 + oa->write_character(to_char_type(0xCE)); + write_number(static_cast(j.m_value.number_integer)); + } + else if (j.m_value.number_unsigned <= (std::numeric_limits::max)()) + { + // uint 64 + oa->write_character(to_char_type(0xCF)); + write_number(static_cast(j.m_value.number_integer)); + } + break; + } + + case value_t::number_float: + { + write_compact_float(j.m_value.number_float, detail::input_format_t::msgpack); + break; + } + + case value_t::string: + { + // step 1: write control byte and the string length + const auto N = j.m_value.string->size(); + if (N <= 31) + { + // fixstr + write_number(static_cast(0xA0 | N)); + } + else if (N <= (std::numeric_limits::max)()) + { + // str 8 + oa->write_character(to_char_type(0xD9)); + write_number(static_cast(N)); + } + else if (N <= (std::numeric_limits::max)()) + { + // str 16 + oa->write_character(to_char_type(0xDA)); + write_number(static_cast(N)); + } + else if (N <= (std::numeric_limits::max)()) + { + // str 32 + oa->write_character(to_char_type(0xDB)); + write_number(static_cast(N)); + } + + // step 2: write the string + oa->write_characters( + reinterpret_cast(j.m_value.string->c_str()), + j.m_value.string->size()); + break; + } + + case value_t::array: + { + // step 1: write control byte and the array size + const auto N = j.m_value.array->size(); + if (N <= 15) + { + // fixarray + write_number(static_cast(0x90 | N)); + } + else if (N <= (std::numeric_limits::max)()) + { + // array 16 + oa->write_character(to_char_type(0xDC)); + write_number(static_cast(N)); + } + else if (N <= (std::numeric_limits::max)()) + { + // array 32 + oa->write_character(to_char_type(0xDD)); + write_number(static_cast(N)); + } + + // step 2: write each element + for (const auto& el : *j.m_value.array) + { + write_msgpack(el); + } + break; + } + + case value_t::binary: + { + // step 0: determine if the binary type has a set subtype to + // determine whether or not to use the ext or fixext types + const bool use_ext = j.m_value.binary->has_subtype(); + + // step 1: write control byte and the byte string length + const auto N = j.m_value.binary->size(); + if (N <= (std::numeric_limits::max)()) + { + std::uint8_t output_type{}; + bool fixed = true; + if (use_ext) + { + switch (N) + { + case 1: + output_type = 0xD4; // fixext 1 + break; + case 2: + output_type = 0xD5; // fixext 2 + break; + case 4: + output_type = 0xD6; // fixext 4 + break; + case 8: + output_type = 0xD7; // fixext 8 + break; + case 16: + output_type = 0xD8; // fixext 16 + break; + default: + output_type = 0xC7; // ext 8 + fixed = false; + break; + } + + } + else + { + output_type = 0xC4; // bin 8 + fixed = false; + } + + oa->write_character(to_char_type(output_type)); + if (!fixed) + { + write_number(static_cast(N)); + } + } + else if (N <= (std::numeric_limits::max)()) + { + std::uint8_t output_type = use_ext + ? 0xC8 // ext 16 + : 0xC5; // bin 16 + + oa->write_character(to_char_type(output_type)); + write_number(static_cast(N)); + } + else if (N <= (std::numeric_limits::max)()) + { + std::uint8_t output_type = use_ext + ? 0xC9 // ext 32 + : 0xC6; // bin 32 + + oa->write_character(to_char_type(output_type)); + write_number(static_cast(N)); + } + + // step 1.5: if this is an ext type, write the subtype + if (use_ext) + { + write_number(static_cast(j.m_value.binary->subtype())); + } + + // step 2: write the byte string + oa->write_characters( + reinterpret_cast(j.m_value.binary->data()), + N); + + break; + } + + case value_t::object: + { + // step 1: write control byte and the object size + const auto N = j.m_value.object->size(); + if (N <= 15) + { + // fixmap + write_number(static_cast(0x80 | (N & 0xF))); + } + else if (N <= (std::numeric_limits::max)()) + { + // map 16 + oa->write_character(to_char_type(0xDE)); + write_number(static_cast(N)); + } + else if (N <= (std::numeric_limits::max)()) + { + // map 32 + oa->write_character(to_char_type(0xDF)); + write_number(static_cast(N)); + } + + // step 2: write each element + for (const auto& el : *j.m_value.object) + { + write_msgpack(el.first); + write_msgpack(el.second); + } + break; + } + + default: + break; + } + } + + /*! + @param[in] j JSON value to serialize + @param[in] use_count whether to use '#' prefixes (optimized format) + @param[in] use_type whether to use '$' prefixes (optimized format) + @param[in] add_prefix whether prefixes need to be used for this value + */ + void write_ubjson(const BasicJsonType& j, const bool use_count, + const bool use_type, const bool add_prefix = true) + { + switch (j.type()) + { + case value_t::null: + { + if (add_prefix) + { + oa->write_character(to_char_type('Z')); + } + break; + } + + case value_t::boolean: + { + if (add_prefix) + { + oa->write_character(j.m_value.boolean + ? to_char_type('T') + : to_char_type('F')); + } + break; + } + + case value_t::number_integer: + { + write_number_with_ubjson_prefix(j.m_value.number_integer, add_prefix); + break; + } + + case value_t::number_unsigned: + { + write_number_with_ubjson_prefix(j.m_value.number_unsigned, add_prefix); + break; + } + + case value_t::number_float: + { + write_number_with_ubjson_prefix(j.m_value.number_float, add_prefix); + break; + } + + case value_t::string: + { + if (add_prefix) + { + oa->write_character(to_char_type('S')); + } + write_number_with_ubjson_prefix(j.m_value.string->size(), true); + oa->write_characters( + reinterpret_cast(j.m_value.string->c_str()), + j.m_value.string->size()); + break; + } + + case value_t::array: + { + if (add_prefix) + { + oa->write_character(to_char_type('[')); + } + + bool prefix_required = true; + if (use_type && !j.m_value.array->empty()) + { + JSON_ASSERT(use_count); + const CharType first_prefix = ubjson_prefix(j.front()); + const bool same_prefix = std::all_of(j.begin() + 1, j.end(), + [this, first_prefix](const BasicJsonType & v) + { + return ubjson_prefix(v) == first_prefix; + }); + + if (same_prefix) + { + prefix_required = false; + oa->write_character(to_char_type('$')); + oa->write_character(first_prefix); + } + } + + if (use_count) + { + oa->write_character(to_char_type('#')); + write_number_with_ubjson_prefix(j.m_value.array->size(), true); + } + + for (const auto& el : *j.m_value.array) + { + write_ubjson(el, use_count, use_type, prefix_required); + } + + if (!use_count) + { + oa->write_character(to_char_type(']')); + } + + break; + } + + case value_t::binary: + { + if (add_prefix) + { + oa->write_character(to_char_type('[')); + } + + if (use_type && !j.m_value.binary->empty()) + { + JSON_ASSERT(use_count); + oa->write_character(to_char_type('$')); + oa->write_character('U'); + } + + if (use_count) + { + oa->write_character(to_char_type('#')); + write_number_with_ubjson_prefix(j.m_value.binary->size(), true); + } + + if (use_type) + { + oa->write_characters( + reinterpret_cast(j.m_value.binary->data()), + j.m_value.binary->size()); + } + else + { + for (size_t i = 0; i < j.m_value.binary->size(); ++i) + { + oa->write_character(to_char_type('U')); + oa->write_character(j.m_value.binary->data()[i]); + } + } + + if (!use_count) + { + oa->write_character(to_char_type(']')); + } + + break; + } + + case value_t::object: + { + if (add_prefix) + { + oa->write_character(to_char_type('{')); + } + + bool prefix_required = true; + if (use_type && !j.m_value.object->empty()) + { + JSON_ASSERT(use_count); + const CharType first_prefix = ubjson_prefix(j.front()); + const bool same_prefix = std::all_of(j.begin(), j.end(), + [this, first_prefix](const BasicJsonType & v) + { + return ubjson_prefix(v) == first_prefix; + }); + + if (same_prefix) + { + prefix_required = false; + oa->write_character(to_char_type('$')); + oa->write_character(first_prefix); + } + } + + if (use_count) + { + oa->write_character(to_char_type('#')); + write_number_with_ubjson_prefix(j.m_value.object->size(), true); + } + + for (const auto& el : *j.m_value.object) + { + write_number_with_ubjson_prefix(el.first.size(), true); + oa->write_characters( + reinterpret_cast(el.first.c_str()), + el.first.size()); + write_ubjson(el.second, use_count, use_type, prefix_required); + } + + if (!use_count) + { + oa->write_character(to_char_type('}')); + } + + break; + } + + default: + break; + } + } + + private: + ////////// + // BSON // + ////////// + + /*! + @return The size of a BSON document entry header, including the id marker + and the entry name size (and its null-terminator). + */ + static std::size_t calc_bson_entry_header_size(const string_t& name) + { + const auto it = name.find(static_cast(0)); + if (JSON_HEDLEY_UNLIKELY(it != BasicJsonType::string_t::npos)) + { + JSON_THROW(out_of_range::create(409, + "BSON key cannot contain code point U+0000 (at byte " + std::to_string(it) + ")")); + } + + return /*id*/ 1ul + name.size() + /*zero-terminator*/1u; + } + + /*! + @brief Writes the given @a element_type and @a name to the output adapter + */ + void write_bson_entry_header(const string_t& name, + const std::uint8_t element_type) + { + oa->write_character(to_char_type(element_type)); // boolean + oa->write_characters( + reinterpret_cast(name.c_str()), + name.size() + 1u); + } + + /*! + @brief Writes a BSON element with key @a name and boolean value @a value + */ + void write_bson_boolean(const string_t& name, + const bool value) + { + write_bson_entry_header(name, 0x08); + oa->write_character(value ? to_char_type(0x01) : to_char_type(0x00)); + } + + /*! + @brief Writes a BSON element with key @a name and double value @a value + */ + void write_bson_double(const string_t& name, + const double value) + { + write_bson_entry_header(name, 0x01); + write_number(value); + } + + /*! + @return The size of the BSON-encoded string in @a value + */ + static std::size_t calc_bson_string_size(const string_t& value) + { + return sizeof(std::int32_t) + value.size() + 1ul; + } + + /*! + @brief Writes a BSON element with key @a name and string value @a value + */ + void write_bson_string(const string_t& name, + const string_t& value) + { + write_bson_entry_header(name, 0x02); + + write_number(static_cast(value.size() + 1ul)); + oa->write_characters( + reinterpret_cast(value.c_str()), + value.size() + 1); + } + + /*! + @brief Writes a BSON element with key @a name and null value + */ + void write_bson_null(const string_t& name) + { + write_bson_entry_header(name, 0x0A); + } + + /*! + @return The size of the BSON-encoded integer @a value + */ + static std::size_t calc_bson_integer_size(const std::int64_t value) + { + return (std::numeric_limits::min)() <= value && value <= (std::numeric_limits::max)() + ? sizeof(std::int32_t) + : sizeof(std::int64_t); + } + + /*! + @brief Writes a BSON element with key @a name and integer @a value + */ + void write_bson_integer(const string_t& name, + const std::int64_t value) + { + if ((std::numeric_limits::min)() <= value && value <= (std::numeric_limits::max)()) + { + write_bson_entry_header(name, 0x10); // int32 + write_number(static_cast(value)); + } + else + { + write_bson_entry_header(name, 0x12); // int64 + write_number(static_cast(value)); + } + } + + /*! + @return The size of the BSON-encoded unsigned integer in @a j + */ + static constexpr std::size_t calc_bson_unsigned_size(const std::uint64_t value) noexcept + { + return (value <= static_cast((std::numeric_limits::max)())) + ? sizeof(std::int32_t) + : sizeof(std::int64_t); + } + + /*! + @brief Writes a BSON element with key @a name and unsigned @a value + */ + void write_bson_unsigned(const string_t& name, + const std::uint64_t value) + { + if (value <= static_cast((std::numeric_limits::max)())) + { + write_bson_entry_header(name, 0x10 /* int32 */); + write_number(static_cast(value)); + } + else if (value <= static_cast((std::numeric_limits::max)())) + { + write_bson_entry_header(name, 0x12 /* int64 */); + write_number(static_cast(value)); + } + else + { + JSON_THROW(out_of_range::create(407, "integer number " + std::to_string(value) + " cannot be represented by BSON as it does not fit int64")); + } + } + + /*! + @brief Writes a BSON element with key @a name and object @a value + */ + void write_bson_object_entry(const string_t& name, + const typename BasicJsonType::object_t& value) + { + write_bson_entry_header(name, 0x03); // object + write_bson_object(value); + } + + /*! + @return The size of the BSON-encoded array @a value + */ + static std::size_t calc_bson_array_size(const typename BasicJsonType::array_t& value) + { + std::size_t array_index = 0ul; + + const std::size_t embedded_document_size = std::accumulate(std::begin(value), std::end(value), std::size_t(0), [&array_index](std::size_t result, const typename BasicJsonType::array_t::value_type & el) + { + return result + calc_bson_element_size(std::to_string(array_index++), el); + }); + + return sizeof(std::int32_t) + embedded_document_size + 1ul; + } + + /*! + @return The size of the BSON-encoded binary array @a value + */ + static std::size_t calc_bson_binary_size(const typename BasicJsonType::binary_t& value) + { + return sizeof(std::int32_t) + value.size() + 1ul; + } + + /*! + @brief Writes a BSON element with key @a name and array @a value + */ + void write_bson_array(const string_t& name, + const typename BasicJsonType::array_t& value) + { + write_bson_entry_header(name, 0x04); // array + write_number(static_cast(calc_bson_array_size(value))); + + std::size_t array_index = 0ul; + + for (const auto& el : value) + { + write_bson_element(std::to_string(array_index++), el); + } + + oa->write_character(to_char_type(0x00)); + } + + /*! + @brief Writes a BSON element with key @a name and binary value @a value + */ + void write_bson_binary(const string_t& name, + const binary_t& value) + { + write_bson_entry_header(name, 0x05); + + write_number(static_cast(value.size())); + write_number(value.has_subtype() ? value.subtype() : std::uint8_t(0x00)); + + oa->write_characters(reinterpret_cast(value.data()), value.size()); + } + + /*! + @brief Calculates the size necessary to serialize the JSON value @a j with its @a name + @return The calculated size for the BSON document entry for @a j with the given @a name. + */ + static std::size_t calc_bson_element_size(const string_t& name, + const BasicJsonType& j) + { + const auto header_size = calc_bson_entry_header_size(name); + switch (j.type()) + { + case value_t::object: + return header_size + calc_bson_object_size(*j.m_value.object); + + case value_t::array: + return header_size + calc_bson_array_size(*j.m_value.array); + + case value_t::binary: + return header_size + calc_bson_binary_size(*j.m_value.binary); + + case value_t::boolean: + return header_size + 1ul; + + case value_t::number_float: + return header_size + 8ul; + + case value_t::number_integer: + return header_size + calc_bson_integer_size(j.m_value.number_integer); + + case value_t::number_unsigned: + return header_size + calc_bson_unsigned_size(j.m_value.number_unsigned); + + case value_t::string: + return header_size + calc_bson_string_size(*j.m_value.string); + + case value_t::null: + return header_size + 0ul; + + // LCOV_EXCL_START + default: + JSON_ASSERT(false); + return 0ul; + // LCOV_EXCL_STOP + } + } + + /*! + @brief Serializes the JSON value @a j to BSON and associates it with the + key @a name. + @param name The name to associate with the JSON entity @a j within the + current BSON document + @return The size of the BSON entry + */ + void write_bson_element(const string_t& name, + const BasicJsonType& j) + { + switch (j.type()) + { + case value_t::object: + return write_bson_object_entry(name, *j.m_value.object); + + case value_t::array: + return write_bson_array(name, *j.m_value.array); + + case value_t::binary: + return write_bson_binary(name, *j.m_value.binary); + + case value_t::boolean: + return write_bson_boolean(name, j.m_value.boolean); + + case value_t::number_float: + return write_bson_double(name, j.m_value.number_float); + + case value_t::number_integer: + return write_bson_integer(name, j.m_value.number_integer); + + case value_t::number_unsigned: + return write_bson_unsigned(name, j.m_value.number_unsigned); + + case value_t::string: + return write_bson_string(name, *j.m_value.string); + + case value_t::null: + return write_bson_null(name); + + // LCOV_EXCL_START + default: + JSON_ASSERT(false); + return; + // LCOV_EXCL_STOP + } + } + + /*! + @brief Calculates the size of the BSON serialization of the given + JSON-object @a j. + @param[in] j JSON value to serialize + @pre j.type() == value_t::object + */ + static std::size_t calc_bson_object_size(const typename BasicJsonType::object_t& value) + { + std::size_t document_size = std::accumulate(value.begin(), value.end(), std::size_t(0), + [](size_t result, const typename BasicJsonType::object_t::value_type & el) + { + return result += calc_bson_element_size(el.first, el.second); + }); + + return sizeof(std::int32_t) + document_size + 1ul; + } + + /*! + @param[in] j JSON value to serialize + @pre j.type() == value_t::object + */ + void write_bson_object(const typename BasicJsonType::object_t& value) + { + write_number(static_cast(calc_bson_object_size(value))); + + for (const auto& el : value) + { + write_bson_element(el.first, el.second); + } + + oa->write_character(to_char_type(0x00)); + } + + ////////// + // CBOR // + ////////// + + static constexpr CharType get_cbor_float_prefix(float /*unused*/) + { + return to_char_type(0xFA); // Single-Precision Float + } + + static constexpr CharType get_cbor_float_prefix(double /*unused*/) + { + return to_char_type(0xFB); // Double-Precision Float + } + + ///////////// + // MsgPack // + ///////////// + + static constexpr CharType get_msgpack_float_prefix(float /*unused*/) + { + return to_char_type(0xCA); // float 32 + } + + static constexpr CharType get_msgpack_float_prefix(double /*unused*/) + { + return to_char_type(0xCB); // float 64 + } + + //////////// + // UBJSON // + //////////// + + // UBJSON: write number (floating point) + template::value, int>::type = 0> + void write_number_with_ubjson_prefix(const NumberType n, + const bool add_prefix) + { + if (add_prefix) + { + oa->write_character(get_ubjson_float_prefix(n)); + } + write_number(n); + } + + // UBJSON: write number (unsigned integer) + template::value, int>::type = 0> + void write_number_with_ubjson_prefix(const NumberType n, + const bool add_prefix) + { + if (n <= static_cast((std::numeric_limits::max)())) + { + if (add_prefix) + { + oa->write_character(to_char_type('i')); // int8 + } + write_number(static_cast(n)); + } + else if (n <= (std::numeric_limits::max)()) + { + if (add_prefix) + { + oa->write_character(to_char_type('U')); // uint8 + } + write_number(static_cast(n)); + } + else if (n <= static_cast((std::numeric_limits::max)())) + { + if (add_prefix) + { + oa->write_character(to_char_type('I')); // int16 + } + write_number(static_cast(n)); + } + else if (n <= static_cast((std::numeric_limits::max)())) + { + if (add_prefix) + { + oa->write_character(to_char_type('l')); // int32 + } + write_number(static_cast(n)); + } + else if (n <= static_cast((std::numeric_limits::max)())) + { + if (add_prefix) + { + oa->write_character(to_char_type('L')); // int64 + } + write_number(static_cast(n)); + } + else + { + if (add_prefix) + { + oa->write_character(to_char_type('H')); // high-precision number + } + + const auto number = BasicJsonType(n).dump(); + write_number_with_ubjson_prefix(number.size(), true); + for (std::size_t i = 0; i < number.size(); ++i) + { + oa->write_character(to_char_type(static_cast(number[i]))); + } + } + } + + // UBJSON: write number (signed integer) + template < typename NumberType, typename std::enable_if < + std::is_signed::value&& + !std::is_floating_point::value, int >::type = 0 > + void write_number_with_ubjson_prefix(const NumberType n, + const bool add_prefix) + { + if ((std::numeric_limits::min)() <= n && n <= (std::numeric_limits::max)()) + { + if (add_prefix) + { + oa->write_character(to_char_type('i')); // int8 + } + write_number(static_cast(n)); + } + else if (static_cast((std::numeric_limits::min)()) <= n && n <= static_cast((std::numeric_limits::max)())) + { + if (add_prefix) + { + oa->write_character(to_char_type('U')); // uint8 + } + write_number(static_cast(n)); + } + else if ((std::numeric_limits::min)() <= n && n <= (std::numeric_limits::max)()) + { + if (add_prefix) + { + oa->write_character(to_char_type('I')); // int16 + } + write_number(static_cast(n)); + } + else if ((std::numeric_limits::min)() <= n && n <= (std::numeric_limits::max)()) + { + if (add_prefix) + { + oa->write_character(to_char_type('l')); // int32 + } + write_number(static_cast(n)); + } + else if ((std::numeric_limits::min)() <= n && n <= (std::numeric_limits::max)()) + { + if (add_prefix) + { + oa->write_character(to_char_type('L')); // int64 + } + write_number(static_cast(n)); + } + // LCOV_EXCL_START + else + { + if (add_prefix) + { + oa->write_character(to_char_type('H')); // high-precision number + } + + const auto number = BasicJsonType(n).dump(); + write_number_with_ubjson_prefix(number.size(), true); + for (std::size_t i = 0; i < number.size(); ++i) + { + oa->write_character(to_char_type(static_cast(number[i]))); + } + } + // LCOV_EXCL_STOP + } + + /*! + @brief determine the type prefix of container values + */ + CharType ubjson_prefix(const BasicJsonType& j) const noexcept + { + switch (j.type()) + { + case value_t::null: + return 'Z'; + + case value_t::boolean: + return j.m_value.boolean ? 'T' : 'F'; + + case value_t::number_integer: + { + if ((std::numeric_limits::min)() <= j.m_value.number_integer && j.m_value.number_integer <= (std::numeric_limits::max)()) + { + return 'i'; + } + if ((std::numeric_limits::min)() <= j.m_value.number_integer && j.m_value.number_integer <= (std::numeric_limits::max)()) + { + return 'U'; + } + if ((std::numeric_limits::min)() <= j.m_value.number_integer && j.m_value.number_integer <= (std::numeric_limits::max)()) + { + return 'I'; + } + if ((std::numeric_limits::min)() <= j.m_value.number_integer && j.m_value.number_integer <= (std::numeric_limits::max)()) + { + return 'l'; + } + if ((std::numeric_limits::min)() <= j.m_value.number_integer && j.m_value.number_integer <= (std::numeric_limits::max)()) + { + return 'L'; + } + // anything else is treated as high-precision number + return 'H'; // LCOV_EXCL_LINE + } + + case value_t::number_unsigned: + { + if (j.m_value.number_unsigned <= static_cast((std::numeric_limits::max)())) + { + return 'i'; + } + if (j.m_value.number_unsigned <= static_cast((std::numeric_limits::max)())) + { + return 'U'; + } + if (j.m_value.number_unsigned <= static_cast((std::numeric_limits::max)())) + { + return 'I'; + } + if (j.m_value.number_unsigned <= static_cast((std::numeric_limits::max)())) + { + return 'l'; + } + if (j.m_value.number_unsigned <= static_cast((std::numeric_limits::max)())) + { + return 'L'; + } + // anything else is treated as high-precision number + return 'H'; // LCOV_EXCL_LINE + } + + case value_t::number_float: + return get_ubjson_float_prefix(j.m_value.number_float); + + case value_t::string: + return 'S'; + + case value_t::array: // fallthrough + case value_t::binary: + return '['; + + case value_t::object: + return '{'; + + default: // discarded values + return 'N'; + } + } + + static constexpr CharType get_ubjson_float_prefix(float /*unused*/) + { + return 'd'; // float 32 + } + + static constexpr CharType get_ubjson_float_prefix(double /*unused*/) + { + return 'D'; // float 64 + } + + /////////////////////// + // Utility functions // + /////////////////////// + + /* + @brief write a number to output input + @param[in] n number of type @a NumberType + @tparam NumberType the type of the number + @tparam OutputIsLittleEndian Set to true if output data is + required to be little endian + + @note This function needs to respect the system's endianess, because bytes + in CBOR, MessagePack, and UBJSON are stored in network order (big + endian) and therefore need reordering on little endian systems. + */ + template + void write_number(const NumberType n) + { + // step 1: write number to array of length NumberType + std::array vec; + std::memcpy(vec.data(), &n, sizeof(NumberType)); + + // step 2: write array to output (with possible reordering) + if (is_little_endian != OutputIsLittleEndian) + { + // reverse byte order prior to conversion if necessary + std::reverse(vec.begin(), vec.end()); + } + + oa->write_characters(vec.data(), sizeof(NumberType)); + } + + void write_compact_float(const number_float_t n, detail::input_format_t format) + { + if (static_cast(n) >= static_cast(std::numeric_limits::lowest()) && + static_cast(n) <= static_cast((std::numeric_limits::max)()) && + static_cast(static_cast(n)) == static_cast(n)) + { + oa->write_character(format == detail::input_format_t::cbor + ? get_cbor_float_prefix(static_cast(n)) + : get_msgpack_float_prefix(static_cast(n))); + write_number(static_cast(n)); + } + else + { + oa->write_character(format == detail::input_format_t::cbor + ? get_cbor_float_prefix(n) + : get_msgpack_float_prefix(n)); + write_number(n); + } + } + + public: + // The following to_char_type functions are implement the conversion + // between uint8_t and CharType. In case CharType is not unsigned, + // such a conversion is required to allow values greater than 128. + // See for a discussion. + template < typename C = CharType, + enable_if_t < std::is_signed::value && std::is_signed::value > * = nullptr > + static constexpr CharType to_char_type(std::uint8_t x) noexcept + { + return *reinterpret_cast(&x); + } + + template < typename C = CharType, + enable_if_t < std::is_signed::value && std::is_unsigned::value > * = nullptr > + static CharType to_char_type(std::uint8_t x) noexcept + { + static_assert(sizeof(std::uint8_t) == sizeof(CharType), "size of CharType must be equal to std::uint8_t"); + static_assert(std::is_trivial::value, "CharType must be trivial"); + CharType result; + std::memcpy(&result, &x, sizeof(x)); + return result; + } + + template::value>* = nullptr> + static constexpr CharType to_char_type(std::uint8_t x) noexcept + { + return x; + } + + template < typename InputCharType, typename C = CharType, + enable_if_t < + std::is_signed::value && + std::is_signed::value && + std::is_same::type>::value + > * = nullptr > + static constexpr CharType to_char_type(InputCharType x) noexcept + { + return x; + } + + private: + /// whether we can assume little endianess + const bool is_little_endian = little_endianess(); + + /// the output + output_adapter_t oa = nullptr; +}; +} // namespace detail +} // namespace nlohmann + +// #include + +// #include + + +#include // reverse, remove, fill, find, none_of +#include // array +#include // localeconv, lconv +#include // labs, isfinite, isnan, signbit +#include // size_t, ptrdiff_t +#include // uint8_t +#include // snprintf +#include // numeric_limits +#include // string, char_traits +#include // is_same +#include // move + +// #include + + +#include // array +#include // signbit, isfinite +#include // intN_t, uintN_t +#include // memcpy, memmove +#include // numeric_limits +#include // conditional + +// #include + + +namespace nlohmann +{ +namespace detail +{ + +/*! +@brief implements the Grisu2 algorithm for binary to decimal floating-point +conversion. + +This implementation is a slightly modified version of the reference +implementation which may be obtained from +http://florian.loitsch.com/publications (bench.tar.gz). + +The code is distributed under the MIT license, Copyright (c) 2009 Florian Loitsch. + +For a detailed description of the algorithm see: + +[1] Loitsch, "Printing Floating-Point Numbers Quickly and Accurately with + Integers", Proceedings of the ACM SIGPLAN 2010 Conference on Programming + Language Design and Implementation, PLDI 2010 +[2] Burger, Dybvig, "Printing Floating-Point Numbers Quickly and Accurately", + Proceedings of the ACM SIGPLAN 1996 Conference on Programming Language + Design and Implementation, PLDI 1996 +*/ +namespace dtoa_impl +{ + +template +Target reinterpret_bits(const Source source) +{ + static_assert(sizeof(Target) == sizeof(Source), "size mismatch"); + + Target target; + std::memcpy(&target, &source, sizeof(Source)); + return target; +} + +struct diyfp // f * 2^e +{ + static constexpr int kPrecision = 64; // = q + + std::uint64_t f = 0; + int e = 0; + + constexpr diyfp(std::uint64_t f_, int e_) noexcept : f(f_), e(e_) {} + + /*! + @brief returns x - y + @pre x.e == y.e and x.f >= y.f + */ + static diyfp sub(const diyfp& x, const diyfp& y) noexcept + { + JSON_ASSERT(x.e == y.e); + JSON_ASSERT(x.f >= y.f); + + return {x.f - y.f, x.e}; + } + + /*! + @brief returns x * y + @note The result is rounded. (Only the upper q bits are returned.) + */ + static diyfp mul(const diyfp& x, const diyfp& y) noexcept + { + static_assert(kPrecision == 64, "internal error"); + + // Computes: + // f = round((x.f * y.f) / 2^q) + // e = x.e + y.e + q + + // Emulate the 64-bit * 64-bit multiplication: + // + // p = u * v + // = (u_lo + 2^32 u_hi) (v_lo + 2^32 v_hi) + // = (u_lo v_lo ) + 2^32 ((u_lo v_hi ) + (u_hi v_lo )) + 2^64 (u_hi v_hi ) + // = (p0 ) + 2^32 ((p1 ) + (p2 )) + 2^64 (p3 ) + // = (p0_lo + 2^32 p0_hi) + 2^32 ((p1_lo + 2^32 p1_hi) + (p2_lo + 2^32 p2_hi)) + 2^64 (p3 ) + // = (p0_lo ) + 2^32 (p0_hi + p1_lo + p2_lo ) + 2^64 (p1_hi + p2_hi + p3) + // = (p0_lo ) + 2^32 (Q ) + 2^64 (H ) + // = (p0_lo ) + 2^32 (Q_lo + 2^32 Q_hi ) + 2^64 (H ) + // + // (Since Q might be larger than 2^32 - 1) + // + // = (p0_lo + 2^32 Q_lo) + 2^64 (Q_hi + H) + // + // (Q_hi + H does not overflow a 64-bit int) + // + // = p_lo + 2^64 p_hi + + const std::uint64_t u_lo = x.f & 0xFFFFFFFFu; + const std::uint64_t u_hi = x.f >> 32u; + const std::uint64_t v_lo = y.f & 0xFFFFFFFFu; + const std::uint64_t v_hi = y.f >> 32u; + + const std::uint64_t p0 = u_lo * v_lo; + const std::uint64_t p1 = u_lo * v_hi; + const std::uint64_t p2 = u_hi * v_lo; + const std::uint64_t p3 = u_hi * v_hi; + + const std::uint64_t p0_hi = p0 >> 32u; + const std::uint64_t p1_lo = p1 & 0xFFFFFFFFu; + const std::uint64_t p1_hi = p1 >> 32u; + const std::uint64_t p2_lo = p2 & 0xFFFFFFFFu; + const std::uint64_t p2_hi = p2 >> 32u; + + std::uint64_t Q = p0_hi + p1_lo + p2_lo; + + // The full product might now be computed as + // + // p_hi = p3 + p2_hi + p1_hi + (Q >> 32) + // p_lo = p0_lo + (Q << 32) + // + // But in this particular case here, the full p_lo is not required. + // Effectively we only need to add the highest bit in p_lo to p_hi (and + // Q_hi + 1 does not overflow). + + Q += std::uint64_t{1} << (64u - 32u - 1u); // round, ties up + + const std::uint64_t h = p3 + p2_hi + p1_hi + (Q >> 32u); + + return {h, x.e + y.e + 64}; + } + + /*! + @brief normalize x such that the significand is >= 2^(q-1) + @pre x.f != 0 + */ + static diyfp normalize(diyfp x) noexcept + { + JSON_ASSERT(x.f != 0); + + while ((x.f >> 63u) == 0) + { + x.f <<= 1u; + x.e--; + } + + return x; + } + + /*! + @brief normalize x such that the result has the exponent E + @pre e >= x.e and the upper e - x.e bits of x.f must be zero. + */ + static diyfp normalize_to(const diyfp& x, const int target_exponent) noexcept + { + const int delta = x.e - target_exponent; + + JSON_ASSERT(delta >= 0); + JSON_ASSERT(((x.f << delta) >> delta) == x.f); + + return {x.f << delta, target_exponent}; + } +}; + +struct boundaries +{ + diyfp w; + diyfp minus; + diyfp plus; +}; + +/*! +Compute the (normalized) diyfp representing the input number 'value' and its +boundaries. + +@pre value must be finite and positive +*/ +template +boundaries compute_boundaries(FloatType value) +{ + JSON_ASSERT(std::isfinite(value)); + JSON_ASSERT(value > 0); + + // Convert the IEEE representation into a diyfp. + // + // If v is denormal: + // value = 0.F * 2^(1 - bias) = ( F) * 2^(1 - bias - (p-1)) + // If v is normalized: + // value = 1.F * 2^(E - bias) = (2^(p-1) + F) * 2^(E - bias - (p-1)) + + static_assert(std::numeric_limits::is_iec559, + "internal error: dtoa_short requires an IEEE-754 floating-point implementation"); + + constexpr int kPrecision = std::numeric_limits::digits; // = p (includes the hidden bit) + constexpr int kBias = std::numeric_limits::max_exponent - 1 + (kPrecision - 1); + constexpr int kMinExp = 1 - kBias; + constexpr std::uint64_t kHiddenBit = std::uint64_t{1} << (kPrecision - 1); // = 2^(p-1) + + using bits_type = typename std::conditional::type; + + const std::uint64_t bits = reinterpret_bits(value); + const std::uint64_t E = bits >> (kPrecision - 1); + const std::uint64_t F = bits & (kHiddenBit - 1); + + const bool is_denormal = E == 0; + const diyfp v = is_denormal + ? diyfp(F, kMinExp) + : diyfp(F + kHiddenBit, static_cast(E) - kBias); + + // Compute the boundaries m- and m+ of the floating-point value + // v = f * 2^e. + // + // Determine v- and v+, the floating-point predecessor and successor if v, + // respectively. + // + // v- = v - 2^e if f != 2^(p-1) or e == e_min (A) + // = v - 2^(e-1) if f == 2^(p-1) and e > e_min (B) + // + // v+ = v + 2^e + // + // Let m- = (v- + v) / 2 and m+ = (v + v+) / 2. All real numbers _strictly_ + // between m- and m+ round to v, regardless of how the input rounding + // algorithm breaks ties. + // + // ---+-------------+-------------+-------------+-------------+--- (A) + // v- m- v m+ v+ + // + // -----------------+------+------+-------------+-------------+--- (B) + // v- m- v m+ v+ + + const bool lower_boundary_is_closer = F == 0 && E > 1; + const diyfp m_plus = diyfp(2 * v.f + 1, v.e - 1); + const diyfp m_minus = lower_boundary_is_closer + ? diyfp(4 * v.f - 1, v.e - 2) // (B) + : diyfp(2 * v.f - 1, v.e - 1); // (A) + + // Determine the normalized w+ = m+. + const diyfp w_plus = diyfp::normalize(m_plus); + + // Determine w- = m- such that e_(w-) = e_(w+). + const diyfp w_minus = diyfp::normalize_to(m_minus, w_plus.e); + + return {diyfp::normalize(v), w_minus, w_plus}; +} + +// Given normalized diyfp w, Grisu needs to find a (normalized) cached +// power-of-ten c, such that the exponent of the product c * w = f * 2^e lies +// within a certain range [alpha, gamma] (Definition 3.2 from [1]) +// +// alpha <= e = e_c + e_w + q <= gamma +// +// or +// +// f_c * f_w * 2^alpha <= f_c 2^(e_c) * f_w 2^(e_w) * 2^q +// <= f_c * f_w * 2^gamma +// +// Since c and w are normalized, i.e. 2^(q-1) <= f < 2^q, this implies +// +// 2^(q-1) * 2^(q-1) * 2^alpha <= c * w * 2^q < 2^q * 2^q * 2^gamma +// +// or +// +// 2^(q - 2 + alpha) <= c * w < 2^(q + gamma) +// +// The choice of (alpha,gamma) determines the size of the table and the form of +// the digit generation procedure. Using (alpha,gamma)=(-60,-32) works out well +// in practice: +// +// The idea is to cut the number c * w = f * 2^e into two parts, which can be +// processed independently: An integral part p1, and a fractional part p2: +// +// f * 2^e = ( (f div 2^-e) * 2^-e + (f mod 2^-e) ) * 2^e +// = (f div 2^-e) + (f mod 2^-e) * 2^e +// = p1 + p2 * 2^e +// +// The conversion of p1 into decimal form requires a series of divisions and +// modulos by (a power of) 10. These operations are faster for 32-bit than for +// 64-bit integers, so p1 should ideally fit into a 32-bit integer. This can be +// achieved by choosing +// +// -e >= 32 or e <= -32 := gamma +// +// In order to convert the fractional part +// +// p2 * 2^e = p2 / 2^-e = d[-1] / 10^1 + d[-2] / 10^2 + ... +// +// into decimal form, the fraction is repeatedly multiplied by 10 and the digits +// d[-i] are extracted in order: +// +// (10 * p2) div 2^-e = d[-1] +// (10 * p2) mod 2^-e = d[-2] / 10^1 + ... +// +// The multiplication by 10 must not overflow. It is sufficient to choose +// +// 10 * p2 < 16 * p2 = 2^4 * p2 <= 2^64. +// +// Since p2 = f mod 2^-e < 2^-e, +// +// -e <= 60 or e >= -60 := alpha + +constexpr int kAlpha = -60; +constexpr int kGamma = -32; + +struct cached_power // c = f * 2^e ~= 10^k +{ + std::uint64_t f; + int e; + int k; +}; + +/*! +For a normalized diyfp w = f * 2^e, this function returns a (normalized) cached +power-of-ten c = f_c * 2^e_c, such that the exponent of the product w * c +satisfies (Definition 3.2 from [1]) + + alpha <= e_c + e + q <= gamma. +*/ +inline cached_power get_cached_power_for_binary_exponent(int e) +{ + // Now + // + // alpha <= e_c + e + q <= gamma (1) + // ==> f_c * 2^alpha <= c * 2^e * 2^q + // + // and since the c's are normalized, 2^(q-1) <= f_c, + // + // ==> 2^(q - 1 + alpha) <= c * 2^(e + q) + // ==> 2^(alpha - e - 1) <= c + // + // If c were an exact power of ten, i.e. c = 10^k, one may determine k as + // + // k = ceil( log_10( 2^(alpha - e - 1) ) ) + // = ceil( (alpha - e - 1) * log_10(2) ) + // + // From the paper: + // "In theory the result of the procedure could be wrong since c is rounded, + // and the computation itself is approximated [...]. In practice, however, + // this simple function is sufficient." + // + // For IEEE double precision floating-point numbers converted into + // normalized diyfp's w = f * 2^e, with q = 64, + // + // e >= -1022 (min IEEE exponent) + // -52 (p - 1) + // -52 (p - 1, possibly normalize denormal IEEE numbers) + // -11 (normalize the diyfp) + // = -1137 + // + // and + // + // e <= +1023 (max IEEE exponent) + // -52 (p - 1) + // -11 (normalize the diyfp) + // = 960 + // + // This binary exponent range [-1137,960] results in a decimal exponent + // range [-307,324]. One does not need to store a cached power for each + // k in this range. For each such k it suffices to find a cached power + // such that the exponent of the product lies in [alpha,gamma]. + // This implies that the difference of the decimal exponents of adjacent + // table entries must be less than or equal to + // + // floor( (gamma - alpha) * log_10(2) ) = 8. + // + // (A smaller distance gamma-alpha would require a larger table.) + + // NB: + // Actually this function returns c, such that -60 <= e_c + e + 64 <= -34. + + constexpr int kCachedPowersMinDecExp = -300; + constexpr int kCachedPowersDecStep = 8; + + static constexpr std::array kCachedPowers = + { + { + { 0xAB70FE17C79AC6CA, -1060, -300 }, + { 0xFF77B1FCBEBCDC4F, -1034, -292 }, + { 0xBE5691EF416BD60C, -1007, -284 }, + { 0x8DD01FAD907FFC3C, -980, -276 }, + { 0xD3515C2831559A83, -954, -268 }, + { 0x9D71AC8FADA6C9B5, -927, -260 }, + { 0xEA9C227723EE8BCB, -901, -252 }, + { 0xAECC49914078536D, -874, -244 }, + { 0x823C12795DB6CE57, -847, -236 }, + { 0xC21094364DFB5637, -821, -228 }, + { 0x9096EA6F3848984F, -794, -220 }, + { 0xD77485CB25823AC7, -768, -212 }, + { 0xA086CFCD97BF97F4, -741, -204 }, + { 0xEF340A98172AACE5, -715, -196 }, + { 0xB23867FB2A35B28E, -688, -188 }, + { 0x84C8D4DFD2C63F3B, -661, -180 }, + { 0xC5DD44271AD3CDBA, -635, -172 }, + { 0x936B9FCEBB25C996, -608, -164 }, + { 0xDBAC6C247D62A584, -582, -156 }, + { 0xA3AB66580D5FDAF6, -555, -148 }, + { 0xF3E2F893DEC3F126, -529, -140 }, + { 0xB5B5ADA8AAFF80B8, -502, -132 }, + { 0x87625F056C7C4A8B, -475, -124 }, + { 0xC9BCFF6034C13053, -449, -116 }, + { 0x964E858C91BA2655, -422, -108 }, + { 0xDFF9772470297EBD, -396, -100 }, + { 0xA6DFBD9FB8E5B88F, -369, -92 }, + { 0xF8A95FCF88747D94, -343, -84 }, + { 0xB94470938FA89BCF, -316, -76 }, + { 0x8A08F0F8BF0F156B, -289, -68 }, + { 0xCDB02555653131B6, -263, -60 }, + { 0x993FE2C6D07B7FAC, -236, -52 }, + { 0xE45C10C42A2B3B06, -210, -44 }, + { 0xAA242499697392D3, -183, -36 }, + { 0xFD87B5F28300CA0E, -157, -28 }, + { 0xBCE5086492111AEB, -130, -20 }, + { 0x8CBCCC096F5088CC, -103, -12 }, + { 0xD1B71758E219652C, -77, -4 }, + { 0x9C40000000000000, -50, 4 }, + { 0xE8D4A51000000000, -24, 12 }, + { 0xAD78EBC5AC620000, 3, 20 }, + { 0x813F3978F8940984, 30, 28 }, + { 0xC097CE7BC90715B3, 56, 36 }, + { 0x8F7E32CE7BEA5C70, 83, 44 }, + { 0xD5D238A4ABE98068, 109, 52 }, + { 0x9F4F2726179A2245, 136, 60 }, + { 0xED63A231D4C4FB27, 162, 68 }, + { 0xB0DE65388CC8ADA8, 189, 76 }, + { 0x83C7088E1AAB65DB, 216, 84 }, + { 0xC45D1DF942711D9A, 242, 92 }, + { 0x924D692CA61BE758, 269, 100 }, + { 0xDA01EE641A708DEA, 295, 108 }, + { 0xA26DA3999AEF774A, 322, 116 }, + { 0xF209787BB47D6B85, 348, 124 }, + { 0xB454E4A179DD1877, 375, 132 }, + { 0x865B86925B9BC5C2, 402, 140 }, + { 0xC83553C5C8965D3D, 428, 148 }, + { 0x952AB45CFA97A0B3, 455, 156 }, + { 0xDE469FBD99A05FE3, 481, 164 }, + { 0xA59BC234DB398C25, 508, 172 }, + { 0xF6C69A72A3989F5C, 534, 180 }, + { 0xB7DCBF5354E9BECE, 561, 188 }, + { 0x88FCF317F22241E2, 588, 196 }, + { 0xCC20CE9BD35C78A5, 614, 204 }, + { 0x98165AF37B2153DF, 641, 212 }, + { 0xE2A0B5DC971F303A, 667, 220 }, + { 0xA8D9D1535CE3B396, 694, 228 }, + { 0xFB9B7CD9A4A7443C, 720, 236 }, + { 0xBB764C4CA7A44410, 747, 244 }, + { 0x8BAB8EEFB6409C1A, 774, 252 }, + { 0xD01FEF10A657842C, 800, 260 }, + { 0x9B10A4E5E9913129, 827, 268 }, + { 0xE7109BFBA19C0C9D, 853, 276 }, + { 0xAC2820D9623BF429, 880, 284 }, + { 0x80444B5E7AA7CF85, 907, 292 }, + { 0xBF21E44003ACDD2D, 933, 300 }, + { 0x8E679C2F5E44FF8F, 960, 308 }, + { 0xD433179D9C8CB841, 986, 316 }, + { 0x9E19DB92B4E31BA9, 1013, 324 }, + } + }; + + // This computation gives exactly the same results for k as + // k = ceil((kAlpha - e - 1) * 0.30102999566398114) + // for |e| <= 1500, but doesn't require floating-point operations. + // NB: log_10(2) ~= 78913 / 2^18 + JSON_ASSERT(e >= -1500); + JSON_ASSERT(e <= 1500); + const int f = kAlpha - e - 1; + const int k = (f * 78913) / (1 << 18) + static_cast(f > 0); + + const int index = (-kCachedPowersMinDecExp + k + (kCachedPowersDecStep - 1)) / kCachedPowersDecStep; + JSON_ASSERT(index >= 0); + JSON_ASSERT(static_cast(index) < kCachedPowers.size()); + + const cached_power cached = kCachedPowers[static_cast(index)]; + JSON_ASSERT(kAlpha <= cached.e + e + 64); + JSON_ASSERT(kGamma >= cached.e + e + 64); + + return cached; +} + +/*! +For n != 0, returns k, such that pow10 := 10^(k-1) <= n < 10^k. +For n == 0, returns 1 and sets pow10 := 1. +*/ +inline int find_largest_pow10(const std::uint32_t n, std::uint32_t& pow10) +{ + // LCOV_EXCL_START + if (n >= 1000000000) + { + pow10 = 1000000000; + return 10; + } + // LCOV_EXCL_STOP + else if (n >= 100000000) + { + pow10 = 100000000; + return 9; + } + else if (n >= 10000000) + { + pow10 = 10000000; + return 8; + } + else if (n >= 1000000) + { + pow10 = 1000000; + return 7; + } + else if (n >= 100000) + { + pow10 = 100000; + return 6; + } + else if (n >= 10000) + { + pow10 = 10000; + return 5; + } + else if (n >= 1000) + { + pow10 = 1000; + return 4; + } + else if (n >= 100) + { + pow10 = 100; + return 3; + } + else if (n >= 10) + { + pow10 = 10; + return 2; + } + else + { + pow10 = 1; + return 1; + } +} + +inline void grisu2_round(char* buf, int len, std::uint64_t dist, std::uint64_t delta, + std::uint64_t rest, std::uint64_t ten_k) +{ + JSON_ASSERT(len >= 1); + JSON_ASSERT(dist <= delta); + JSON_ASSERT(rest <= delta); + JSON_ASSERT(ten_k > 0); + + // <--------------------------- delta ----> + // <---- dist ---------> + // --------------[------------------+-------------------]-------------- + // M- w M+ + // + // ten_k + // <------> + // <---- rest ----> + // --------------[------------------+----+--------------]-------------- + // w V + // = buf * 10^k + // + // ten_k represents a unit-in-the-last-place in the decimal representation + // stored in buf. + // Decrement buf by ten_k while this takes buf closer to w. + + // The tests are written in this order to avoid overflow in unsigned + // integer arithmetic. + + while (rest < dist + && delta - rest >= ten_k + && (rest + ten_k < dist || dist - rest > rest + ten_k - dist)) + { + JSON_ASSERT(buf[len - 1] != '0'); + buf[len - 1]--; + rest += ten_k; + } +} + +/*! +Generates V = buffer * 10^decimal_exponent, such that M- <= V <= M+. +M- and M+ must be normalized and share the same exponent -60 <= e <= -32. +*/ +inline void grisu2_digit_gen(char* buffer, int& length, int& decimal_exponent, + diyfp M_minus, diyfp w, diyfp M_plus) +{ + static_assert(kAlpha >= -60, "internal error"); + static_assert(kGamma <= -32, "internal error"); + + // Generates the digits (and the exponent) of a decimal floating-point + // number V = buffer * 10^decimal_exponent in the range [M-, M+]. The diyfp's + // w, M- and M+ share the same exponent e, which satisfies alpha <= e <= gamma. + // + // <--------------------------- delta ----> + // <---- dist ---------> + // --------------[------------------+-------------------]-------------- + // M- w M+ + // + // Grisu2 generates the digits of M+ from left to right and stops as soon as + // V is in [M-,M+]. + + JSON_ASSERT(M_plus.e >= kAlpha); + JSON_ASSERT(M_plus.e <= kGamma); + + std::uint64_t delta = diyfp::sub(M_plus, M_minus).f; // (significand of (M+ - M-), implicit exponent is e) + std::uint64_t dist = diyfp::sub(M_plus, w ).f; // (significand of (M+ - w ), implicit exponent is e) + + // Split M+ = f * 2^e into two parts p1 and p2 (note: e < 0): + // + // M+ = f * 2^e + // = ((f div 2^-e) * 2^-e + (f mod 2^-e)) * 2^e + // = ((p1 ) * 2^-e + (p2 )) * 2^e + // = p1 + p2 * 2^e + + const diyfp one(std::uint64_t{1} << -M_plus.e, M_plus.e); + + auto p1 = static_cast(M_plus.f >> -one.e); // p1 = f div 2^-e (Since -e >= 32, p1 fits into a 32-bit int.) + std::uint64_t p2 = M_plus.f & (one.f - 1); // p2 = f mod 2^-e + + // 1) + // + // Generate the digits of the integral part p1 = d[n-1]...d[1]d[0] + + JSON_ASSERT(p1 > 0); + + std::uint32_t pow10; + const int k = find_largest_pow10(p1, pow10); + + // 10^(k-1) <= p1 < 10^k, pow10 = 10^(k-1) + // + // p1 = (p1 div 10^(k-1)) * 10^(k-1) + (p1 mod 10^(k-1)) + // = (d[k-1] ) * 10^(k-1) + (p1 mod 10^(k-1)) + // + // M+ = p1 + p2 * 2^e + // = d[k-1] * 10^(k-1) + (p1 mod 10^(k-1)) + p2 * 2^e + // = d[k-1] * 10^(k-1) + ((p1 mod 10^(k-1)) * 2^-e + p2) * 2^e + // = d[k-1] * 10^(k-1) + ( rest) * 2^e + // + // Now generate the digits d[n] of p1 from left to right (n = k-1,...,0) + // + // p1 = d[k-1]...d[n] * 10^n + d[n-1]...d[0] + // + // but stop as soon as + // + // rest * 2^e = (d[n-1]...d[0] * 2^-e + p2) * 2^e <= delta * 2^e + + int n = k; + while (n > 0) + { + // Invariants: + // M+ = buffer * 10^n + (p1 + p2 * 2^e) (buffer = 0 for n = k) + // pow10 = 10^(n-1) <= p1 < 10^n + // + const std::uint32_t d = p1 / pow10; // d = p1 div 10^(n-1) + const std::uint32_t r = p1 % pow10; // r = p1 mod 10^(n-1) + // + // M+ = buffer * 10^n + (d * 10^(n-1) + r) + p2 * 2^e + // = (buffer * 10 + d) * 10^(n-1) + (r + p2 * 2^e) + // + JSON_ASSERT(d <= 9); + buffer[length++] = static_cast('0' + d); // buffer := buffer * 10 + d + // + // M+ = buffer * 10^(n-1) + (r + p2 * 2^e) + // + p1 = r; + n--; + // + // M+ = buffer * 10^n + (p1 + p2 * 2^e) + // pow10 = 10^n + // + + // Now check if enough digits have been generated. + // Compute + // + // p1 + p2 * 2^e = (p1 * 2^-e + p2) * 2^e = rest * 2^e + // + // Note: + // Since rest and delta share the same exponent e, it suffices to + // compare the significands. + const std::uint64_t rest = (std::uint64_t{p1} << -one.e) + p2; + if (rest <= delta) + { + // V = buffer * 10^n, with M- <= V <= M+. + + decimal_exponent += n; + + // We may now just stop. But instead look if the buffer could be + // decremented to bring V closer to w. + // + // pow10 = 10^n is now 1 ulp in the decimal representation V. + // The rounding procedure works with diyfp's with an implicit + // exponent of e. + // + // 10^n = (10^n * 2^-e) * 2^e = ulp * 2^e + // + const std::uint64_t ten_n = std::uint64_t{pow10} << -one.e; + grisu2_round(buffer, length, dist, delta, rest, ten_n); + + return; + } + + pow10 /= 10; + // + // pow10 = 10^(n-1) <= p1 < 10^n + // Invariants restored. + } + + // 2) + // + // The digits of the integral part have been generated: + // + // M+ = d[k-1]...d[1]d[0] + p2 * 2^e + // = buffer + p2 * 2^e + // + // Now generate the digits of the fractional part p2 * 2^e. + // + // Note: + // No decimal point is generated: the exponent is adjusted instead. + // + // p2 actually represents the fraction + // + // p2 * 2^e + // = p2 / 2^-e + // = d[-1] / 10^1 + d[-2] / 10^2 + ... + // + // Now generate the digits d[-m] of p1 from left to right (m = 1,2,...) + // + // p2 * 2^e = d[-1]d[-2]...d[-m] * 10^-m + // + 10^-m * (d[-m-1] / 10^1 + d[-m-2] / 10^2 + ...) + // + // using + // + // 10^m * p2 = ((10^m * p2) div 2^-e) * 2^-e + ((10^m * p2) mod 2^-e) + // = ( d) * 2^-e + ( r) + // + // or + // 10^m * p2 * 2^e = d + r * 2^e + // + // i.e. + // + // M+ = buffer + p2 * 2^e + // = buffer + 10^-m * (d + r * 2^e) + // = (buffer * 10^m + d) * 10^-m + 10^-m * r * 2^e + // + // and stop as soon as 10^-m * r * 2^e <= delta * 2^e + + JSON_ASSERT(p2 > delta); + + int m = 0; + for (;;) + { + // Invariant: + // M+ = buffer * 10^-m + 10^-m * (d[-m-1] / 10 + d[-m-2] / 10^2 + ...) * 2^e + // = buffer * 10^-m + 10^-m * (p2 ) * 2^e + // = buffer * 10^-m + 10^-m * (1/10 * (10 * p2) ) * 2^e + // = buffer * 10^-m + 10^-m * (1/10 * ((10*p2 div 2^-e) * 2^-e + (10*p2 mod 2^-e)) * 2^e + // + JSON_ASSERT(p2 <= (std::numeric_limits::max)() / 10); + p2 *= 10; + const std::uint64_t d = p2 >> -one.e; // d = (10 * p2) div 2^-e + const std::uint64_t r = p2 & (one.f - 1); // r = (10 * p2) mod 2^-e + // + // M+ = buffer * 10^-m + 10^-m * (1/10 * (d * 2^-e + r) * 2^e + // = buffer * 10^-m + 10^-m * (1/10 * (d + r * 2^e)) + // = (buffer * 10 + d) * 10^(-m-1) + 10^(-m-1) * r * 2^e + // + JSON_ASSERT(d <= 9); + buffer[length++] = static_cast('0' + d); // buffer := buffer * 10 + d + // + // M+ = buffer * 10^(-m-1) + 10^(-m-1) * r * 2^e + // + p2 = r; + m++; + // + // M+ = buffer * 10^-m + 10^-m * p2 * 2^e + // Invariant restored. + + // Check if enough digits have been generated. + // + // 10^-m * p2 * 2^e <= delta * 2^e + // p2 * 2^e <= 10^m * delta * 2^e + // p2 <= 10^m * delta + delta *= 10; + dist *= 10; + if (p2 <= delta) + { + break; + } + } + + // V = buffer * 10^-m, with M- <= V <= M+. + + decimal_exponent -= m; + + // 1 ulp in the decimal representation is now 10^-m. + // Since delta and dist are now scaled by 10^m, we need to do the + // same with ulp in order to keep the units in sync. + // + // 10^m * 10^-m = 1 = 2^-e * 2^e = ten_m * 2^e + // + const std::uint64_t ten_m = one.f; + grisu2_round(buffer, length, dist, delta, p2, ten_m); + + // By construction this algorithm generates the shortest possible decimal + // number (Loitsch, Theorem 6.2) which rounds back to w. + // For an input number of precision p, at least + // + // N = 1 + ceil(p * log_10(2)) + // + // decimal digits are sufficient to identify all binary floating-point + // numbers (Matula, "In-and-Out conversions"). + // This implies that the algorithm does not produce more than N decimal + // digits. + // + // N = 17 for p = 53 (IEEE double precision) + // N = 9 for p = 24 (IEEE single precision) +} + +/*! +v = buf * 10^decimal_exponent +len is the length of the buffer (number of decimal digits) +The buffer must be large enough, i.e. >= max_digits10. +*/ +JSON_HEDLEY_NON_NULL(1) +inline void grisu2(char* buf, int& len, int& decimal_exponent, + diyfp m_minus, diyfp v, diyfp m_plus) +{ + JSON_ASSERT(m_plus.e == m_minus.e); + JSON_ASSERT(m_plus.e == v.e); + + // --------(-----------------------+-----------------------)-------- (A) + // m- v m+ + // + // --------------------(-----------+-----------------------)-------- (B) + // m- v m+ + // + // First scale v (and m- and m+) such that the exponent is in the range + // [alpha, gamma]. + + const cached_power cached = get_cached_power_for_binary_exponent(m_plus.e); + + const diyfp c_minus_k(cached.f, cached.e); // = c ~= 10^-k + + // The exponent of the products is = v.e + c_minus_k.e + q and is in the range [alpha,gamma] + const diyfp w = diyfp::mul(v, c_minus_k); + const diyfp w_minus = diyfp::mul(m_minus, c_minus_k); + const diyfp w_plus = diyfp::mul(m_plus, c_minus_k); + + // ----(---+---)---------------(---+---)---------------(---+---)---- + // w- w w+ + // = c*m- = c*v = c*m+ + // + // diyfp::mul rounds its result and c_minus_k is approximated too. w, w- and + // w+ are now off by a small amount. + // In fact: + // + // w - v * 10^k < 1 ulp + // + // To account for this inaccuracy, add resp. subtract 1 ulp. + // + // --------+---[---------------(---+---)---------------]---+-------- + // w- M- w M+ w+ + // + // Now any number in [M-, M+] (bounds included) will round to w when input, + // regardless of how the input rounding algorithm breaks ties. + // + // And digit_gen generates the shortest possible such number in [M-, M+]. + // Note that this does not mean that Grisu2 always generates the shortest + // possible number in the interval (m-, m+). + const diyfp M_minus(w_minus.f + 1, w_minus.e); + const diyfp M_plus (w_plus.f - 1, w_plus.e ); + + decimal_exponent = -cached.k; // = -(-k) = k + + grisu2_digit_gen(buf, len, decimal_exponent, M_minus, w, M_plus); +} + +/*! +v = buf * 10^decimal_exponent +len is the length of the buffer (number of decimal digits) +The buffer must be large enough, i.e. >= max_digits10. +*/ +template +JSON_HEDLEY_NON_NULL(1) +void grisu2(char* buf, int& len, int& decimal_exponent, FloatType value) +{ + static_assert(diyfp::kPrecision >= std::numeric_limits::digits + 3, + "internal error: not enough precision"); + + JSON_ASSERT(std::isfinite(value)); + JSON_ASSERT(value > 0); + + // If the neighbors (and boundaries) of 'value' are always computed for double-precision + // numbers, all float's can be recovered using strtod (and strtof). However, the resulting + // decimal representations are not exactly "short". + // + // The documentation for 'std::to_chars' (https://en.cppreference.com/w/cpp/utility/to_chars) + // says "value is converted to a string as if by std::sprintf in the default ("C") locale" + // and since sprintf promotes float's to double's, I think this is exactly what 'std::to_chars' + // does. + // On the other hand, the documentation for 'std::to_chars' requires that "parsing the + // representation using the corresponding std::from_chars function recovers value exactly". That + // indicates that single precision floating-point numbers should be recovered using + // 'std::strtof'. + // + // NB: If the neighbors are computed for single-precision numbers, there is a single float + // (7.0385307e-26f) which can't be recovered using strtod. The resulting double precision + // value is off by 1 ulp. +#if 0 + const boundaries w = compute_boundaries(static_cast(value)); +#else + const boundaries w = compute_boundaries(value); +#endif + + grisu2(buf, len, decimal_exponent, w.minus, w.w, w.plus); +} + +/*! +@brief appends a decimal representation of e to buf +@return a pointer to the element following the exponent. +@pre -1000 < e < 1000 +*/ +JSON_HEDLEY_NON_NULL(1) +JSON_HEDLEY_RETURNS_NON_NULL +inline char* append_exponent(char* buf, int e) +{ + JSON_ASSERT(e > -1000); + JSON_ASSERT(e < 1000); + + if (e < 0) + { + e = -e; + *buf++ = '-'; + } + else + { + *buf++ = '+'; + } + + auto k = static_cast(e); + if (k < 10) + { + // Always print at least two digits in the exponent. + // This is for compatibility with printf("%g"). + *buf++ = '0'; + *buf++ = static_cast('0' + k); + } + else if (k < 100) + { + *buf++ = static_cast('0' + k / 10); + k %= 10; + *buf++ = static_cast('0' + k); + } + else + { + *buf++ = static_cast('0' + k / 100); + k %= 100; + *buf++ = static_cast('0' + k / 10); + k %= 10; + *buf++ = static_cast('0' + k); + } + + return buf; +} + +/*! +@brief prettify v = buf * 10^decimal_exponent + +If v is in the range [10^min_exp, 10^max_exp) it will be printed in fixed-point +notation. Otherwise it will be printed in exponential notation. + +@pre min_exp < 0 +@pre max_exp > 0 +*/ +JSON_HEDLEY_NON_NULL(1) +JSON_HEDLEY_RETURNS_NON_NULL +inline char* format_buffer(char* buf, int len, int decimal_exponent, + int min_exp, int max_exp) +{ + JSON_ASSERT(min_exp < 0); + JSON_ASSERT(max_exp > 0); + + const int k = len; + const int n = len + decimal_exponent; + + // v = buf * 10^(n-k) + // k is the length of the buffer (number of decimal digits) + // n is the position of the decimal point relative to the start of the buffer. + + if (k <= n && n <= max_exp) + { + // digits[000] + // len <= max_exp + 2 + + std::memset(buf + k, '0', static_cast(n) - static_cast(k)); + // Make it look like a floating-point number (#362, #378) + buf[n + 0] = '.'; + buf[n + 1] = '0'; + return buf + (static_cast(n) + 2); + } + + if (0 < n && n <= max_exp) + { + // dig.its + // len <= max_digits10 + 1 + + JSON_ASSERT(k > n); + + std::memmove(buf + (static_cast(n) + 1), buf + n, static_cast(k) - static_cast(n)); + buf[n] = '.'; + return buf + (static_cast(k) + 1U); + } + + if (min_exp < n && n <= 0) + { + // 0.[000]digits + // len <= 2 + (-min_exp - 1) + max_digits10 + + std::memmove(buf + (2 + static_cast(-n)), buf, static_cast(k)); + buf[0] = '0'; + buf[1] = '.'; + std::memset(buf + 2, '0', static_cast(-n)); + return buf + (2U + static_cast(-n) + static_cast(k)); + } + + if (k == 1) + { + // dE+123 + // len <= 1 + 5 + + buf += 1; + } + else + { + // d.igitsE+123 + // len <= max_digits10 + 1 + 5 + + std::memmove(buf + 2, buf + 1, static_cast(k) - 1); + buf[1] = '.'; + buf += 1 + static_cast(k); + } + + *buf++ = 'e'; + return append_exponent(buf, n - 1); +} + +} // namespace dtoa_impl + +/*! +@brief generates a decimal representation of the floating-point number value in [first, last). + +The format of the resulting decimal representation is similar to printf's %g +format. Returns an iterator pointing past-the-end of the decimal representation. + +@note The input number must be finite, i.e. NaN's and Inf's are not supported. +@note The buffer must be large enough. +@note The result is NOT null-terminated. +*/ +template +JSON_HEDLEY_NON_NULL(1, 2) +JSON_HEDLEY_RETURNS_NON_NULL +char* to_chars(char* first, const char* last, FloatType value) +{ + static_cast(last); // maybe unused - fix warning + JSON_ASSERT(std::isfinite(value)); + + // Use signbit(value) instead of (value < 0) since signbit works for -0. + if (std::signbit(value)) + { + value = -value; + *first++ = '-'; + } + + if (value == 0) // +-0 + { + *first++ = '0'; + // Make it look like a floating-point number (#362, #378) + *first++ = '.'; + *first++ = '0'; + return first; + } + + JSON_ASSERT(last - first >= std::numeric_limits::max_digits10); + + // Compute v = buffer * 10^decimal_exponent. + // The decimal digits are stored in the buffer, which needs to be interpreted + // as an unsigned decimal integer. + // len is the length of the buffer, i.e. the number of decimal digits. + int len = 0; + int decimal_exponent = 0; + dtoa_impl::grisu2(first, len, decimal_exponent, value); + + JSON_ASSERT(len <= std::numeric_limits::max_digits10); + + // Format the buffer like printf("%.*g", prec, value) + constexpr int kMinExp = -4; + // Use digits10 here to increase compatibility with version 2. + constexpr int kMaxExp = std::numeric_limits::digits10; + + JSON_ASSERT(last - first >= kMaxExp + 2); + JSON_ASSERT(last - first >= 2 + (-kMinExp - 1) + std::numeric_limits::max_digits10); + JSON_ASSERT(last - first >= std::numeric_limits::max_digits10 + 6); + + return dtoa_impl::format_buffer(first, len, decimal_exponent, kMinExp, kMaxExp); +} + +} // namespace detail +} // namespace nlohmann + +// #include + +// #include + +// #include + +// #include + +// #include + +// #include + + +namespace nlohmann +{ +namespace detail +{ +/////////////////// +// serialization // +/////////////////// + +/// how to treat decoding errors +enum class error_handler_t +{ + strict, ///< throw a type_error exception in case of invalid UTF-8 + replace, ///< replace invalid UTF-8 sequences with U+FFFD + ignore ///< ignore invalid UTF-8 sequences +}; + +template +class serializer +{ + using string_t = typename BasicJsonType::string_t; + using number_float_t = typename BasicJsonType::number_float_t; + using number_integer_t = typename BasicJsonType::number_integer_t; + using number_unsigned_t = typename BasicJsonType::number_unsigned_t; + using binary_char_t = typename BasicJsonType::binary_t::value_type; + static constexpr std::uint8_t UTF8_ACCEPT = 0; + static constexpr std::uint8_t UTF8_REJECT = 1; + + public: + /*! + @param[in] s output stream to serialize to + @param[in] ichar indentation character to use + @param[in] error_handler_ how to react on decoding errors + */ + serializer(output_adapter_t s, const char ichar, + error_handler_t error_handler_ = error_handler_t::strict) + : o(std::move(s)) + , loc(std::localeconv()) + , thousands_sep(loc->thousands_sep == nullptr ? '\0' : std::char_traits::to_char_type(* (loc->thousands_sep))) + , decimal_point(loc->decimal_point == nullptr ? '\0' : std::char_traits::to_char_type(* (loc->decimal_point))) + , indent_char(ichar) + , indent_string(512, indent_char) + , error_handler(error_handler_) + {} + + // delete because of pointer members + serializer(const serializer&) = delete; + serializer& operator=(const serializer&) = delete; + serializer(serializer&&) = delete; + serializer& operator=(serializer&&) = delete; + ~serializer() = default; + + /*! + @brief internal implementation of the serialization function + + This function is called by the public member function dump and organizes + the serialization internally. The indentation level is propagated as + additional parameter. In case of arrays and objects, the function is + called recursively. + + - strings and object keys are escaped using `escape_string()` + - integer numbers are converted implicitly via `operator<<` + - floating-point numbers are converted to a string using `"%g"` format + - binary values are serialized as objects containing the subtype and the + byte array + + @param[in] val value to serialize + @param[in] pretty_print whether the output shall be pretty-printed + @param[in] ensure_ascii If @a ensure_ascii is true, all non-ASCII characters + in the output are escaped with `\uXXXX` sequences, and the result consists + of ASCII characters only. + @param[in] indent_step the indent level + @param[in] current_indent the current indent level (only used internally) + */ + void dump(const BasicJsonType& val, + const bool pretty_print, + const bool ensure_ascii, + const unsigned int indent_step, + const unsigned int current_indent = 0) + { + switch (val.m_type) + { + case value_t::object: + { + if (val.m_value.object->empty()) + { + o->write_characters("{}", 2); + return; + } + + if (pretty_print) + { + o->write_characters("{\n", 2); + + // variable to hold indentation for recursive calls + const auto new_indent = current_indent + indent_step; + if (JSON_HEDLEY_UNLIKELY(indent_string.size() < new_indent)) + { + indent_string.resize(indent_string.size() * 2, ' '); + } + + // first n-1 elements + auto i = val.m_value.object->cbegin(); + for (std::size_t cnt = 0; cnt < val.m_value.object->size() - 1; ++cnt, ++i) + { + o->write_characters(indent_string.c_str(), new_indent); + o->write_character('\"'); + dump_escaped(i->first, ensure_ascii); + o->write_characters("\": ", 3); + dump(i->second, true, ensure_ascii, indent_step, new_indent); + o->write_characters(",\n", 2); + } + + // last element + JSON_ASSERT(i != val.m_value.object->cend()); + JSON_ASSERT(std::next(i) == val.m_value.object->cend()); + o->write_characters(indent_string.c_str(), new_indent); + o->write_character('\"'); + dump_escaped(i->first, ensure_ascii); + o->write_characters("\": ", 3); + dump(i->second, true, ensure_ascii, indent_step, new_indent); + + o->write_character('\n'); + o->write_characters(indent_string.c_str(), current_indent); + o->write_character('}'); + } + else + { + o->write_character('{'); + + // first n-1 elements + auto i = val.m_value.object->cbegin(); + for (std::size_t cnt = 0; cnt < val.m_value.object->size() - 1; ++cnt, ++i) + { + o->write_character('\"'); + dump_escaped(i->first, ensure_ascii); + o->write_characters("\":", 2); + dump(i->second, false, ensure_ascii, indent_step, current_indent); + o->write_character(','); + } + + // last element + JSON_ASSERT(i != val.m_value.object->cend()); + JSON_ASSERT(std::next(i) == val.m_value.object->cend()); + o->write_character('\"'); + dump_escaped(i->first, ensure_ascii); + o->write_characters("\":", 2); + dump(i->second, false, ensure_ascii, indent_step, current_indent); + + o->write_character('}'); + } + + return; + } + + case value_t::array: + { + if (val.m_value.array->empty()) + { + o->write_characters("[]", 2); + return; + } + + if (pretty_print) + { + o->write_characters("[\n", 2); + + // variable to hold indentation for recursive calls + const auto new_indent = current_indent + indent_step; + if (JSON_HEDLEY_UNLIKELY(indent_string.size() < new_indent)) + { + indent_string.resize(indent_string.size() * 2, ' '); + } + + // first n-1 elements + for (auto i = val.m_value.array->cbegin(); + i != val.m_value.array->cend() - 1; ++i) + { + o->write_characters(indent_string.c_str(), new_indent); + dump(*i, true, ensure_ascii, indent_step, new_indent); + o->write_characters(",\n", 2); + } + + // last element + JSON_ASSERT(!val.m_value.array->empty()); + o->write_characters(indent_string.c_str(), new_indent); + dump(val.m_value.array->back(), true, ensure_ascii, indent_step, new_indent); + + o->write_character('\n'); + o->write_characters(indent_string.c_str(), current_indent); + o->write_character(']'); + } + else + { + o->write_character('['); + + // first n-1 elements + for (auto i = val.m_value.array->cbegin(); + i != val.m_value.array->cend() - 1; ++i) + { + dump(*i, false, ensure_ascii, indent_step, current_indent); + o->write_character(','); + } + + // last element + JSON_ASSERT(!val.m_value.array->empty()); + dump(val.m_value.array->back(), false, ensure_ascii, indent_step, current_indent); + + o->write_character(']'); + } + + return; + } + + case value_t::string: + { + o->write_character('\"'); + dump_escaped(*val.m_value.string, ensure_ascii); + o->write_character('\"'); + return; + } + + case value_t::binary: + { + if (pretty_print) + { + o->write_characters("{\n", 2); + + // variable to hold indentation for recursive calls + const auto new_indent = current_indent + indent_step; + if (JSON_HEDLEY_UNLIKELY(indent_string.size() < new_indent)) + { + indent_string.resize(indent_string.size() * 2, ' '); + } + + o->write_characters(indent_string.c_str(), new_indent); + + o->write_characters("\"bytes\": [", 10); + + if (!val.m_value.binary->empty()) + { + for (auto i = val.m_value.binary->cbegin(); + i != val.m_value.binary->cend() - 1; ++i) + { + dump_integer(*i); + o->write_characters(", ", 2); + } + dump_integer(val.m_value.binary->back()); + } + + o->write_characters("],\n", 3); + o->write_characters(indent_string.c_str(), new_indent); + + o->write_characters("\"subtype\": ", 11); + if (val.m_value.binary->has_subtype()) + { + dump_integer(val.m_value.binary->subtype()); + } + else + { + o->write_characters("null", 4); + } + o->write_character('\n'); + o->write_characters(indent_string.c_str(), current_indent); + o->write_character('}'); + } + else + { + o->write_characters("{\"bytes\":[", 10); + + if (!val.m_value.binary->empty()) + { + for (auto i = val.m_value.binary->cbegin(); + i != val.m_value.binary->cend() - 1; ++i) + { + dump_integer(*i); + o->write_character(','); + } + dump_integer(val.m_value.binary->back()); + } + + o->write_characters("],\"subtype\":", 12); + if (val.m_value.binary->has_subtype()) + { + dump_integer(val.m_value.binary->subtype()); + o->write_character('}'); + } + else + { + o->write_characters("null}", 5); + } + } + return; + } + + case value_t::boolean: + { + if (val.m_value.boolean) + { + o->write_characters("true", 4); + } + else + { + o->write_characters("false", 5); + } + return; + } + + case value_t::number_integer: + { + dump_integer(val.m_value.number_integer); + return; + } + + case value_t::number_unsigned: + { + dump_integer(val.m_value.number_unsigned); + return; + } + + case value_t::number_float: + { + dump_float(val.m_value.number_float); + return; + } + + case value_t::discarded: + { + o->write_characters("", 11); + return; + } + + case value_t::null: + { + o->write_characters("null", 4); + return; + } + + default: // LCOV_EXCL_LINE + JSON_ASSERT(false); // LCOV_EXCL_LINE + } + } + + private: + /*! + @brief dump escaped string + + Escape a string by replacing certain special characters by a sequence of an + escape character (backslash) and another character and other control + characters by a sequence of "\u" followed by a four-digit hex + representation. The escaped string is written to output stream @a o. + + @param[in] s the string to escape + @param[in] ensure_ascii whether to escape non-ASCII characters with + \uXXXX sequences + + @complexity Linear in the length of string @a s. + */ + void dump_escaped(const string_t& s, const bool ensure_ascii) + { + std::uint32_t codepoint; + std::uint8_t state = UTF8_ACCEPT; + std::size_t bytes = 0; // number of bytes written to string_buffer + + // number of bytes written at the point of the last valid byte + std::size_t bytes_after_last_accept = 0; + std::size_t undumped_chars = 0; + + for (std::size_t i = 0; i < s.size(); ++i) + { + const auto byte = static_cast(s[i]); + + switch (decode(state, codepoint, byte)) + { + case UTF8_ACCEPT: // decode found a new code point + { + switch (codepoint) + { + case 0x08: // backspace + { + string_buffer[bytes++] = '\\'; + string_buffer[bytes++] = 'b'; + break; + } + + case 0x09: // horizontal tab + { + string_buffer[bytes++] = '\\'; + string_buffer[bytes++] = 't'; + break; + } + + case 0x0A: // newline + { + string_buffer[bytes++] = '\\'; + string_buffer[bytes++] = 'n'; + break; + } + + case 0x0C: // formfeed + { + string_buffer[bytes++] = '\\'; + string_buffer[bytes++] = 'f'; + break; + } + + case 0x0D: // carriage return + { + string_buffer[bytes++] = '\\'; + string_buffer[bytes++] = 'r'; + break; + } + + case 0x22: // quotation mark + { + string_buffer[bytes++] = '\\'; + string_buffer[bytes++] = '\"'; + break; + } + + case 0x5C: // reverse solidus + { + string_buffer[bytes++] = '\\'; + string_buffer[bytes++] = '\\'; + break; + } + + default: + { + // escape control characters (0x00..0x1F) or, if + // ensure_ascii parameter is used, non-ASCII characters + if ((codepoint <= 0x1F) || (ensure_ascii && (codepoint >= 0x7F))) + { + if (codepoint <= 0xFFFF) + { + (std::snprintf)(string_buffer.data() + bytes, 7, "\\u%04x", + static_cast(codepoint)); + bytes += 6; + } + else + { + (std::snprintf)(string_buffer.data() + bytes, 13, "\\u%04x\\u%04x", + static_cast(0xD7C0u + (codepoint >> 10u)), + static_cast(0xDC00u + (codepoint & 0x3FFu))); + bytes += 12; + } + } + else + { + // copy byte to buffer (all previous bytes + // been copied have in default case above) + string_buffer[bytes++] = s[i]; + } + break; + } + } + + // write buffer and reset index; there must be 13 bytes + // left, as this is the maximal number of bytes to be + // written ("\uxxxx\uxxxx\0") for one code point + if (string_buffer.size() - bytes < 13) + { + o->write_characters(string_buffer.data(), bytes); + bytes = 0; + } + + // remember the byte position of this accept + bytes_after_last_accept = bytes; + undumped_chars = 0; + break; + } + + case UTF8_REJECT: // decode found invalid UTF-8 byte + { + switch (error_handler) + { + case error_handler_t::strict: + { + std::string sn(3, '\0'); + (std::snprintf)(&sn[0], sn.size(), "%.2X", byte); + JSON_THROW(type_error::create(316, "invalid UTF-8 byte at index " + std::to_string(i) + ": 0x" + sn)); + } + + case error_handler_t::ignore: + case error_handler_t::replace: + { + // in case we saw this character the first time, we + // would like to read it again, because the byte + // may be OK for itself, but just not OK for the + // previous sequence + if (undumped_chars > 0) + { + --i; + } + + // reset length buffer to the last accepted index; + // thus removing/ignoring the invalid characters + bytes = bytes_after_last_accept; + + if (error_handler == error_handler_t::replace) + { + // add a replacement character + if (ensure_ascii) + { + string_buffer[bytes++] = '\\'; + string_buffer[bytes++] = 'u'; + string_buffer[bytes++] = 'f'; + string_buffer[bytes++] = 'f'; + string_buffer[bytes++] = 'f'; + string_buffer[bytes++] = 'd'; + } + else + { + string_buffer[bytes++] = detail::binary_writer::to_char_type('\xEF'); + string_buffer[bytes++] = detail::binary_writer::to_char_type('\xBF'); + string_buffer[bytes++] = detail::binary_writer::to_char_type('\xBD'); + } + + // write buffer and reset index; there must be 13 bytes + // left, as this is the maximal number of bytes to be + // written ("\uxxxx\uxxxx\0") for one code point + if (string_buffer.size() - bytes < 13) + { + o->write_characters(string_buffer.data(), bytes); + bytes = 0; + } + + bytes_after_last_accept = bytes; + } + + undumped_chars = 0; + + // continue processing the string + state = UTF8_ACCEPT; + break; + } + + default: // LCOV_EXCL_LINE + JSON_ASSERT(false); // LCOV_EXCL_LINE + } + break; + } + + default: // decode found yet incomplete multi-byte code point + { + if (!ensure_ascii) + { + // code point will not be escaped - copy byte to buffer + string_buffer[bytes++] = s[i]; + } + ++undumped_chars; + break; + } + } + } + + // we finished processing the string + if (JSON_HEDLEY_LIKELY(state == UTF8_ACCEPT)) + { + // write buffer + if (bytes > 0) + { + o->write_characters(string_buffer.data(), bytes); + } + } + else + { + // we finish reading, but do not accept: string was incomplete + switch (error_handler) + { + case error_handler_t::strict: + { + std::string sn(3, '\0'); + (std::snprintf)(&sn[0], sn.size(), "%.2X", static_cast(s.back())); + JSON_THROW(type_error::create(316, "incomplete UTF-8 string; last byte: 0x" + sn)); + } + + case error_handler_t::ignore: + { + // write all accepted bytes + o->write_characters(string_buffer.data(), bytes_after_last_accept); + break; + } + + case error_handler_t::replace: + { + // write all accepted bytes + o->write_characters(string_buffer.data(), bytes_after_last_accept); + // add a replacement character + if (ensure_ascii) + { + o->write_characters("\\ufffd", 6); + } + else + { + o->write_characters("\xEF\xBF\xBD", 3); + } + break; + } + + default: // LCOV_EXCL_LINE + JSON_ASSERT(false); // LCOV_EXCL_LINE + } + } + } + + /*! + @brief count digits + + Count the number of decimal (base 10) digits for an input unsigned integer. + + @param[in] x unsigned integer number to count its digits + @return number of decimal digits + */ + inline unsigned int count_digits(number_unsigned_t x) noexcept + { + unsigned int n_digits = 1; + for (;;) + { + if (x < 10) + { + return n_digits; + } + if (x < 100) + { + return n_digits + 1; + } + if (x < 1000) + { + return n_digits + 2; + } + if (x < 10000) + { + return n_digits + 3; + } + x = x / 10000u; + n_digits += 4; + } + } + + /*! + @brief dump an integer + + Dump a given integer to output stream @a o. Works internally with + @a number_buffer. + + @param[in] x integer number (signed or unsigned) to dump + @tparam NumberType either @a number_integer_t or @a number_unsigned_t + */ + template < typename NumberType, detail::enable_if_t < + std::is_same::value || + std::is_same::value || + std::is_same::value, + int > = 0 > + void dump_integer(NumberType x) + { + static constexpr std::array, 100> digits_to_99 + { + { + {{'0', '0'}}, {{'0', '1'}}, {{'0', '2'}}, {{'0', '3'}}, {{'0', '4'}}, {{'0', '5'}}, {{'0', '6'}}, {{'0', '7'}}, {{'0', '8'}}, {{'0', '9'}}, + {{'1', '0'}}, {{'1', '1'}}, {{'1', '2'}}, {{'1', '3'}}, {{'1', '4'}}, {{'1', '5'}}, {{'1', '6'}}, {{'1', '7'}}, {{'1', '8'}}, {{'1', '9'}}, + {{'2', '0'}}, {{'2', '1'}}, {{'2', '2'}}, {{'2', '3'}}, {{'2', '4'}}, {{'2', '5'}}, {{'2', '6'}}, {{'2', '7'}}, {{'2', '8'}}, {{'2', '9'}}, + {{'3', '0'}}, {{'3', '1'}}, {{'3', '2'}}, {{'3', '3'}}, {{'3', '4'}}, {{'3', '5'}}, {{'3', '6'}}, {{'3', '7'}}, {{'3', '8'}}, {{'3', '9'}}, + {{'4', '0'}}, {{'4', '1'}}, {{'4', '2'}}, {{'4', '3'}}, {{'4', '4'}}, {{'4', '5'}}, {{'4', '6'}}, {{'4', '7'}}, {{'4', '8'}}, {{'4', '9'}}, + {{'5', '0'}}, {{'5', '1'}}, {{'5', '2'}}, {{'5', '3'}}, {{'5', '4'}}, {{'5', '5'}}, {{'5', '6'}}, {{'5', '7'}}, {{'5', '8'}}, {{'5', '9'}}, + {{'6', '0'}}, {{'6', '1'}}, {{'6', '2'}}, {{'6', '3'}}, {{'6', '4'}}, {{'6', '5'}}, {{'6', '6'}}, {{'6', '7'}}, {{'6', '8'}}, {{'6', '9'}}, + {{'7', '0'}}, {{'7', '1'}}, {{'7', '2'}}, {{'7', '3'}}, {{'7', '4'}}, {{'7', '5'}}, {{'7', '6'}}, {{'7', '7'}}, {{'7', '8'}}, {{'7', '9'}}, + {{'8', '0'}}, {{'8', '1'}}, {{'8', '2'}}, {{'8', '3'}}, {{'8', '4'}}, {{'8', '5'}}, {{'8', '6'}}, {{'8', '7'}}, {{'8', '8'}}, {{'8', '9'}}, + {{'9', '0'}}, {{'9', '1'}}, {{'9', '2'}}, {{'9', '3'}}, {{'9', '4'}}, {{'9', '5'}}, {{'9', '6'}}, {{'9', '7'}}, {{'9', '8'}}, {{'9', '9'}}, + } + }; + + // special case for "0" + if (x == 0) + { + o->write_character('0'); + return; + } + + // use a pointer to fill the buffer + auto buffer_ptr = number_buffer.begin(); + + const bool is_negative = std::is_same::value && !(x >= 0); // see issue #755 + number_unsigned_t abs_value; + + unsigned int n_chars; + + if (is_negative) + { + *buffer_ptr = '-'; + abs_value = remove_sign(static_cast(x)); + + // account one more byte for the minus sign + n_chars = 1 + count_digits(abs_value); + } + else + { + abs_value = static_cast(x); + n_chars = count_digits(abs_value); + } + + // spare 1 byte for '\0' + JSON_ASSERT(n_chars < number_buffer.size() - 1); + + // jump to the end to generate the string from backward + // so we later avoid reversing the result + buffer_ptr += n_chars; + + // Fast int2ascii implementation inspired by "Fastware" talk by Andrei Alexandrescu + // See: https://www.youtube.com/watch?v=o4-CwDo2zpg + while (abs_value >= 100) + { + const auto digits_index = static_cast((abs_value % 100)); + abs_value /= 100; + *(--buffer_ptr) = digits_to_99[digits_index][1]; + *(--buffer_ptr) = digits_to_99[digits_index][0]; + } + + if (abs_value >= 10) + { + const auto digits_index = static_cast(abs_value); + *(--buffer_ptr) = digits_to_99[digits_index][1]; + *(--buffer_ptr) = digits_to_99[digits_index][0]; + } + else + { + *(--buffer_ptr) = static_cast('0' + abs_value); + } + + o->write_characters(number_buffer.data(), n_chars); + } + + /*! + @brief dump a floating-point number + + Dump a given floating-point number to output stream @a o. Works internally + with @a number_buffer. + + @param[in] x floating-point number to dump + */ + void dump_float(number_float_t x) + { + // NaN / inf + if (!std::isfinite(x)) + { + o->write_characters("null", 4); + return; + } + + // If number_float_t is an IEEE-754 single or double precision number, + // use the Grisu2 algorithm to produce short numbers which are + // guaranteed to round-trip, using strtof and strtod, resp. + // + // NB: The test below works if == . + static constexpr bool is_ieee_single_or_double + = (std::numeric_limits::is_iec559 && std::numeric_limits::digits == 24 && std::numeric_limits::max_exponent == 128) || + (std::numeric_limits::is_iec559 && std::numeric_limits::digits == 53 && std::numeric_limits::max_exponent == 1024); + + dump_float(x, std::integral_constant()); + } + + void dump_float(number_float_t x, std::true_type /*is_ieee_single_or_double*/) + { + char* begin = number_buffer.data(); + char* end = ::nlohmann::detail::to_chars(begin, begin + number_buffer.size(), x); + + o->write_characters(begin, static_cast(end - begin)); + } + + void dump_float(number_float_t x, std::false_type /*is_ieee_single_or_double*/) + { + // get number of digits for a float -> text -> float round-trip + static constexpr auto d = std::numeric_limits::max_digits10; + + // the actual conversion + std::ptrdiff_t len = (std::snprintf)(number_buffer.data(), number_buffer.size(), "%.*g", d, x); + + // negative value indicates an error + JSON_ASSERT(len > 0); + // check if buffer was large enough + JSON_ASSERT(static_cast(len) < number_buffer.size()); + + // erase thousands separator + if (thousands_sep != '\0') + { + const auto end = std::remove(number_buffer.begin(), + number_buffer.begin() + len, thousands_sep); + std::fill(end, number_buffer.end(), '\0'); + JSON_ASSERT((end - number_buffer.begin()) <= len); + len = (end - number_buffer.begin()); + } + + // convert decimal point to '.' + if (decimal_point != '\0' && decimal_point != '.') + { + const auto dec_pos = std::find(number_buffer.begin(), number_buffer.end(), decimal_point); + if (dec_pos != number_buffer.end()) + { + *dec_pos = '.'; + } + } + + o->write_characters(number_buffer.data(), static_cast(len)); + + // determine if need to append ".0" + const bool value_is_int_like = + std::none_of(number_buffer.begin(), number_buffer.begin() + len + 1, + [](char c) + { + return c == '.' || c == 'e'; + }); + + if (value_is_int_like) + { + o->write_characters(".0", 2); + } + } + + /*! + @brief check whether a string is UTF-8 encoded + + The function checks each byte of a string whether it is UTF-8 encoded. The + result of the check is stored in the @a state parameter. The function must + be called initially with state 0 (accept). State 1 means the string must + be rejected, because the current byte is not allowed. If the string is + completely processed, but the state is non-zero, the string ended + prematurely; that is, the last byte indicated more bytes should have + followed. + + @param[in,out] state the state of the decoding + @param[in,out] codep codepoint (valid only if resulting state is UTF8_ACCEPT) + @param[in] byte next byte to decode + @return new state + + @note The function has been edited: a std::array is used. + + @copyright Copyright (c) 2008-2009 Bjoern Hoehrmann + @sa http://bjoern.hoehrmann.de/utf-8/decoder/dfa/ + */ + static std::uint8_t decode(std::uint8_t& state, std::uint32_t& codep, const std::uint8_t byte) noexcept + { + static const std::array utf8d = + { + { + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 00..1F + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 20..3F + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 40..5F + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 60..7F + 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, // 80..9F + 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, // A0..BF + 8, 8, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, // C0..DF + 0xA, 0x3, 0x3, 0x3, 0x3, 0x3, 0x3, 0x3, 0x3, 0x3, 0x3, 0x3, 0x3, 0x4, 0x3, 0x3, // E0..EF + 0xB, 0x6, 0x6, 0x6, 0x5, 0x8, 0x8, 0x8, 0x8, 0x8, 0x8, 0x8, 0x8, 0x8, 0x8, 0x8, // F0..FF + 0x0, 0x1, 0x2, 0x3, 0x5, 0x8, 0x7, 0x1, 0x1, 0x1, 0x4, 0x6, 0x1, 0x1, 0x1, 0x1, // s0..s0 + 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, // s1..s2 + 1, 2, 1, 1, 1, 1, 1, 2, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, // s3..s4 + 1, 2, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 1, 3, 1, 1, 1, 1, 1, 1, // s5..s6 + 1, 3, 1, 1, 1, 1, 1, 3, 1, 3, 1, 1, 1, 1, 1, 1, 1, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 // s7..s8 + } + }; + + const std::uint8_t type = utf8d[byte]; + + codep = (state != UTF8_ACCEPT) + ? (byte & 0x3fu) | (codep << 6u) + : (0xFFu >> type) & (byte); + + std::size_t index = 256u + static_cast(state) * 16u + static_cast(type); + JSON_ASSERT(index < 400); + state = utf8d[index]; + return state; + } + + /* + * Overload to make the compiler happy while it is instantiating + * dump_integer for number_unsigned_t. + * Must never be called. + */ + number_unsigned_t remove_sign(number_unsigned_t x) + { + JSON_ASSERT(false); // LCOV_EXCL_LINE + return x; // LCOV_EXCL_LINE + } + + /* + * Helper function for dump_integer + * + * This function takes a negative signed integer and returns its absolute + * value as unsigned integer. The plus/minus shuffling is necessary as we can + * not directly remove the sign of an arbitrary signed integer as the + * absolute values of INT_MIN and INT_MAX are usually not the same. See + * #1708 for details. + */ + inline number_unsigned_t remove_sign(number_integer_t x) noexcept + { + JSON_ASSERT(x < 0 && x < (std::numeric_limits::max)()); + return static_cast(-(x + 1)) + 1; + } + + private: + /// the output of the serializer + output_adapter_t o = nullptr; + + /// a (hopefully) large enough character buffer + std::array number_buffer{{}}; + + /// the locale + const std::lconv* loc = nullptr; + /// the locale's thousand separator character + const char thousands_sep = '\0'; + /// the locale's decimal point character + const char decimal_point = '\0'; + + /// string buffer + std::array string_buffer{{}}; + + /// the indentation character + const char indent_char; + /// the indentation string + string_t indent_string; + + /// error_handler how to react on decoding errors + const error_handler_t error_handler; +}; +} // namespace detail +} // namespace nlohmann + +// #include + +// #include + +// #include + + +#include // less +#include // allocator +#include // pair +#include // vector + +namespace nlohmann +{ + +/// ordered_map: a minimal map-like container that preserves insertion order +/// for use within nlohmann::basic_json +template , + class Allocator = std::allocator>> + struct ordered_map : std::vector, Allocator> +{ + using key_type = Key; + using mapped_type = T; + using Container = std::vector, Allocator>; + using typename Container::iterator; + using typename Container::const_iterator; + using typename Container::size_type; + using typename Container::value_type; + + // Explicit constructors instead of `using Container::Container` + // otherwise older compilers choke on it (GCC <= 5.5, xcode <= 9.4) + ordered_map(const Allocator& alloc = Allocator()) : Container{alloc} {} + template + ordered_map(It first, It last, const Allocator& alloc = Allocator()) + : Container{first, last, alloc} {} + ordered_map(std::initializer_list init, const Allocator& alloc = Allocator() ) + : Container{init, alloc} {} + + std::pair emplace(const key_type& key, T&& t) + { + for (auto it = this->begin(); it != this->end(); ++it) + { + if (it->first == key) + { + return {it, false}; + } + } + Container::emplace_back(key, t); + return {--this->end(), true}; + } + + T& operator[](const Key& key) + { + return emplace(key, T{}).first->second; + } + + const T& operator[](const Key& key) const + { + return at(key); + } + + T& at(const Key& key) + { + for (auto it = this->begin(); it != this->end(); ++it) + { + if (it->first == key) + { + return it->second; + } + } + + throw std::out_of_range("key not found"); + } + + const T& at(const Key& key) const + { + for (auto it = this->begin(); it != this->end(); ++it) + { + if (it->first == key) + { + return it->second; + } + } + + throw std::out_of_range("key not found"); + } + + size_type erase(const Key& key) + { + for (auto it = this->begin(); it != this->end(); ++it) + { + if (it->first == key) + { + // Since we cannot move const Keys, re-construct them in place + for (auto next = it; ++next != this->end(); ++it) + { + it->~value_type(); // Destroy but keep allocation + new (&*it) value_type{std::move(*next)}; + } + Container::pop_back(); + return 1; + } + } + return 0; + } + + iterator erase(iterator pos) + { + auto it = pos; + + // Since we cannot move const Keys, re-construct them in place + for (auto next = it; ++next != this->end(); ++it) + { + it->~value_type(); // Destroy but keep allocation + new (&*it) value_type{std::move(*next)}; + } + Container::pop_back(); + return pos; + } + + size_type count(const Key& key) const + { + for (auto it = this->begin(); it != this->end(); ++it) + { + if (it->first == key) + { + return 1; + } + } + return 0; + } + + iterator find(const Key& key) + { + for (auto it = this->begin(); it != this->end(); ++it) + { + if (it->first == key) + { + return it; + } + } + return Container::end(); + } + + const_iterator find(const Key& key) const + { + for (auto it = this->begin(); it != this->end(); ++it) + { + if (it->first == key) + { + return it; + } + } + return Container::end(); + } + + std::pair insert( value_type&& value ) + { + return emplace(value.first, std::move(value.second)); + } + + std::pair insert( const value_type& value ) + { + for (auto it = this->begin(); it != this->end(); ++it) + { + if (it->first == value.first) + { + return {it, false}; + } + } + Container::push_back(value); + return {--this->end(), true}; + } +}; + +} // namespace nlohmann + + +/*! +@brief namespace for Niels Lohmann +@see https://github.com/nlohmann +@since version 1.0.0 +*/ +namespace nlohmann +{ + +/*! +@brief a class to store JSON values + +@tparam ObjectType type for JSON objects (`std::map` by default; will be used +in @ref object_t) +@tparam ArrayType type for JSON arrays (`std::vector` by default; will be used +in @ref array_t) +@tparam StringType type for JSON strings and object keys (`std::string` by +default; will be used in @ref string_t) +@tparam BooleanType type for JSON booleans (`bool` by default; will be used +in @ref boolean_t) +@tparam NumberIntegerType type for JSON integer numbers (`int64_t` by +default; will be used in @ref number_integer_t) +@tparam NumberUnsignedType type for JSON unsigned integer numbers (@c +`uint64_t` by default; will be used in @ref number_unsigned_t) +@tparam NumberFloatType type for JSON floating-point numbers (`double` by +default; will be used in @ref number_float_t) +@tparam BinaryType type for packed binary data for compatibility with binary +serialization formats (`std::vector` by default; will be used in +@ref binary_t) +@tparam AllocatorType type of the allocator to use (`std::allocator` by +default) +@tparam JSONSerializer the serializer to resolve internal calls to `to_json()` +and `from_json()` (@ref adl_serializer by default) + +@requirement The class satisfies the following concept requirements: +- Basic + - [DefaultConstructible](https://en.cppreference.com/w/cpp/named_req/DefaultConstructible): + JSON values can be default constructed. The result will be a JSON null + value. + - [MoveConstructible](https://en.cppreference.com/w/cpp/named_req/MoveConstructible): + A JSON value can be constructed from an rvalue argument. + - [CopyConstructible](https://en.cppreference.com/w/cpp/named_req/CopyConstructible): + A JSON value can be copy-constructed from an lvalue expression. + - [MoveAssignable](https://en.cppreference.com/w/cpp/named_req/MoveAssignable): + A JSON value van be assigned from an rvalue argument. + - [CopyAssignable](https://en.cppreference.com/w/cpp/named_req/CopyAssignable): + A JSON value can be copy-assigned from an lvalue expression. + - [Destructible](https://en.cppreference.com/w/cpp/named_req/Destructible): + JSON values can be destructed. +- Layout + - [StandardLayoutType](https://en.cppreference.com/w/cpp/named_req/StandardLayoutType): + JSON values have + [standard layout](https://en.cppreference.com/w/cpp/language/data_members#Standard_layout): + All non-static data members are private and standard layout types, the + class has no virtual functions or (virtual) base classes. +- Library-wide + - [EqualityComparable](https://en.cppreference.com/w/cpp/named_req/EqualityComparable): + JSON values can be compared with `==`, see @ref + operator==(const_reference,const_reference). + - [LessThanComparable](https://en.cppreference.com/w/cpp/named_req/LessThanComparable): + JSON values can be compared with `<`, see @ref + operator<(const_reference,const_reference). + - [Swappable](https://en.cppreference.com/w/cpp/named_req/Swappable): + Any JSON lvalue or rvalue of can be swapped with any lvalue or rvalue of + other compatible types, using unqualified function call @ref swap(). + - [NullablePointer](https://en.cppreference.com/w/cpp/named_req/NullablePointer): + JSON values can be compared against `std::nullptr_t` objects which are used + to model the `null` value. +- Container + - [Container](https://en.cppreference.com/w/cpp/named_req/Container): + JSON values can be used like STL containers and provide iterator access. + - [ReversibleContainer](https://en.cppreference.com/w/cpp/named_req/ReversibleContainer); + JSON values can be used like STL containers and provide reverse iterator + access. + +@invariant The member variables @a m_value and @a m_type have the following +relationship: +- If `m_type == value_t::object`, then `m_value.object != nullptr`. +- If `m_type == value_t::array`, then `m_value.array != nullptr`. +- If `m_type == value_t::string`, then `m_value.string != nullptr`. +The invariants are checked by member function assert_invariant(). + +@internal +@note ObjectType trick from https://stackoverflow.com/a/9860911 +@endinternal + +@see [RFC 7159: The JavaScript Object Notation (JSON) Data Interchange +Format](http://rfc7159.net/rfc7159) + +@since version 1.0.0 + +@nosubgrouping +*/ +NLOHMANN_BASIC_JSON_TPL_DECLARATION +class basic_json +{ + private: + template friend struct detail::external_constructor; + friend ::nlohmann::json_pointer; + + template + friend class ::nlohmann::detail::parser; + friend ::nlohmann::detail::serializer; + template + friend class ::nlohmann::detail::iter_impl; + template + friend class ::nlohmann::detail::binary_writer; + template + friend class ::nlohmann::detail::binary_reader; + template + friend class ::nlohmann::detail::json_sax_dom_parser; + template + friend class ::nlohmann::detail::json_sax_dom_callback_parser; + + /// workaround type for MSVC + using basic_json_t = NLOHMANN_BASIC_JSON_TPL; + + // convenience aliases for types residing in namespace detail; + using lexer = ::nlohmann::detail::lexer_base; + + template + static ::nlohmann::detail::parser parser( + InputAdapterType adapter, + detail::parser_callback_tcb = nullptr, + const bool allow_exceptions = true, + const bool ignore_comments = false + ) + { + return ::nlohmann::detail::parser(std::move(adapter), + std::move(cb), allow_exceptions, ignore_comments); + } + + using primitive_iterator_t = ::nlohmann::detail::primitive_iterator_t; + template + using internal_iterator = ::nlohmann::detail::internal_iterator; + template + using iter_impl = ::nlohmann::detail::iter_impl; + template + using iteration_proxy = ::nlohmann::detail::iteration_proxy; + template using json_reverse_iterator = ::nlohmann::detail::json_reverse_iterator; + + template + using output_adapter_t = ::nlohmann::detail::output_adapter_t; + + template + using binary_reader = ::nlohmann::detail::binary_reader; + template using binary_writer = ::nlohmann::detail::binary_writer; + + using serializer = ::nlohmann::detail::serializer; + + public: + using value_t = detail::value_t; + /// JSON Pointer, see @ref nlohmann::json_pointer + using json_pointer = ::nlohmann::json_pointer; + template + using json_serializer = JSONSerializer; + /// how to treat decoding errors + using error_handler_t = detail::error_handler_t; + /// how to treat CBOR tags + using cbor_tag_handler_t = detail::cbor_tag_handler_t; + /// helper type for initializer lists of basic_json values + using initializer_list_t = std::initializer_list>; + + using input_format_t = detail::input_format_t; + /// SAX interface type, see @ref nlohmann::json_sax + using json_sax_t = json_sax; + + //////////////// + // exceptions // + //////////////// + + /// @name exceptions + /// Classes to implement user-defined exceptions. + /// @{ + + /// @copydoc detail::exception + using exception = detail::exception; + /// @copydoc detail::parse_error + using parse_error = detail::parse_error; + /// @copydoc detail::invalid_iterator + using invalid_iterator = detail::invalid_iterator; + /// @copydoc detail::type_error + using type_error = detail::type_error; + /// @copydoc detail::out_of_range + using out_of_range = detail::out_of_range; + /// @copydoc detail::other_error + using other_error = detail::other_error; + + /// @} + + + ///////////////////// + // container types // + ///////////////////// + + /// @name container types + /// The canonic container types to use @ref basic_json like any other STL + /// container. + /// @{ + + /// the type of elements in a basic_json container + using value_type = basic_json; + + /// the type of an element reference + using reference = value_type&; + /// the type of an element const reference + using const_reference = const value_type&; + + /// a type to represent differences between iterators + using difference_type = std::ptrdiff_t; + /// a type to represent container sizes + using size_type = std::size_t; + + /// the allocator type + using allocator_type = AllocatorType; + + /// the type of an element pointer + using pointer = typename std::allocator_traits::pointer; + /// the type of an element const pointer + using const_pointer = typename std::allocator_traits::const_pointer; + + /// an iterator for a basic_json container + using iterator = iter_impl; + /// a const iterator for a basic_json container + using const_iterator = iter_impl; + /// a reverse iterator for a basic_json container + using reverse_iterator = json_reverse_iterator; + /// a const reverse iterator for a basic_json container + using const_reverse_iterator = json_reverse_iterator; + + /// @} + + + /*! + @brief returns the allocator associated with the container + */ + static allocator_type get_allocator() + { + return allocator_type(); + } + + /*! + @brief returns version information on the library + + This function returns a JSON object with information about the library, + including the version number and information on the platform and compiler. + + @return JSON object holding version information + key | description + ----------- | --------------- + `compiler` | Information on the used compiler. It is an object with the following keys: `c++` (the used C++ standard), `family` (the compiler family; possible values are `clang`, `icc`, `gcc`, `ilecpp`, `msvc`, `pgcpp`, `sunpro`, and `unknown`), and `version` (the compiler version). + `copyright` | The copyright line for the library as string. + `name` | The name of the library as string. + `platform` | The used platform as string. Possible values are `win32`, `linux`, `apple`, `unix`, and `unknown`. + `url` | The URL of the project as string. + `version` | The version of the library. It is an object with the following keys: `major`, `minor`, and `patch` as defined by [Semantic Versioning](http://semver.org), and `string` (the version string). + + @liveexample{The following code shows an example output of the `meta()` + function.,meta} + + @exceptionsafety Strong guarantee: if an exception is thrown, there are no + changes to any JSON value. + + @complexity Constant. + + @since 2.1.0 + */ + JSON_HEDLEY_WARN_UNUSED_RESULT + static basic_json meta() + { + basic_json result; + + result["copyright"] = "(C) 2013-2020 Niels Lohmann"; + result["name"] = "JSON for Modern C++"; + result["url"] = "https://github.com/nlohmann/json"; + result["version"]["string"] = + std::to_string(NLOHMANN_JSON_VERSION_MAJOR) + "." + + std::to_string(NLOHMANN_JSON_VERSION_MINOR) + "." + + std::to_string(NLOHMANN_JSON_VERSION_PATCH); + result["version"]["major"] = NLOHMANN_JSON_VERSION_MAJOR; + result["version"]["minor"] = NLOHMANN_JSON_VERSION_MINOR; + result["version"]["patch"] = NLOHMANN_JSON_VERSION_PATCH; + +#ifdef _WIN32 + result["platform"] = "win32"; +#elif defined __linux__ + result["platform"] = "linux"; +#elif defined __APPLE__ + result["platform"] = "apple"; +#elif defined __unix__ + result["platform"] = "unix"; +#else + result["platform"] = "unknown"; +#endif + +#if defined(__ICC) || defined(__INTEL_COMPILER) + result["compiler"] = {{"family", "icc"}, {"version", __INTEL_COMPILER}}; +#elif defined(__clang__) + result["compiler"] = {{"family", "clang"}, {"version", __clang_version__}}; +#elif defined(__GNUC__) || defined(__GNUG__) + result["compiler"] = {{"family", "gcc"}, {"version", std::to_string(__GNUC__) + "." + std::to_string(__GNUC_MINOR__) + "." + std::to_string(__GNUC_PATCHLEVEL__)}}; +#elif defined(__HP_cc) || defined(__HP_aCC) + result["compiler"] = "hp" +#elif defined(__IBMCPP__) + result["compiler"] = {{"family", "ilecpp"}, {"version", __IBMCPP__}}; +#elif defined(_MSC_VER) + result["compiler"] = {{"family", "msvc"}, {"version", _MSC_VER}}; +#elif defined(__PGI) + result["compiler"] = {{"family", "pgcpp"}, {"version", __PGI}}; +#elif defined(__SUNPRO_CC) + result["compiler"] = {{"family", "sunpro"}, {"version", __SUNPRO_CC}}; +#else + result["compiler"] = {{"family", "unknown"}, {"version", "unknown"}}; +#endif + +#ifdef __cplusplus + result["compiler"]["c++"] = std::to_string(__cplusplus); +#else + result["compiler"]["c++"] = "unknown"; +#endif + return result; + } + + + /////////////////////////// + // JSON value data types // + /////////////////////////// + + /// @name JSON value data types + /// The data types to store a JSON value. These types are derived from + /// the template arguments passed to class @ref basic_json. + /// @{ + +#if defined(JSON_HAS_CPP_14) + // Use transparent comparator if possible, combined with perfect forwarding + // on find() and count() calls prevents unnecessary string construction. + using object_comparator_t = std::less<>; +#else + using object_comparator_t = std::less; +#endif + + /*! + @brief a type for an object + + [RFC 7159](http://rfc7159.net/rfc7159) describes JSON objects as follows: + > An object is an unordered collection of zero or more name/value pairs, + > where a name is a string and a value is a string, number, boolean, null, + > object, or array. + + To store objects in C++, a type is defined by the template parameters + described below. + + @tparam ObjectType the container to store objects (e.g., `std::map` or + `std::unordered_map`) + @tparam StringType the type of the keys or names (e.g., `std::string`). + The comparison function `std::less` is used to order elements + inside the container. + @tparam AllocatorType the allocator to use for objects (e.g., + `std::allocator`) + + #### Default type + + With the default values for @a ObjectType (`std::map`), @a StringType + (`std::string`), and @a AllocatorType (`std::allocator`), the default + value for @a object_t is: + + @code {.cpp} + std::map< + std::string, // key_type + basic_json, // value_type + std::less, // key_compare + std::allocator> // allocator_type + > + @endcode + + #### Behavior + + The choice of @a object_t influences the behavior of the JSON class. With + the default type, objects have the following behavior: + + - When all names are unique, objects will be interoperable in the sense + that all software implementations receiving that object will agree on + the name-value mappings. + - When the names within an object are not unique, it is unspecified which + one of the values for a given key will be chosen. For instance, + `{"key": 2, "key": 1}` could be equal to either `{"key": 1}` or + `{"key": 2}`. + - Internally, name/value pairs are stored in lexicographical order of the + names. Objects will also be serialized (see @ref dump) in this order. + For instance, `{"b": 1, "a": 2}` and `{"a": 2, "b": 1}` will be stored + and serialized as `{"a": 2, "b": 1}`. + - When comparing objects, the order of the name/value pairs is irrelevant. + This makes objects interoperable in the sense that they will not be + affected by these differences. For instance, `{"b": 1, "a": 2}` and + `{"a": 2, "b": 1}` will be treated as equal. + + #### Limits + + [RFC 7159](http://rfc7159.net/rfc7159) specifies: + > An implementation may set limits on the maximum depth of nesting. + + In this class, the object's limit of nesting is not explicitly constrained. + However, a maximum depth of nesting may be introduced by the compiler or + runtime environment. A theoretical limit can be queried by calling the + @ref max_size function of a JSON object. + + #### Storage + + Objects are stored as pointers in a @ref basic_json type. That is, for any + access to object values, a pointer of type `object_t*` must be + dereferenced. + + @sa @ref array_t -- type for an array value + + @since version 1.0.0 + + @note The order name/value pairs are added to the object is *not* + preserved by the library. Therefore, iterating an object may return + name/value pairs in a different order than they were originally stored. In + fact, keys will be traversed in alphabetical order as `std::map` with + `std::less` is used by default. Please note this behavior conforms to [RFC + 7159](http://rfc7159.net/rfc7159), because any order implements the + specified "unordered" nature of JSON objects. + */ + using object_t = ObjectType>>; + + /*! + @brief a type for an array + + [RFC 7159](http://rfc7159.net/rfc7159) describes JSON arrays as follows: + > An array is an ordered sequence of zero or more values. + + To store objects in C++, a type is defined by the template parameters + explained below. + + @tparam ArrayType container type to store arrays (e.g., `std::vector` or + `std::list`) + @tparam AllocatorType allocator to use for arrays (e.g., `std::allocator`) + + #### Default type + + With the default values for @a ArrayType (`std::vector`) and @a + AllocatorType (`std::allocator`), the default value for @a array_t is: + + @code {.cpp} + std::vector< + basic_json, // value_type + std::allocator // allocator_type + > + @endcode + + #### Limits + + [RFC 7159](http://rfc7159.net/rfc7159) specifies: + > An implementation may set limits on the maximum depth of nesting. + + In this class, the array's limit of nesting is not explicitly constrained. + However, a maximum depth of nesting may be introduced by the compiler or + runtime environment. A theoretical limit can be queried by calling the + @ref max_size function of a JSON array. + + #### Storage + + Arrays are stored as pointers in a @ref basic_json type. That is, for any + access to array values, a pointer of type `array_t*` must be dereferenced. + + @sa @ref object_t -- type for an object value + + @since version 1.0.0 + */ + using array_t = ArrayType>; + + /*! + @brief a type for a string + + [RFC 7159](http://rfc7159.net/rfc7159) describes JSON strings as follows: + > A string is a sequence of zero or more Unicode characters. + + To store objects in C++, a type is defined by the template parameter + described below. Unicode values are split by the JSON class into + byte-sized characters during deserialization. + + @tparam StringType the container to store strings (e.g., `std::string`). + Note this container is used for keys/names in objects, see @ref object_t. + + #### Default type + + With the default values for @a StringType (`std::string`), the default + value for @a string_t is: + + @code {.cpp} + std::string + @endcode + + #### Encoding + + Strings are stored in UTF-8 encoding. Therefore, functions like + `std::string::size()` or `std::string::length()` return the number of + bytes in the string rather than the number of characters or glyphs. + + #### String comparison + + [RFC 7159](http://rfc7159.net/rfc7159) states: + > Software implementations are typically required to test names of object + > members for equality. Implementations that transform the textual + > representation into sequences of Unicode code units and then perform the + > comparison numerically, code unit by code unit, are interoperable in the + > sense that implementations will agree in all cases on equality or + > inequality of two strings. For example, implementations that compare + > strings with escaped characters unconverted may incorrectly find that + > `"a\\b"` and `"a\u005Cb"` are not equal. + + This implementation is interoperable as it does compare strings code unit + by code unit. + + #### Storage + + String values are stored as pointers in a @ref basic_json type. That is, + for any access to string values, a pointer of type `string_t*` must be + dereferenced. + + @since version 1.0.0 + */ + using string_t = StringType; + + /*! + @brief a type for a boolean + + [RFC 7159](http://rfc7159.net/rfc7159) implicitly describes a boolean as a + type which differentiates the two literals `true` and `false`. + + To store objects in C++, a type is defined by the template parameter @a + BooleanType which chooses the type to use. + + #### Default type + + With the default values for @a BooleanType (`bool`), the default value for + @a boolean_t is: + + @code {.cpp} + bool + @endcode + + #### Storage + + Boolean values are stored directly inside a @ref basic_json type. + + @since version 1.0.0 + */ + using boolean_t = BooleanType; + + /*! + @brief a type for a number (integer) + + [RFC 7159](http://rfc7159.net/rfc7159) describes numbers as follows: + > The representation of numbers is similar to that used in most + > programming languages. A number is represented in base 10 using decimal + > digits. It contains an integer component that may be prefixed with an + > optional minus sign, which may be followed by a fraction part and/or an + > exponent part. Leading zeros are not allowed. (...) Numeric values that + > cannot be represented in the grammar below (such as Infinity and NaN) + > are not permitted. + + This description includes both integer and floating-point numbers. + However, C++ allows more precise storage if it is known whether the number + is a signed integer, an unsigned integer or a floating-point number. + Therefore, three different types, @ref number_integer_t, @ref + number_unsigned_t and @ref number_float_t are used. + + To store integer numbers in C++, a type is defined by the template + parameter @a NumberIntegerType which chooses the type to use. + + #### Default type + + With the default values for @a NumberIntegerType (`int64_t`), the default + value for @a number_integer_t is: + + @code {.cpp} + int64_t + @endcode + + #### Default behavior + + - The restrictions about leading zeros is not enforced in C++. Instead, + leading zeros in integer literals lead to an interpretation as octal + number. Internally, the value will be stored as decimal number. For + instance, the C++ integer literal `010` will be serialized to `8`. + During deserialization, leading zeros yield an error. + - Not-a-number (NaN) values will be serialized to `null`. + + #### Limits + + [RFC 7159](http://rfc7159.net/rfc7159) specifies: + > An implementation may set limits on the range and precision of numbers. + + When the default type is used, the maximal integer number that can be + stored is `9223372036854775807` (INT64_MAX) and the minimal integer number + that can be stored is `-9223372036854775808` (INT64_MIN). Integer numbers + that are out of range will yield over/underflow when used in a + constructor. During deserialization, too large or small integer numbers + will be automatically be stored as @ref number_unsigned_t or @ref + number_float_t. + + [RFC 7159](http://rfc7159.net/rfc7159) further states: + > Note that when such software is used, numbers that are integers and are + > in the range \f$[-2^{53}+1, 2^{53}-1]\f$ are interoperable in the sense + > that implementations will agree exactly on their numeric values. + + As this range is a subrange of the exactly supported range [INT64_MIN, + INT64_MAX], this class's integer type is interoperable. + + #### Storage + + Integer number values are stored directly inside a @ref basic_json type. + + @sa @ref number_float_t -- type for number values (floating-point) + + @sa @ref number_unsigned_t -- type for number values (unsigned integer) + + @since version 1.0.0 + */ + using number_integer_t = NumberIntegerType; + + /*! + @brief a type for a number (unsigned) + + [RFC 7159](http://rfc7159.net/rfc7159) describes numbers as follows: + > The representation of numbers is similar to that used in most + > programming languages. A number is represented in base 10 using decimal + > digits. It contains an integer component that may be prefixed with an + > optional minus sign, which may be followed by a fraction part and/or an + > exponent part. Leading zeros are not allowed. (...) Numeric values that + > cannot be represented in the grammar below (such as Infinity and NaN) + > are not permitted. + + This description includes both integer and floating-point numbers. + However, C++ allows more precise storage if it is known whether the number + is a signed integer, an unsigned integer or a floating-point number. + Therefore, three different types, @ref number_integer_t, @ref + number_unsigned_t and @ref number_float_t are used. + + To store unsigned integer numbers in C++, a type is defined by the + template parameter @a NumberUnsignedType which chooses the type to use. + + #### Default type + + With the default values for @a NumberUnsignedType (`uint64_t`), the + default value for @a number_unsigned_t is: + + @code {.cpp} + uint64_t + @endcode + + #### Default behavior + + - The restrictions about leading zeros is not enforced in C++. Instead, + leading zeros in integer literals lead to an interpretation as octal + number. Internally, the value will be stored as decimal number. For + instance, the C++ integer literal `010` will be serialized to `8`. + During deserialization, leading zeros yield an error. + - Not-a-number (NaN) values will be serialized to `null`. + + #### Limits + + [RFC 7159](http://rfc7159.net/rfc7159) specifies: + > An implementation may set limits on the range and precision of numbers. + + When the default type is used, the maximal integer number that can be + stored is `18446744073709551615` (UINT64_MAX) and the minimal integer + number that can be stored is `0`. Integer numbers that are out of range + will yield over/underflow when used in a constructor. During + deserialization, too large or small integer numbers will be automatically + be stored as @ref number_integer_t or @ref number_float_t. + + [RFC 7159](http://rfc7159.net/rfc7159) further states: + > Note that when such software is used, numbers that are integers and are + > in the range \f$[-2^{53}+1, 2^{53}-1]\f$ are interoperable in the sense + > that implementations will agree exactly on their numeric values. + + As this range is a subrange (when considered in conjunction with the + number_integer_t type) of the exactly supported range [0, UINT64_MAX], + this class's integer type is interoperable. + + #### Storage + + Integer number values are stored directly inside a @ref basic_json type. + + @sa @ref number_float_t -- type for number values (floating-point) + @sa @ref number_integer_t -- type for number values (integer) + + @since version 2.0.0 + */ + using number_unsigned_t = NumberUnsignedType; + + /*! + @brief a type for a number (floating-point) + + [RFC 7159](http://rfc7159.net/rfc7159) describes numbers as follows: + > The representation of numbers is similar to that used in most + > programming languages. A number is represented in base 10 using decimal + > digits. It contains an integer component that may be prefixed with an + > optional minus sign, which may be followed by a fraction part and/or an + > exponent part. Leading zeros are not allowed. (...) Numeric values that + > cannot be represented in the grammar below (such as Infinity and NaN) + > are not permitted. + + This description includes both integer and floating-point numbers. + However, C++ allows more precise storage if it is known whether the number + is a signed integer, an unsigned integer or a floating-point number. + Therefore, three different types, @ref number_integer_t, @ref + number_unsigned_t and @ref number_float_t are used. + + To store floating-point numbers in C++, a type is defined by the template + parameter @a NumberFloatType which chooses the type to use. + + #### Default type + + With the default values for @a NumberFloatType (`double`), the default + value for @a number_float_t is: + + @code {.cpp} + double + @endcode + + #### Default behavior + + - The restrictions about leading zeros is not enforced in C++. Instead, + leading zeros in floating-point literals will be ignored. Internally, + the value will be stored as decimal number. For instance, the C++ + floating-point literal `01.2` will be serialized to `1.2`. During + deserialization, leading zeros yield an error. + - Not-a-number (NaN) values will be serialized to `null`. + + #### Limits + + [RFC 7159](http://rfc7159.net/rfc7159) states: + > This specification allows implementations to set limits on the range and + > precision of numbers accepted. Since software that implements IEEE + > 754-2008 binary64 (double precision) numbers is generally available and + > widely used, good interoperability can be achieved by implementations + > that expect no more precision or range than these provide, in the sense + > that implementations will approximate JSON numbers within the expected + > precision. + + This implementation does exactly follow this approach, as it uses double + precision floating-point numbers. Note values smaller than + `-1.79769313486232e+308` and values greater than `1.79769313486232e+308` + will be stored as NaN internally and be serialized to `null`. + + #### Storage + + Floating-point number values are stored directly inside a @ref basic_json + type. + + @sa @ref number_integer_t -- type for number values (integer) + + @sa @ref number_unsigned_t -- type for number values (unsigned integer) + + @since version 1.0.0 + */ + using number_float_t = NumberFloatType; + + /*! + @brief a type for a packed binary type + + This type is a type designed to carry binary data that appears in various + serialized formats, such as CBOR's Major Type 2, MessagePack's bin, and + BSON's generic binary subtype. This type is NOT a part of standard JSON and + exists solely for compatibility with these binary types. As such, it is + simply defined as an ordered sequence of zero or more byte values. + + Additionally, as an implementation detail, the subtype of the binary data is + carried around as a `std::uint8_t`, which is compatible with both of the + binary data formats that use binary subtyping, (though the specific + numbering is incompatible with each other, and it is up to the user to + translate between them). + + [CBOR's RFC 7049](https://tools.ietf.org/html/rfc7049) describes this type + as: + > Major type 2: a byte string. The string's length in bytes is represented + > following the rules for positive integers (major type 0). + + [MessagePack's documentation on the bin type + family](https://github.com/msgpack/msgpack/blob/master/spec.md#bin-format-family) + describes this type as: + > Bin format family stores an byte array in 2, 3, or 5 bytes of extra bytes + > in addition to the size of the byte array. + + [BSON's specifications](http://bsonspec.org/spec.html) describe several + binary types; however, this type is intended to represent the generic binary + type which has the description: + > Generic binary subtype - This is the most commonly used binary subtype and + > should be the 'default' for drivers and tools. + + None of these impose any limitations on the internal representation other + than the basic unit of storage be some type of array whose parts are + decomposable into bytes. + + The default representation of this binary format is a + `std::vector`, which is a very common way to represent a byte + array in modern C++. + + #### Default type + + The default values for @a BinaryType is `std::vector` + + #### Storage + + Binary Arrays are stored as pointers in a @ref basic_json type. That is, + for any access to array values, a pointer of the type `binary_t*` must be + dereferenced. + + #### Notes on subtypes + + - CBOR + - Binary values are represented as byte strings. No subtypes are + supported and will be ignored when CBOR is written. + - MessagePack + - If a subtype is given and the binary array contains exactly 1, 2, 4, 8, + or 16 elements, the fixext family (fixext1, fixext2, fixext4, fixext8) + is used. For other sizes, the ext family (ext8, ext16, ext32) is used. + The subtype is then added as singed 8-bit integer. + - If no subtype is given, the bin family (bin8, bin16, bin32) is used. + - BSON + - If a subtype is given, it is used and added as unsigned 8-bit integer. + - If no subtype is given, the generic binary subtype 0x00 is used. + + @sa @ref binary -- create a binary array + + @since version 3.8.0 + */ + using binary_t = nlohmann::byte_container_with_subtype; + /// @} + + private: + + /// helper for exception-safe object creation + template + JSON_HEDLEY_RETURNS_NON_NULL + static T* create(Args&& ... args) + { + AllocatorType alloc; + using AllocatorTraits = std::allocator_traits>; + + auto deleter = [&](T * object) + { + AllocatorTraits::deallocate(alloc, object, 1); + }; + std::unique_ptr object(AllocatorTraits::allocate(alloc, 1), deleter); + AllocatorTraits::construct(alloc, object.get(), std::forward(args)...); + JSON_ASSERT(object != nullptr); + return object.release(); + } + + //////////////////////// + // JSON value storage // + //////////////////////// + + /*! + @brief a JSON value + + The actual storage for a JSON value of the @ref basic_json class. This + union combines the different storage types for the JSON value types + defined in @ref value_t. + + JSON type | value_t type | used type + --------- | --------------- | ------------------------ + object | object | pointer to @ref object_t + array | array | pointer to @ref array_t + string | string | pointer to @ref string_t + boolean | boolean | @ref boolean_t + number | number_integer | @ref number_integer_t + number | number_unsigned | @ref number_unsigned_t + number | number_float | @ref number_float_t + binary | binary | pointer to @ref binary_t + null | null | *no value is stored* + + @note Variable-length types (objects, arrays, and strings) are stored as + pointers. The size of the union should not exceed 64 bits if the default + value types are used. + + @since version 1.0.0 + */ + union json_value + { + /// object (stored with pointer to save storage) + object_t* object; + /// array (stored with pointer to save storage) + array_t* array; + /// string (stored with pointer to save storage) + string_t* string; + /// binary (stored with pointer to save storage) + binary_t* binary; + /// boolean + boolean_t boolean; + /// number (integer) + number_integer_t number_integer; + /// number (unsigned integer) + number_unsigned_t number_unsigned; + /// number (floating-point) + number_float_t number_float; + + /// default constructor (for null values) + json_value() = default; + /// constructor for booleans + json_value(boolean_t v) noexcept : boolean(v) {} + /// constructor for numbers (integer) + json_value(number_integer_t v) noexcept : number_integer(v) {} + /// constructor for numbers (unsigned) + json_value(number_unsigned_t v) noexcept : number_unsigned(v) {} + /// constructor for numbers (floating-point) + json_value(number_float_t v) noexcept : number_float(v) {} + /// constructor for empty values of a given type + json_value(value_t t) + { + switch (t) + { + case value_t::object: + { + object = create(); + break; + } + + case value_t::array: + { + array = create(); + break; + } + + case value_t::string: + { + string = create(""); + break; + } + + case value_t::binary: + { + binary = create(); + break; + } + + case value_t::boolean: + { + boolean = boolean_t(false); + break; + } + + case value_t::number_integer: + { + number_integer = number_integer_t(0); + break; + } + + case value_t::number_unsigned: + { + number_unsigned = number_unsigned_t(0); + break; + } + + case value_t::number_float: + { + number_float = number_float_t(0.0); + break; + } + + case value_t::null: + { + object = nullptr; // silence warning, see #821 + break; + } + + default: + { + object = nullptr; // silence warning, see #821 + if (JSON_HEDLEY_UNLIKELY(t == value_t::null)) + { + JSON_THROW(other_error::create(500, "961c151d2e87f2686a955a9be24d316f1362bf21 3.9.1")); // LCOV_EXCL_LINE + } + break; + } + } + } + + /// constructor for strings + json_value(const string_t& value) + { + string = create(value); + } + + /// constructor for rvalue strings + json_value(string_t&& value) + { + string = create(std::move(value)); + } + + /// constructor for objects + json_value(const object_t& value) + { + object = create(value); + } + + /// constructor for rvalue objects + json_value(object_t&& value) + { + object = create(std::move(value)); + } + + /// constructor for arrays + json_value(const array_t& value) + { + array = create(value); + } + + /// constructor for rvalue arrays + json_value(array_t&& value) + { + array = create(std::move(value)); + } + + /// constructor for binary arrays + json_value(const typename binary_t::container_type& value) + { + binary = create(value); + } + + /// constructor for rvalue binary arrays + json_value(typename binary_t::container_type&& value) + { + binary = create(std::move(value)); + } + + /// constructor for binary arrays (internal type) + json_value(const binary_t& value) + { + binary = create(value); + } + + /// constructor for rvalue binary arrays (internal type) + json_value(binary_t&& value) + { + binary = create(std::move(value)); + } + + void destroy(value_t t) noexcept + { + // flatten the current json_value to a heap-allocated stack + std::vector stack; + + // move the top-level items to stack + if (t == value_t::array) + { + stack.reserve(array->size()); + std::move(array->begin(), array->end(), std::back_inserter(stack)); + } + else if (t == value_t::object) + { + stack.reserve(object->size()); + for (auto&& it : *object) + { + stack.push_back(std::move(it.second)); + } + } + + while (!stack.empty()) + { + // move the last item to local variable to be processed + basic_json current_item(std::move(stack.back())); + stack.pop_back(); + + // if current_item is array/object, move + // its children to the stack to be processed later + if (current_item.is_array()) + { + std::move(current_item.m_value.array->begin(), current_item.m_value.array->end(), + std::back_inserter(stack)); + + current_item.m_value.array->clear(); + } + else if (current_item.is_object()) + { + for (auto&& it : *current_item.m_value.object) + { + stack.push_back(std::move(it.second)); + } + + current_item.m_value.object->clear(); + } + + // it's now safe that current_item get destructed + // since it doesn't have any children + } + + switch (t) + { + case value_t::object: + { + AllocatorType alloc; + std::allocator_traits::destroy(alloc, object); + std::allocator_traits::deallocate(alloc, object, 1); + break; + } + + case value_t::array: + { + AllocatorType alloc; + std::allocator_traits::destroy(alloc, array); + std::allocator_traits::deallocate(alloc, array, 1); + break; + } + + case value_t::string: + { + AllocatorType alloc; + std::allocator_traits::destroy(alloc, string); + std::allocator_traits::deallocate(alloc, string, 1); + break; + } + + case value_t::binary: + { + AllocatorType alloc; + std::allocator_traits::destroy(alloc, binary); + std::allocator_traits::deallocate(alloc, binary, 1); + break; + } + + default: + { + break; + } + } + } + }; + + /*! + @brief checks the class invariants + + This function asserts the class invariants. It needs to be called at the + end of every constructor to make sure that created objects respect the + invariant. Furthermore, it has to be called each time the type of a JSON + value is changed, because the invariant expresses a relationship between + @a m_type and @a m_value. + */ + void assert_invariant() const noexcept + { + JSON_ASSERT(m_type != value_t::object || m_value.object != nullptr); + JSON_ASSERT(m_type != value_t::array || m_value.array != nullptr); + JSON_ASSERT(m_type != value_t::string || m_value.string != nullptr); + JSON_ASSERT(m_type != value_t::binary || m_value.binary != nullptr); + } + + public: + ////////////////////////// + // JSON parser callback // + ////////////////////////// + + /*! + @brief parser event types + + The parser callback distinguishes the following events: + - `object_start`: the parser read `{` and started to process a JSON object + - `key`: the parser read a key of a value in an object + - `object_end`: the parser read `}` and finished processing a JSON object + - `array_start`: the parser read `[` and started to process a JSON array + - `array_end`: the parser read `]` and finished processing a JSON array + - `value`: the parser finished reading a JSON value + + @image html callback_events.png "Example when certain parse events are triggered" + + @sa @ref parser_callback_t for more information and examples + */ + using parse_event_t = detail::parse_event_t; + + /*! + @brief per-element parser callback type + + With a parser callback function, the result of parsing a JSON text can be + influenced. When passed to @ref parse, it is called on certain events + (passed as @ref parse_event_t via parameter @a event) with a set recursion + depth @a depth and context JSON value @a parsed. The return value of the + callback function is a boolean indicating whether the element that emitted + the callback shall be kept or not. + + We distinguish six scenarios (determined by the event type) in which the + callback function can be called. The following table describes the values + of the parameters @a depth, @a event, and @a parsed. + + parameter @a event | description | parameter @a depth | parameter @a parsed + ------------------ | ----------- | ------------------ | ------------------- + parse_event_t::object_start | the parser read `{` and started to process a JSON object | depth of the parent of the JSON object | a JSON value with type discarded + parse_event_t::key | the parser read a key of a value in an object | depth of the currently parsed JSON object | a JSON string containing the key + parse_event_t::object_end | the parser read `}` and finished processing a JSON object | depth of the parent of the JSON object | the parsed JSON object + parse_event_t::array_start | the parser read `[` and started to process a JSON array | depth of the parent of the JSON array | a JSON value with type discarded + parse_event_t::array_end | the parser read `]` and finished processing a JSON array | depth of the parent of the JSON array | the parsed JSON array + parse_event_t::value | the parser finished reading a JSON value | depth of the value | the parsed JSON value + + @image html callback_events.png "Example when certain parse events are triggered" + + Discarding a value (i.e., returning `false`) has different effects + depending on the context in which function was called: + + - Discarded values in structured types are skipped. That is, the parser + will behave as if the discarded value was never read. + - In case a value outside a structured type is skipped, it is replaced + with `null`. This case happens if the top-level element is skipped. + + @param[in] depth the depth of the recursion during parsing + + @param[in] event an event of type parse_event_t indicating the context in + the callback function has been called + + @param[in,out] parsed the current intermediate parse result; note that + writing to this value has no effect for parse_event_t::key events + + @return Whether the JSON value which called the function during parsing + should be kept (`true`) or not (`false`). In the latter case, it is either + skipped completely or replaced by an empty discarded object. + + @sa @ref parse for examples + + @since version 1.0.0 + */ + using parser_callback_t = detail::parser_callback_t; + + ////////////////// + // constructors // + ////////////////// + + /// @name constructors and destructors + /// Constructors of class @ref basic_json, copy/move constructor, copy + /// assignment, static functions creating objects, and the destructor. + /// @{ + + /*! + @brief create an empty value with a given type + + Create an empty JSON value with a given type. The value will be default + initialized with an empty value which depends on the type: + + Value type | initial value + ----------- | ------------- + null | `null` + boolean | `false` + string | `""` + number | `0` + object | `{}` + array | `[]` + binary | empty array + + @param[in] v the type of the value to create + + @complexity Constant. + + @exceptionsafety Strong guarantee: if an exception is thrown, there are no + changes to any JSON value. + + @liveexample{The following code shows the constructor for different @ref + value_t values,basic_json__value_t} + + @sa @ref clear() -- restores the postcondition of this constructor + + @since version 1.0.0 + */ + basic_json(const value_t v) + : m_type(v), m_value(v) + { + assert_invariant(); + } + + /*! + @brief create a null object + + Create a `null` JSON value. It either takes a null pointer as parameter + (explicitly creating `null`) or no parameter (implicitly creating `null`). + The passed null pointer itself is not read -- it is only used to choose + the right constructor. + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this constructor never throws + exceptions. + + @liveexample{The following code shows the constructor with and without a + null pointer parameter.,basic_json__nullptr_t} + + @since version 1.0.0 + */ + basic_json(std::nullptr_t = nullptr) noexcept + : basic_json(value_t::null) + { + assert_invariant(); + } + + /*! + @brief create a JSON value + + This is a "catch all" constructor for all compatible JSON types; that is, + types for which a `to_json()` method exists. The constructor forwards the + parameter @a val to that method (to `json_serializer::to_json` method + with `U = uncvref_t`, to be exact). + + Template type @a CompatibleType includes, but is not limited to, the + following types: + - **arrays**: @ref array_t and all kinds of compatible containers such as + `std::vector`, `std::deque`, `std::list`, `std::forward_list`, + `std::array`, `std::valarray`, `std::set`, `std::unordered_set`, + `std::multiset`, and `std::unordered_multiset` with a `value_type` from + which a @ref basic_json value can be constructed. + - **objects**: @ref object_t and all kinds of compatible associative + containers such as `std::map`, `std::unordered_map`, `std::multimap`, + and `std::unordered_multimap` with a `key_type` compatible to + @ref string_t and a `value_type` from which a @ref basic_json value can + be constructed. + - **strings**: @ref string_t, string literals, and all compatible string + containers can be used. + - **numbers**: @ref number_integer_t, @ref number_unsigned_t, + @ref number_float_t, and all convertible number types such as `int`, + `size_t`, `int64_t`, `float` or `double` can be used. + - **boolean**: @ref boolean_t / `bool` can be used. + - **binary**: @ref binary_t / `std::vector` may be used, + unfortunately because string literals cannot be distinguished from binary + character arrays by the C++ type system, all types compatible with `const + char*` will be directed to the string constructor instead. This is both + for backwards compatibility, and due to the fact that a binary type is not + a standard JSON type. + + See the examples below. + + @tparam CompatibleType a type such that: + - @a CompatibleType is not derived from `std::istream`, + - @a CompatibleType is not @ref basic_json (to avoid hijacking copy/move + constructors), + - @a CompatibleType is not a different @ref basic_json type (i.e. with different template arguments) + - @a CompatibleType is not a @ref basic_json nested type (e.g., + @ref json_pointer, @ref iterator, etc ...) + - @ref @ref json_serializer has a + `to_json(basic_json_t&, CompatibleType&&)` method + + @tparam U = `uncvref_t` + + @param[in] val the value to be forwarded to the respective constructor + + @complexity Usually linear in the size of the passed @a val, also + depending on the implementation of the called `to_json()` + method. + + @exceptionsafety Depends on the called constructor. For types directly + supported by the library (i.e., all types for which no `to_json()` function + was provided), strong guarantee holds: if an exception is thrown, there are + no changes to any JSON value. + + @liveexample{The following code shows the constructor with several + compatible types.,basic_json__CompatibleType} + + @since version 2.1.0 + */ + template < typename CompatibleType, + typename U = detail::uncvref_t, + detail::enable_if_t < + !detail::is_basic_json::value && detail::is_compatible_type::value, int > = 0 > + basic_json(CompatibleType && val) noexcept(noexcept( + JSONSerializer::to_json(std::declval(), + std::forward(val)))) + { + JSONSerializer::to_json(*this, std::forward(val)); + assert_invariant(); + } + + /*! + @brief create a JSON value from an existing one + + This is a constructor for existing @ref basic_json types. + It does not hijack copy/move constructors, since the parameter has different + template arguments than the current ones. + + The constructor tries to convert the internal @ref m_value of the parameter. + + @tparam BasicJsonType a type such that: + - @a BasicJsonType is a @ref basic_json type. + - @a BasicJsonType has different template arguments than @ref basic_json_t. + + @param[in] val the @ref basic_json value to be converted. + + @complexity Usually linear in the size of the passed @a val, also + depending on the implementation of the called `to_json()` + method. + + @exceptionsafety Depends on the called constructor. For types directly + supported by the library (i.e., all types for which no `to_json()` function + was provided), strong guarantee holds: if an exception is thrown, there are + no changes to any JSON value. + + @since version 3.2.0 + */ + template < typename BasicJsonType, + detail::enable_if_t < + detail::is_basic_json::value&& !std::is_same::value, int > = 0 > + basic_json(const BasicJsonType& val) + { + using other_boolean_t = typename BasicJsonType::boolean_t; + using other_number_float_t = typename BasicJsonType::number_float_t; + using other_number_integer_t = typename BasicJsonType::number_integer_t; + using other_number_unsigned_t = typename BasicJsonType::number_unsigned_t; + using other_string_t = typename BasicJsonType::string_t; + using other_object_t = typename BasicJsonType::object_t; + using other_array_t = typename BasicJsonType::array_t; + using other_binary_t = typename BasicJsonType::binary_t; + + switch (val.type()) + { + case value_t::boolean: + JSONSerializer::to_json(*this, val.template get()); + break; + case value_t::number_float: + JSONSerializer::to_json(*this, val.template get()); + break; + case value_t::number_integer: + JSONSerializer::to_json(*this, val.template get()); + break; + case value_t::number_unsigned: + JSONSerializer::to_json(*this, val.template get()); + break; + case value_t::string: + JSONSerializer::to_json(*this, val.template get_ref()); + break; + case value_t::object: + JSONSerializer::to_json(*this, val.template get_ref()); + break; + case value_t::array: + JSONSerializer::to_json(*this, val.template get_ref()); + break; + case value_t::binary: + JSONSerializer::to_json(*this, val.template get_ref()); + break; + case value_t::null: + *this = nullptr; + break; + case value_t::discarded: + m_type = value_t::discarded; + break; + default: // LCOV_EXCL_LINE + JSON_ASSERT(false); // LCOV_EXCL_LINE + } + assert_invariant(); + } + + /*! + @brief create a container (array or object) from an initializer list + + Creates a JSON value of type array or object from the passed initializer + list @a init. In case @a type_deduction is `true` (default), the type of + the JSON value to be created is deducted from the initializer list @a init + according to the following rules: + + 1. If the list is empty, an empty JSON object value `{}` is created. + 2. If the list consists of pairs whose first element is a string, a JSON + object value is created where the first elements of the pairs are + treated as keys and the second elements are as values. + 3. In all other cases, an array is created. + + The rules aim to create the best fit between a C++ initializer list and + JSON values. The rationale is as follows: + + 1. The empty initializer list is written as `{}` which is exactly an empty + JSON object. + 2. C++ has no way of describing mapped types other than to list a list of + pairs. As JSON requires that keys must be of type string, rule 2 is the + weakest constraint one can pose on initializer lists to interpret them + as an object. + 3. In all other cases, the initializer list could not be interpreted as + JSON object type, so interpreting it as JSON array type is safe. + + With the rules described above, the following JSON values cannot be + expressed by an initializer list: + + - the empty array (`[]`): use @ref array(initializer_list_t) + with an empty initializer list in this case + - arrays whose elements satisfy rule 2: use @ref + array(initializer_list_t) with the same initializer list + in this case + + @note When used without parentheses around an empty initializer list, @ref + basic_json() is called instead of this function, yielding the JSON null + value. + + @param[in] init initializer list with JSON values + + @param[in] type_deduction internal parameter; when set to `true`, the type + of the JSON value is deducted from the initializer list @a init; when set + to `false`, the type provided via @a manual_type is forced. This mode is + used by the functions @ref array(initializer_list_t) and + @ref object(initializer_list_t). + + @param[in] manual_type internal parameter; when @a type_deduction is set + to `false`, the created JSON value will use the provided type (only @ref + value_t::array and @ref value_t::object are valid); when @a type_deduction + is set to `true`, this parameter has no effect + + @throw type_error.301 if @a type_deduction is `false`, @a manual_type is + `value_t::object`, but @a init contains an element which is not a pair + whose first element is a string. In this case, the constructor could not + create an object. If @a type_deduction would have be `true`, an array + would have been created. See @ref object(initializer_list_t) + for an example. + + @complexity Linear in the size of the initializer list @a init. + + @exceptionsafety Strong guarantee: if an exception is thrown, there are no + changes to any JSON value. + + @liveexample{The example below shows how JSON values are created from + initializer lists.,basic_json__list_init_t} + + @sa @ref array(initializer_list_t) -- create a JSON array + value from an initializer list + @sa @ref object(initializer_list_t) -- create a JSON object + value from an initializer list + + @since version 1.0.0 + */ + basic_json(initializer_list_t init, + bool type_deduction = true, + value_t manual_type = value_t::array) + { + // check if each element is an array with two elements whose first + // element is a string + bool is_an_object = std::all_of(init.begin(), init.end(), + [](const detail::json_ref& element_ref) + { + return element_ref->is_array() && element_ref->size() == 2 && (*element_ref)[0].is_string(); + }); + + // adjust type if type deduction is not wanted + if (!type_deduction) + { + // if array is wanted, do not create an object though possible + if (manual_type == value_t::array) + { + is_an_object = false; + } + + // if object is wanted but impossible, throw an exception + if (JSON_HEDLEY_UNLIKELY(manual_type == value_t::object && !is_an_object)) + { + JSON_THROW(type_error::create(301, "cannot create object from initializer list")); + } + } + + if (is_an_object) + { + // the initializer list is a list of pairs -> create object + m_type = value_t::object; + m_value = value_t::object; + + std::for_each(init.begin(), init.end(), [this](const detail::json_ref& element_ref) + { + auto element = element_ref.moved_or_copied(); + m_value.object->emplace( + std::move(*((*element.m_value.array)[0].m_value.string)), + std::move((*element.m_value.array)[1])); + }); + } + else + { + // the initializer list describes an array -> create array + m_type = value_t::array; + m_value.array = create(init.begin(), init.end()); + } + + assert_invariant(); + } + + /*! + @brief explicitly create a binary array (without subtype) + + Creates a JSON binary array value from a given binary container. Binary + values are part of various binary formats, such as CBOR, MessagePack, and + BSON. This constructor is used to create a value for serialization to those + formats. + + @note Note, this function exists because of the difficulty in correctly + specifying the correct template overload in the standard value ctor, as both + JSON arrays and JSON binary arrays are backed with some form of a + `std::vector`. Because JSON binary arrays are a non-standard extension it + was decided that it would be best to prevent automatic initialization of a + binary array type, for backwards compatibility and so it does not happen on + accident. + + @param[in] init container containing bytes to use as binary type + + @return JSON binary array value + + @complexity Linear in the size of @a init. + + @exceptionsafety Strong guarantee: if an exception is thrown, there are no + changes to any JSON value. + + @since version 3.8.0 + */ + JSON_HEDLEY_WARN_UNUSED_RESULT + static basic_json binary(const typename binary_t::container_type& init) + { + auto res = basic_json(); + res.m_type = value_t::binary; + res.m_value = init; + return res; + } + + /*! + @brief explicitly create a binary array (with subtype) + + Creates a JSON binary array value from a given binary container. Binary + values are part of various binary formats, such as CBOR, MessagePack, and + BSON. This constructor is used to create a value for serialization to those + formats. + + @note Note, this function exists because of the difficulty in correctly + specifying the correct template overload in the standard value ctor, as both + JSON arrays and JSON binary arrays are backed with some form of a + `std::vector`. Because JSON binary arrays are a non-standard extension it + was decided that it would be best to prevent automatic initialization of a + binary array type, for backwards compatibility and so it does not happen on + accident. + + @param[in] init container containing bytes to use as binary type + @param[in] subtype subtype to use in MessagePack and BSON + + @return JSON binary array value + + @complexity Linear in the size of @a init. + + @exceptionsafety Strong guarantee: if an exception is thrown, there are no + changes to any JSON value. + + @since version 3.8.0 + */ + JSON_HEDLEY_WARN_UNUSED_RESULT + static basic_json binary(const typename binary_t::container_type& init, std::uint8_t subtype) + { + auto res = basic_json(); + res.m_type = value_t::binary; + res.m_value = binary_t(init, subtype); + return res; + } + + /// @copydoc binary(const typename binary_t::container_type&) + JSON_HEDLEY_WARN_UNUSED_RESULT + static basic_json binary(typename binary_t::container_type&& init) + { + auto res = basic_json(); + res.m_type = value_t::binary; + res.m_value = std::move(init); + return res; + } + + /// @copydoc binary(const typename binary_t::container_type&, std::uint8_t) + JSON_HEDLEY_WARN_UNUSED_RESULT + static basic_json binary(typename binary_t::container_type&& init, std::uint8_t subtype) + { + auto res = basic_json(); + res.m_type = value_t::binary; + res.m_value = binary_t(std::move(init), subtype); + return res; + } + + /*! + @brief explicitly create an array from an initializer list + + Creates a JSON array value from a given initializer list. That is, given a + list of values `a, b, c`, creates the JSON value `[a, b, c]`. If the + initializer list is empty, the empty array `[]` is created. + + @note This function is only needed to express two edge cases that cannot + be realized with the initializer list constructor (@ref + basic_json(initializer_list_t, bool, value_t)). These cases + are: + 1. creating an array whose elements are all pairs whose first element is a + string -- in this case, the initializer list constructor would create an + object, taking the first elements as keys + 2. creating an empty array -- passing the empty initializer list to the + initializer list constructor yields an empty object + + @param[in] init initializer list with JSON values to create an array from + (optional) + + @return JSON array value + + @complexity Linear in the size of @a init. + + @exceptionsafety Strong guarantee: if an exception is thrown, there are no + changes to any JSON value. + + @liveexample{The following code shows an example for the `array` + function.,array} + + @sa @ref basic_json(initializer_list_t, bool, value_t) -- + create a JSON value from an initializer list + @sa @ref object(initializer_list_t) -- create a JSON object + value from an initializer list + + @since version 1.0.0 + */ + JSON_HEDLEY_WARN_UNUSED_RESULT + static basic_json array(initializer_list_t init = {}) + { + return basic_json(init, false, value_t::array); + } + + /*! + @brief explicitly create an object from an initializer list + + Creates a JSON object value from a given initializer list. The initializer + lists elements must be pairs, and their first elements must be strings. If + the initializer list is empty, the empty object `{}` is created. + + @note This function is only added for symmetry reasons. In contrast to the + related function @ref array(initializer_list_t), there are + no cases which can only be expressed by this function. That is, any + initializer list @a init can also be passed to the initializer list + constructor @ref basic_json(initializer_list_t, bool, value_t). + + @param[in] init initializer list to create an object from (optional) + + @return JSON object value + + @throw type_error.301 if @a init is not a list of pairs whose first + elements are strings. In this case, no object can be created. When such a + value is passed to @ref basic_json(initializer_list_t, bool, value_t), + an array would have been created from the passed initializer list @a init. + See example below. + + @complexity Linear in the size of @a init. + + @exceptionsafety Strong guarantee: if an exception is thrown, there are no + changes to any JSON value. + + @liveexample{The following code shows an example for the `object` + function.,object} + + @sa @ref basic_json(initializer_list_t, bool, value_t) -- + create a JSON value from an initializer list + @sa @ref array(initializer_list_t) -- create a JSON array + value from an initializer list + + @since version 1.0.0 + */ + JSON_HEDLEY_WARN_UNUSED_RESULT + static basic_json object(initializer_list_t init = {}) + { + return basic_json(init, false, value_t::object); + } + + /*! + @brief construct an array with count copies of given value + + Constructs a JSON array value by creating @a cnt copies of a passed value. + In case @a cnt is `0`, an empty array is created. + + @param[in] cnt the number of JSON copies of @a val to create + @param[in] val the JSON value to copy + + @post `std::distance(begin(),end()) == cnt` holds. + + @complexity Linear in @a cnt. + + @exceptionsafety Strong guarantee: if an exception is thrown, there are no + changes to any JSON value. + + @liveexample{The following code shows examples for the @ref + basic_json(size_type\, const basic_json&) + constructor.,basic_json__size_type_basic_json} + + @since version 1.0.0 + */ + basic_json(size_type cnt, const basic_json& val) + : m_type(value_t::array) + { + m_value.array = create(cnt, val); + assert_invariant(); + } + + /*! + @brief construct a JSON container given an iterator range + + Constructs the JSON value with the contents of the range `[first, last)`. + The semantics depends on the different types a JSON value can have: + - In case of a null type, invalid_iterator.206 is thrown. + - In case of other primitive types (number, boolean, or string), @a first + must be `begin()` and @a last must be `end()`. In this case, the value is + copied. Otherwise, invalid_iterator.204 is thrown. + - In case of structured types (array, object), the constructor behaves as + similar versions for `std::vector` or `std::map`; that is, a JSON array + or object is constructed from the values in the range. + + @tparam InputIT an input iterator type (@ref iterator or @ref + const_iterator) + + @param[in] first begin of the range to copy from (included) + @param[in] last end of the range to copy from (excluded) + + @pre Iterators @a first and @a last must be initialized. **This + precondition is enforced with an assertion (see warning).** If + assertions are switched off, a violation of this precondition yields + undefined behavior. + + @pre Range `[first, last)` is valid. Usually, this precondition cannot be + checked efficiently. Only certain edge cases are detected; see the + description of the exceptions below. A violation of this precondition + yields undefined behavior. + + @warning A precondition is enforced with a runtime assertion that will + result in calling `std::abort` if this precondition is not met. + Assertions can be disabled by defining `NDEBUG` at compile time. + See https://en.cppreference.com/w/cpp/error/assert for more + information. + + @throw invalid_iterator.201 if iterators @a first and @a last are not + compatible (i.e., do not belong to the same JSON value). In this case, + the range `[first, last)` is undefined. + @throw invalid_iterator.204 if iterators @a first and @a last belong to a + primitive type (number, boolean, or string), but @a first does not point + to the first element any more. In this case, the range `[first, last)` is + undefined. See example code below. + @throw invalid_iterator.206 if iterators @a first and @a last belong to a + null value. In this case, the range `[first, last)` is undefined. + + @complexity Linear in distance between @a first and @a last. + + @exceptionsafety Strong guarantee: if an exception is thrown, there are no + changes to any JSON value. + + @liveexample{The example below shows several ways to create JSON values by + specifying a subrange with iterators.,basic_json__InputIt_InputIt} + + @since version 1.0.0 + */ + template < class InputIT, typename std::enable_if < + std::is_same::value || + std::is_same::value, int >::type = 0 > + basic_json(InputIT first, InputIT last) + { + JSON_ASSERT(first.m_object != nullptr); + JSON_ASSERT(last.m_object != nullptr); + + // make sure iterator fits the current value + if (JSON_HEDLEY_UNLIKELY(first.m_object != last.m_object)) + { + JSON_THROW(invalid_iterator::create(201, "iterators are not compatible")); + } + + // copy type from first iterator + m_type = first.m_object->m_type; + + // check if iterator range is complete for primitive values + switch (m_type) + { + case value_t::boolean: + case value_t::number_float: + case value_t::number_integer: + case value_t::number_unsigned: + case value_t::string: + { + if (JSON_HEDLEY_UNLIKELY(!first.m_it.primitive_iterator.is_begin() + || !last.m_it.primitive_iterator.is_end())) + { + JSON_THROW(invalid_iterator::create(204, "iterators out of range")); + } + break; + } + + default: + break; + } + + switch (m_type) + { + case value_t::number_integer: + { + m_value.number_integer = first.m_object->m_value.number_integer; + break; + } + + case value_t::number_unsigned: + { + m_value.number_unsigned = first.m_object->m_value.number_unsigned; + break; + } + + case value_t::number_float: + { + m_value.number_float = first.m_object->m_value.number_float; + break; + } + + case value_t::boolean: + { + m_value.boolean = first.m_object->m_value.boolean; + break; + } + + case value_t::string: + { + m_value = *first.m_object->m_value.string; + break; + } + + case value_t::object: + { + m_value.object = create(first.m_it.object_iterator, + last.m_it.object_iterator); + break; + } + + case value_t::array: + { + m_value.array = create(first.m_it.array_iterator, + last.m_it.array_iterator); + break; + } + + case value_t::binary: + { + m_value = *first.m_object->m_value.binary; + break; + } + + default: + JSON_THROW(invalid_iterator::create(206, "cannot construct with iterators from " + + std::string(first.m_object->type_name()))); + } + + assert_invariant(); + } + + + /////////////////////////////////////// + // other constructors and destructor // + /////////////////////////////////////// + + template, + std::is_same>::value, int> = 0 > + basic_json(const JsonRef& ref) : basic_json(ref.moved_or_copied()) {} + + /*! + @brief copy constructor + + Creates a copy of a given JSON value. + + @param[in] other the JSON value to copy + + @post `*this == other` + + @complexity Linear in the size of @a other. + + @exceptionsafety Strong guarantee: if an exception is thrown, there are no + changes to any JSON value. + + @requirement This function helps `basic_json` satisfying the + [Container](https://en.cppreference.com/w/cpp/named_req/Container) + requirements: + - The complexity is linear. + - As postcondition, it holds: `other == basic_json(other)`. + + @liveexample{The following code shows an example for the copy + constructor.,basic_json__basic_json} + + @since version 1.0.0 + */ + basic_json(const basic_json& other) + : m_type(other.m_type) + { + // check of passed value is valid + other.assert_invariant(); + + switch (m_type) + { + case value_t::object: + { + m_value = *other.m_value.object; + break; + } + + case value_t::array: + { + m_value = *other.m_value.array; + break; + } + + case value_t::string: + { + m_value = *other.m_value.string; + break; + } + + case value_t::boolean: + { + m_value = other.m_value.boolean; + break; + } + + case value_t::number_integer: + { + m_value = other.m_value.number_integer; + break; + } + + case value_t::number_unsigned: + { + m_value = other.m_value.number_unsigned; + break; + } + + case value_t::number_float: + { + m_value = other.m_value.number_float; + break; + } + + case value_t::binary: + { + m_value = *other.m_value.binary; + break; + } + + default: + break; + } + + assert_invariant(); + } + + /*! + @brief move constructor + + Move constructor. Constructs a JSON value with the contents of the given + value @a other using move semantics. It "steals" the resources from @a + other and leaves it as JSON null value. + + @param[in,out] other value to move to this object + + @post `*this` has the same value as @a other before the call. + @post @a other is a JSON null value. + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this constructor never throws + exceptions. + + @requirement This function helps `basic_json` satisfying the + [MoveConstructible](https://en.cppreference.com/w/cpp/named_req/MoveConstructible) + requirements. + + @liveexample{The code below shows the move constructor explicitly called + via std::move.,basic_json__moveconstructor} + + @since version 1.0.0 + */ + basic_json(basic_json&& other) noexcept + : m_type(std::move(other.m_type)), + m_value(std::move(other.m_value)) + { + // check that passed value is valid + other.assert_invariant(); + + // invalidate payload + other.m_type = value_t::null; + other.m_value = {}; + + assert_invariant(); + } + + /*! + @brief copy assignment + + Copy assignment operator. Copies a JSON value via the "copy and swap" + strategy: It is expressed in terms of the copy constructor, destructor, + and the `swap()` member function. + + @param[in] other value to copy from + + @complexity Linear. + + @requirement This function helps `basic_json` satisfying the + [Container](https://en.cppreference.com/w/cpp/named_req/Container) + requirements: + - The complexity is linear. + + @liveexample{The code below shows and example for the copy assignment. It + creates a copy of value `a` which is then swapped with `b`. Finally\, the + copy of `a` (which is the null value after the swap) is + destroyed.,basic_json__copyassignment} + + @since version 1.0.0 + */ + basic_json& operator=(basic_json other) noexcept ( + std::is_nothrow_move_constructible::value&& + std::is_nothrow_move_assignable::value&& + std::is_nothrow_move_constructible::value&& + std::is_nothrow_move_assignable::value + ) + { + // check that passed value is valid + other.assert_invariant(); + + using std::swap; + swap(m_type, other.m_type); + swap(m_value, other.m_value); + + assert_invariant(); + return *this; + } + + /*! + @brief destructor + + Destroys the JSON value and frees all allocated memory. + + @complexity Linear. + + @requirement This function helps `basic_json` satisfying the + [Container](https://en.cppreference.com/w/cpp/named_req/Container) + requirements: + - The complexity is linear. + - All stored elements are destroyed and all memory is freed. + + @since version 1.0.0 + */ + ~basic_json() noexcept + { + assert_invariant(); + m_value.destroy(m_type); + } + + /// @} + + public: + /////////////////////// + // object inspection // + /////////////////////// + + /// @name object inspection + /// Functions to inspect the type of a JSON value. + /// @{ + + /*! + @brief serialization + + Serialization function for JSON values. The function tries to mimic + Python's `json.dumps()` function, and currently supports its @a indent + and @a ensure_ascii parameters. + + @param[in] indent If indent is nonnegative, then array elements and object + members will be pretty-printed with that indent level. An indent level of + `0` will only insert newlines. `-1` (the default) selects the most compact + representation. + @param[in] indent_char The character to use for indentation if @a indent is + greater than `0`. The default is ` ` (space). + @param[in] ensure_ascii If @a ensure_ascii is true, all non-ASCII characters + in the output are escaped with `\uXXXX` sequences, and the result consists + of ASCII characters only. + @param[in] error_handler how to react on decoding errors; there are three + possible values: `strict` (throws and exception in case a decoding error + occurs; default), `replace` (replace invalid UTF-8 sequences with U+FFFD), + and `ignore` (ignore invalid UTF-8 sequences during serialization; all + bytes are copied to the output unchanged). + + @return string containing the serialization of the JSON value + + @throw type_error.316 if a string stored inside the JSON value is not + UTF-8 encoded and @a error_handler is set to strict + + @note Binary values are serialized as object containing two keys: + - "bytes": an array of bytes as integers + - "subtype": the subtype as integer or "null" if the binary has no subtype + + @complexity Linear. + + @exceptionsafety Strong guarantee: if an exception is thrown, there are no + changes in the JSON value. + + @liveexample{The following example shows the effect of different @a indent\, + @a indent_char\, and @a ensure_ascii parameters to the result of the + serialization.,dump} + + @see https://docs.python.org/2/library/json.html#json.dump + + @since version 1.0.0; indentation character @a indent_char, option + @a ensure_ascii and exceptions added in version 3.0.0; error + handlers added in version 3.4.0; serialization of binary values added + in version 3.8.0. + */ + string_t dump(const int indent = -1, + const char indent_char = ' ', + const bool ensure_ascii = false, + const error_handler_t error_handler = error_handler_t::strict) const + { + string_t result; + serializer s(detail::output_adapter(result), indent_char, error_handler); + + if (indent >= 0) + { + s.dump(*this, true, ensure_ascii, static_cast(indent)); + } + else + { + s.dump(*this, false, ensure_ascii, 0); + } + + return result; + } + + /*! + @brief return the type of the JSON value (explicit) + + Return the type of the JSON value as a value from the @ref value_t + enumeration. + + @return the type of the JSON value + Value type | return value + ------------------------- | ------------------------- + null | value_t::null + boolean | value_t::boolean + string | value_t::string + number (integer) | value_t::number_integer + number (unsigned integer) | value_t::number_unsigned + number (floating-point) | value_t::number_float + object | value_t::object + array | value_t::array + binary | value_t::binary + discarded | value_t::discarded + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this member function never throws + exceptions. + + @liveexample{The following code exemplifies `type()` for all JSON + types.,type} + + @sa @ref operator value_t() -- return the type of the JSON value (implicit) + @sa @ref type_name() -- return the type as string + + @since version 1.0.0 + */ + constexpr value_t type() const noexcept + { + return m_type; + } + + /*! + @brief return whether type is primitive + + This function returns true if and only if the JSON type is primitive + (string, number, boolean, or null). + + @return `true` if type is primitive (string, number, boolean, or null), + `false` otherwise. + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this member function never throws + exceptions. + + @liveexample{The following code exemplifies `is_primitive()` for all JSON + types.,is_primitive} + + @sa @ref is_structured() -- returns whether JSON value is structured + @sa @ref is_null() -- returns whether JSON value is `null` + @sa @ref is_string() -- returns whether JSON value is a string + @sa @ref is_boolean() -- returns whether JSON value is a boolean + @sa @ref is_number() -- returns whether JSON value is a number + @sa @ref is_binary() -- returns whether JSON value is a binary array + + @since version 1.0.0 + */ + constexpr bool is_primitive() const noexcept + { + return is_null() || is_string() || is_boolean() || is_number() || is_binary(); + } + + /*! + @brief return whether type is structured + + This function returns true if and only if the JSON type is structured + (array or object). + + @return `true` if type is structured (array or object), `false` otherwise. + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this member function never throws + exceptions. + + @liveexample{The following code exemplifies `is_structured()` for all JSON + types.,is_structured} + + @sa @ref is_primitive() -- returns whether value is primitive + @sa @ref is_array() -- returns whether value is an array + @sa @ref is_object() -- returns whether value is an object + + @since version 1.0.0 + */ + constexpr bool is_structured() const noexcept + { + return is_array() || is_object(); + } + + /*! + @brief return whether value is null + + This function returns true if and only if the JSON value is null. + + @return `true` if type is null, `false` otherwise. + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this member function never throws + exceptions. + + @liveexample{The following code exemplifies `is_null()` for all JSON + types.,is_null} + + @since version 1.0.0 + */ + constexpr bool is_null() const noexcept + { + return m_type == value_t::null; + } + + /*! + @brief return whether value is a boolean + + This function returns true if and only if the JSON value is a boolean. + + @return `true` if type is boolean, `false` otherwise. + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this member function never throws + exceptions. + + @liveexample{The following code exemplifies `is_boolean()` for all JSON + types.,is_boolean} + + @since version 1.0.0 + */ + constexpr bool is_boolean() const noexcept + { + return m_type == value_t::boolean; + } + + /*! + @brief return whether value is a number + + This function returns true if and only if the JSON value is a number. This + includes both integer (signed and unsigned) and floating-point values. + + @return `true` if type is number (regardless whether integer, unsigned + integer or floating-type), `false` otherwise. + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this member function never throws + exceptions. + + @liveexample{The following code exemplifies `is_number()` for all JSON + types.,is_number} + + @sa @ref is_number_integer() -- check if value is an integer or unsigned + integer number + @sa @ref is_number_unsigned() -- check if value is an unsigned integer + number + @sa @ref is_number_float() -- check if value is a floating-point number + + @since version 1.0.0 + */ + constexpr bool is_number() const noexcept + { + return is_number_integer() || is_number_float(); + } + + /*! + @brief return whether value is an integer number + + This function returns true if and only if the JSON value is a signed or + unsigned integer number. This excludes floating-point values. + + @return `true` if type is an integer or unsigned integer number, `false` + otherwise. + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this member function never throws + exceptions. + + @liveexample{The following code exemplifies `is_number_integer()` for all + JSON types.,is_number_integer} + + @sa @ref is_number() -- check if value is a number + @sa @ref is_number_unsigned() -- check if value is an unsigned integer + number + @sa @ref is_number_float() -- check if value is a floating-point number + + @since version 1.0.0 + */ + constexpr bool is_number_integer() const noexcept + { + return m_type == value_t::number_integer || m_type == value_t::number_unsigned; + } + + /*! + @brief return whether value is an unsigned integer number + + This function returns true if and only if the JSON value is an unsigned + integer number. This excludes floating-point and signed integer values. + + @return `true` if type is an unsigned integer number, `false` otherwise. + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this member function never throws + exceptions. + + @liveexample{The following code exemplifies `is_number_unsigned()` for all + JSON types.,is_number_unsigned} + + @sa @ref is_number() -- check if value is a number + @sa @ref is_number_integer() -- check if value is an integer or unsigned + integer number + @sa @ref is_number_float() -- check if value is a floating-point number + + @since version 2.0.0 + */ + constexpr bool is_number_unsigned() const noexcept + { + return m_type == value_t::number_unsigned; + } + + /*! + @brief return whether value is a floating-point number + + This function returns true if and only if the JSON value is a + floating-point number. This excludes signed and unsigned integer values. + + @return `true` if type is a floating-point number, `false` otherwise. + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this member function never throws + exceptions. + + @liveexample{The following code exemplifies `is_number_float()` for all + JSON types.,is_number_float} + + @sa @ref is_number() -- check if value is number + @sa @ref is_number_integer() -- check if value is an integer number + @sa @ref is_number_unsigned() -- check if value is an unsigned integer + number + + @since version 1.0.0 + */ + constexpr bool is_number_float() const noexcept + { + return m_type == value_t::number_float; + } + + /*! + @brief return whether value is an object + + This function returns true if and only if the JSON value is an object. + + @return `true` if type is object, `false` otherwise. + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this member function never throws + exceptions. + + @liveexample{The following code exemplifies `is_object()` for all JSON + types.,is_object} + + @since version 1.0.0 + */ + constexpr bool is_object() const noexcept + { + return m_type == value_t::object; + } + + /*! + @brief return whether value is an array + + This function returns true if and only if the JSON value is an array. + + @return `true` if type is array, `false` otherwise. + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this member function never throws + exceptions. + + @liveexample{The following code exemplifies `is_array()` for all JSON + types.,is_array} + + @since version 1.0.0 + */ + constexpr bool is_array() const noexcept + { + return m_type == value_t::array; + } + + /*! + @brief return whether value is a string + + This function returns true if and only if the JSON value is a string. + + @return `true` if type is string, `false` otherwise. + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this member function never throws + exceptions. + + @liveexample{The following code exemplifies `is_string()` for all JSON + types.,is_string} + + @since version 1.0.0 + */ + constexpr bool is_string() const noexcept + { + return m_type == value_t::string; + } + + /*! + @brief return whether value is a binary array + + This function returns true if and only if the JSON value is a binary array. + + @return `true` if type is binary array, `false` otherwise. + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this member function never throws + exceptions. + + @liveexample{The following code exemplifies `is_binary()` for all JSON + types.,is_binary} + + @since version 3.8.0 + */ + constexpr bool is_binary() const noexcept + { + return m_type == value_t::binary; + } + + /*! + @brief return whether value is discarded + + This function returns true if and only if the JSON value was discarded + during parsing with a callback function (see @ref parser_callback_t). + + @note This function will always be `false` for JSON values after parsing. + That is, discarded values can only occur during parsing, but will be + removed when inside a structured value or replaced by null in other cases. + + @return `true` if type is discarded, `false` otherwise. + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this member function never throws + exceptions. + + @liveexample{The following code exemplifies `is_discarded()` for all JSON + types.,is_discarded} + + @since version 1.0.0 + */ + constexpr bool is_discarded() const noexcept + { + return m_type == value_t::discarded; + } + + /*! + @brief return the type of the JSON value (implicit) + + Implicitly return the type of the JSON value as a value from the @ref + value_t enumeration. + + @return the type of the JSON value + + @complexity Constant. + + @exceptionsafety No-throw guarantee: this member function never throws + exceptions. + + @liveexample{The following code exemplifies the @ref value_t operator for + all JSON types.,operator__value_t} + + @sa @ref type() -- return the type of the JSON value (explicit) + @sa @ref type_name() -- return the type as string + + @since version 1.0.0 + */ + constexpr operator value_t() const noexcept + { + return m_type; + } + + /// @} + + private: + ////////////////// + // value access // + ////////////////// + + /// get a boolean (explicit) + boolean_t get_impl(boolean_t* /*unused*/) const + { + if (JSON_HEDLEY_LIKELY(is_boolean())) + { + return m_value.boolean; + } + + JSON_THROW(type_error::create(302, "type must be boolean, but is " + std::string(type_name()))); + } + + /// get a pointer to the value (object) + object_t* get_impl_ptr(object_t* /*unused*/) noexcept + { + return is_object() ? m_value.object : nullptr; + } + + /// get a pointer to the value (object) + constexpr const object_t* get_impl_ptr(const object_t* /*unused*/) const noexcept + { + return is_object() ? m_value.object : nullptr; + } + + /// get a pointer to the value (array) + array_t* get_impl_ptr(array_t* /*unused*/) noexcept + { + return is_array() ? m_value.array : nullptr; + } + + /// get a pointer to the value (array) + constexpr const array_t* get_impl_ptr(const array_t* /*unused*/) const noexcept + { + return is_array() ? m_value.array : nullptr; + } + + /// get a pointer to the value (string) + string_t* get_impl_ptr(string_t* /*unused*/) noexcept + { + return is_string() ? m_value.string : nullptr; + } + + /// get a pointer to the value (string) + constexpr const string_t* get_impl_ptr(const string_t* /*unused*/) const noexcept + { + return is_string() ? m_value.string : nullptr; + } + + /// get a pointer to the value (boolean) + boolean_t* get_impl_ptr(boolean_t* /*unused*/) noexcept + { + return is_boolean() ? &m_value.boolean : nullptr; + } + + /// get a pointer to the value (boolean) + constexpr const boolean_t* get_impl_ptr(const boolean_t* /*unused*/) const noexcept + { + return is_boolean() ? &m_value.boolean : nullptr; + } + + /// get a pointer to the value (integer number) + number_integer_t* get_impl_ptr(number_integer_t* /*unused*/) noexcept + { + return is_number_integer() ? &m_value.number_integer : nullptr; + } + + /// get a pointer to the value (integer number) + constexpr const number_integer_t* get_impl_ptr(const number_integer_t* /*unused*/) const noexcept + { + return is_number_integer() ? &m_value.number_integer : nullptr; + } + + /// get a pointer to the value (unsigned number) + number_unsigned_t* get_impl_ptr(number_unsigned_t* /*unused*/) noexcept + { + return is_number_unsigned() ? &m_value.number_unsigned : nullptr; + } + + /// get a pointer to the value (unsigned number) + constexpr const number_unsigned_t* get_impl_ptr(const number_unsigned_t* /*unused*/) const noexcept + { + return is_number_unsigned() ? &m_value.number_unsigned : nullptr; + } + + /// get a pointer to the value (floating-point number) + number_float_t* get_impl_ptr(number_float_t* /*unused*/) noexcept + { + return is_number_float() ? &m_value.number_float : nullptr; + } + + /// get a pointer to the value (floating-point number) + constexpr const number_float_t* get_impl_ptr(const number_float_t* /*unused*/) const noexcept + { + return is_number_float() ? &m_value.number_float : nullptr; + } + + /// get a pointer to the value (binary) + binary_t* get_impl_ptr(binary_t* /*unused*/) noexcept + { + return is_binary() ? m_value.binary : nullptr; + } + + /// get a pointer to the value (binary) + constexpr const binary_t* get_impl_ptr(const binary_t* /*unused*/) const noexcept + { + return is_binary() ? m_value.binary : nullptr; + } + + /*! + @brief helper function to implement get_ref() + + This function helps to implement get_ref() without code duplication for + const and non-const overloads + + @tparam ThisType will be deduced as `basic_json` or `const basic_json` + + @throw type_error.303 if ReferenceType does not match underlying value + type of the current JSON + */ + template + static ReferenceType get_ref_impl(ThisType& obj) + { + // delegate the call to get_ptr<>() + auto ptr = obj.template get_ptr::type>(); + + if (JSON_HEDLEY_LIKELY(ptr != nullptr)) + { + return *ptr; + } + + JSON_THROW(type_error::create(303, "incompatible ReferenceType for get_ref, actual type is " + std::string(obj.type_name()))); + } + + public: + /// @name value access + /// Direct access to the stored value of a JSON value. + /// @{ + + /*! + @brief get special-case overload + + This overloads avoids a lot of template boilerplate, it can be seen as the + identity method + + @tparam BasicJsonType == @ref basic_json + + @return a copy of *this + + @complexity Constant. + + @since version 2.1.0 + */ + template::type, basic_json_t>::value, + int> = 0> + basic_json get() const + { + return *this; + } + + /*! + @brief get special-case overload + + This overloads converts the current @ref basic_json in a different + @ref basic_json type + + @tparam BasicJsonType == @ref basic_json + + @return a copy of *this, converted into @tparam BasicJsonType + + @complexity Depending on the implementation of the called `from_json()` + method. + + @since version 3.2.0 + */ + template < typename BasicJsonType, detail::enable_if_t < + !std::is_same::value&& + detail::is_basic_json::value, int > = 0 > + BasicJsonType get() const + { + return *this; + } + + /*! + @brief get a value (explicit) + + Explicit type conversion between the JSON value and a compatible value + which is [CopyConstructible](https://en.cppreference.com/w/cpp/named_req/CopyConstructible) + and [DefaultConstructible](https://en.cppreference.com/w/cpp/named_req/DefaultConstructible). + The value is converted by calling the @ref json_serializer + `from_json()` method. + + The function is equivalent to executing + @code {.cpp} + ValueType ret; + JSONSerializer::from_json(*this, ret); + return ret; + @endcode + + This overloads is chosen if: + - @a ValueType is not @ref basic_json, + - @ref json_serializer has a `from_json()` method of the form + `void from_json(const basic_json&, ValueType&)`, and + - @ref json_serializer does not have a `from_json()` method of + the form `ValueType from_json(const basic_json&)` + + @tparam ValueTypeCV the provided value type + @tparam ValueType the returned value type + + @return copy of the JSON value, converted to @a ValueType + + @throw what @ref json_serializer `from_json()` method throws + + @liveexample{The example below shows several conversions from JSON values + to other types. There a few things to note: (1) Floating-point numbers can + be converted to integers\, (2) A JSON array can be converted to a standard + `std::vector`\, (3) A JSON object can be converted to C++ + associative containers such as `std::unordered_map`.,get__ValueType_const} + + @since version 2.1.0 + */ + template < typename ValueTypeCV, typename ValueType = detail::uncvref_t, + detail::enable_if_t < + !detail::is_basic_json::value && + detail::has_from_json::value && + !detail::has_non_default_from_json::value, + int > = 0 > + ValueType get() const noexcept(noexcept( + JSONSerializer::from_json(std::declval(), std::declval()))) + { + // we cannot static_assert on ValueTypeCV being non-const, because + // there is support for get(), which is why we + // still need the uncvref + static_assert(!std::is_reference::value, + "get() cannot be used with reference types, you might want to use get_ref()"); + static_assert(std::is_default_constructible::value, + "types must be DefaultConstructible when used with get()"); + + ValueType ret; + JSONSerializer::from_json(*this, ret); + return ret; + } + + /*! + @brief get a value (explicit); special case + + Explicit type conversion between the JSON value and a compatible value + which is **not** [CopyConstructible](https://en.cppreference.com/w/cpp/named_req/CopyConstructible) + and **not** [DefaultConstructible](https://en.cppreference.com/w/cpp/named_req/DefaultConstructible). + The value is converted by calling the @ref json_serializer + `from_json()` method. + + The function is equivalent to executing + @code {.cpp} + return JSONSerializer::from_json(*this); + @endcode + + This overloads is chosen if: + - @a ValueType is not @ref basic_json and + - @ref json_serializer has a `from_json()` method of the form + `ValueType from_json(const basic_json&)` + + @note If @ref json_serializer has both overloads of + `from_json()`, this one is chosen. + + @tparam ValueTypeCV the provided value type + @tparam ValueType the returned value type + + @return copy of the JSON value, converted to @a ValueType + + @throw what @ref json_serializer `from_json()` method throws + + @since version 2.1.0 + */ + template < typename ValueTypeCV, typename ValueType = detail::uncvref_t, + detail::enable_if_t < !std::is_same::value && + detail::has_non_default_from_json::value, + int > = 0 > + ValueType get() const noexcept(noexcept( + JSONSerializer::from_json(std::declval()))) + { + static_assert(!std::is_reference::value, + "get() cannot be used with reference types, you might want to use get_ref()"); + return JSONSerializer::from_json(*this); + } + + /*! + @brief get a value (explicit) + + Explicit type conversion between the JSON value and a compatible value. + The value is filled into the input parameter by calling the @ref json_serializer + `from_json()` method. + + The function is equivalent to executing + @code {.cpp} + ValueType v; + JSONSerializer::from_json(*this, v); + @endcode + + This overloads is chosen if: + - @a ValueType is not @ref basic_json, + - @ref json_serializer has a `from_json()` method of the form + `void from_json(const basic_json&, ValueType&)`, and + + @tparam ValueType the input parameter type. + + @return the input parameter, allowing chaining calls. + + @throw what @ref json_serializer `from_json()` method throws + + @liveexample{The example below shows several conversions from JSON values + to other types. There a few things to note: (1) Floating-point numbers can + be converted to integers\, (2) A JSON array can be converted to a standard + `std::vector`\, (3) A JSON object can be converted to C++ + associative containers such as `std::unordered_map`.,get_to} + + @since version 3.3.0 + */ + template < typename ValueType, + detail::enable_if_t < + !detail::is_basic_json::value&& + detail::has_from_json::value, + int > = 0 > + ValueType & get_to(ValueType& v) const noexcept(noexcept( + JSONSerializer::from_json(std::declval(), v))) + { + JSONSerializer::from_json(*this, v); + return v; + } + + // specialization to allow to call get_to with a basic_json value + // see https://github.com/nlohmann/json/issues/2175 + template::value, + int> = 0> + ValueType & get_to(ValueType& v) const + { + v = *this; + return v; + } + + template < + typename T, std::size_t N, + typename Array = T (&)[N], + detail::enable_if_t < + detail::has_from_json::value, int > = 0 > + Array get_to(T (&v)[N]) const + noexcept(noexcept(JSONSerializer::from_json( + std::declval(), v))) + { + JSONSerializer::from_json(*this, v); + return v; + } + + + /*! + @brief get a pointer value (implicit) + + Implicit pointer access to the internally stored JSON value. No copies are + made. + + @warning Writing data to the pointee of the result yields an undefined + state. + + @tparam PointerType pointer type; must be a pointer to @ref array_t, @ref + object_t, @ref string_t, @ref boolean_t, @ref number_integer_t, + @ref number_unsigned_t, or @ref number_float_t. Enforced by a static + assertion. + + @return pointer to the internally stored JSON value if the requested + pointer type @a PointerType fits to the JSON value; `nullptr` otherwise + + @complexity Constant. + + @liveexample{The example below shows how pointers to internal values of a + JSON value can be requested. Note that no type conversions are made and a + `nullptr` is returned if the value and the requested pointer type does not + match.,get_ptr} + + @since version 1.0.0 + */ + template::value, int>::type = 0> + auto get_ptr() noexcept -> decltype(std::declval().get_impl_ptr(std::declval())) + { + // delegate the call to get_impl_ptr<>() + return get_impl_ptr(static_cast(nullptr)); + } + + /*! + @brief get a pointer value (implicit) + @copydoc get_ptr() + */ + template < typename PointerType, typename std::enable_if < + std::is_pointer::value&& + std::is_const::type>::value, int >::type = 0 > + constexpr auto get_ptr() const noexcept -> decltype(std::declval().get_impl_ptr(std::declval())) + { + // delegate the call to get_impl_ptr<>() const + return get_impl_ptr(static_cast(nullptr)); + } + + /*! + @brief get a pointer value (explicit) + + Explicit pointer access to the internally stored JSON value. No copies are + made. + + @warning The pointer becomes invalid if the underlying JSON object + changes. + + @tparam PointerType pointer type; must be a pointer to @ref array_t, @ref + object_t, @ref string_t, @ref boolean_t, @ref number_integer_t, + @ref number_unsigned_t, or @ref number_float_t. + + @return pointer to the internally stored JSON value if the requested + pointer type @a PointerType fits to the JSON value; `nullptr` otherwise + + @complexity Constant. + + @liveexample{The example below shows how pointers to internal values of a + JSON value can be requested. Note that no type conversions are made and a + `nullptr` is returned if the value and the requested pointer type does not + match.,get__PointerType} + + @sa @ref get_ptr() for explicit pointer-member access + + @since version 1.0.0 + */ + template::value, int>::type = 0> + auto get() noexcept -> decltype(std::declval().template get_ptr()) + { + // delegate the call to get_ptr + return get_ptr(); + } + + /*! + @brief get a pointer value (explicit) + @copydoc get() + */ + template::value, int>::type = 0> + constexpr auto get() const noexcept -> decltype(std::declval().template get_ptr()) + { + // delegate the call to get_ptr + return get_ptr(); + } + + /*! + @brief get a reference value (implicit) + + Implicit reference access to the internally stored JSON value. No copies + are made. + + @warning Writing data to the referee of the result yields an undefined + state. + + @tparam ReferenceType reference type; must be a reference to @ref array_t, + @ref object_t, @ref string_t, @ref boolean_t, @ref number_integer_t, or + @ref number_float_t. Enforced by static assertion. + + @return reference to the internally stored JSON value if the requested + reference type @a ReferenceType fits to the JSON value; throws + type_error.303 otherwise + + @throw type_error.303 in case passed type @a ReferenceType is incompatible + with the stored JSON value; see example below + + @complexity Constant. + + @liveexample{The example shows several calls to `get_ref()`.,get_ref} + + @since version 1.1.0 + */ + template::value, int>::type = 0> + ReferenceType get_ref() + { + // delegate call to get_ref_impl + return get_ref_impl(*this); + } + + /*! + @brief get a reference value (implicit) + @copydoc get_ref() + */ + template < typename ReferenceType, typename std::enable_if < + std::is_reference::value&& + std::is_const::type>::value, int >::type = 0 > + ReferenceType get_ref() const + { + // delegate call to get_ref_impl + return get_ref_impl(*this); + } + + /*! + @brief get a value (implicit) + + Implicit type conversion between the JSON value and a compatible value. + The call is realized by calling @ref get() const. + + @tparam ValueType non-pointer type compatible to the JSON value, for + instance `int` for JSON integer numbers, `bool` for JSON booleans, or + `std::vector` types for JSON arrays. The character type of @ref string_t + as well as an initializer list of this type is excluded to avoid + ambiguities as these types implicitly convert to `std::string`. + + @return copy of the JSON value, converted to type @a ValueType + + @throw type_error.302 in case passed type @a ValueType is incompatible + to the JSON value type (e.g., the JSON value is of type boolean, but a + string is requested); see example below + + @complexity Linear in the size of the JSON value. + + @liveexample{The example below shows several conversions from JSON values + to other types. There a few things to note: (1) Floating-point numbers can + be converted to integers\, (2) A JSON array can be converted to a standard + `std::vector`\, (3) A JSON object can be converted to C++ + associative containers such as `std::unordered_map`.,operator__ValueType} + + @since version 1.0.0 + */ + template < typename ValueType, typename std::enable_if < + !std::is_pointer::value&& + !std::is_same>::value&& + !std::is_same::value&& + !detail::is_basic_json::value + && !std::is_same>::value +#if defined(JSON_HAS_CPP_17) && (defined(__GNUC__) || (defined(_MSC_VER) && _MSC_VER >= 1910 && _MSC_VER <= 1914)) + && !std::is_same::value +#endif + && detail::is_detected::value + , int >::type = 0 > + JSON_EXPLICIT operator ValueType() const + { + // delegate the call to get<>() const + return get(); + } + + /*! + @return reference to the binary value + + @throw type_error.302 if the value is not binary + + @sa @ref is_binary() to check if the value is binary + + @since version 3.8.0 + */ + binary_t& get_binary() + { + if (!is_binary()) + { + JSON_THROW(type_error::create(302, "type must be binary, but is " + std::string(type_name()))); + } + + return *get_ptr(); + } + + /// @copydoc get_binary() + const binary_t& get_binary() const + { + if (!is_binary()) + { + JSON_THROW(type_error::create(302, "type must be binary, but is " + std::string(type_name()))); + } + + return *get_ptr(); + } + + /// @} + + + //////////////////// + // element access // + //////////////////// + + /// @name element access + /// Access to the JSON value. + /// @{ + + /*! + @brief access specified array element with bounds checking + + Returns a reference to the element at specified location @a idx, with + bounds checking. + + @param[in] idx index of the element to access + + @return reference to the element at index @a idx + + @throw type_error.304 if the JSON value is not an array; in this case, + calling `at` with an index makes no sense. See example below. + @throw out_of_range.401 if the index @a idx is out of range of the array; + that is, `idx >= size()`. See example below. + + @exceptionsafety Strong guarantee: if an exception is thrown, there are no + changes in the JSON value. + + @complexity Constant. + + @since version 1.0.0 + + @liveexample{The example below shows how array elements can be read and + written using `at()`. It also demonstrates the different exceptions that + can be thrown.,at__size_type} + */ + reference at(size_type idx) + { + // at only works for arrays + if (JSON_HEDLEY_LIKELY(is_array())) + { + JSON_TRY + { + return m_value.array->at(idx); + } + JSON_CATCH (std::out_of_range&) + { + // create better exception explanation + JSON_THROW(out_of_range::create(401, "array index " + std::to_string(idx) + " is out of range")); + } + } + else + { + JSON_THROW(type_error::create(304, "cannot use at() with " + std::string(type_name()))); + } + } + + /*! + @brief access specified array element with bounds checking + + Returns a const reference to the element at specified location @a idx, + with bounds checking. + + @param[in] idx index of the element to access + + @return const reference to the element at index @a idx + + @throw type_error.304 if the JSON value is not an array; in this case, + calling `at` with an index makes no sense. See example below. + @throw out_of_range.401 if the index @a idx is out of range of the array; + that is, `idx >= size()`. See example below. + + @exceptionsafety Strong guarantee: if an exception is thrown, there are no + changes in the JSON value. + + @complexity Constant. + + @since version 1.0.0 + + @liveexample{The example below shows how array elements can be read using + `at()`. It also demonstrates the different exceptions that can be thrown., + at__size_type_const} + */ + const_reference at(size_type idx) const + { + // at only works for arrays + if (JSON_HEDLEY_LIKELY(is_array())) + { + JSON_TRY + { + return m_value.array->at(idx); + } + JSON_CATCH (std::out_of_range&) + { + // create better exception explanation + JSON_THROW(out_of_range::create(401, "array index " + std::to_string(idx) + " is out of range")); + } + } + else + { + JSON_THROW(type_error::create(304, "cannot use at() with " + std::string(type_name()))); + } + } + + /*! + @brief access specified object element with bounds checking + + Returns a reference to the element at with specified key @a key, with + bounds checking. + + @param[in] key key of the element to access + + @return reference to the element at key @a key + + @throw type_error.304 if the JSON value is not an object; in this case, + calling `at` with a key makes no sense. See example below. + @throw out_of_range.403 if the key @a key is is not stored in the object; + that is, `find(key) == end()`. See example below. + + @exceptionsafety Strong guarantee: if an exception is thrown, there are no + changes in the JSON value. + + @complexity Logarithmic in the size of the container. + + @sa @ref operator[](const typename object_t::key_type&) for unchecked + access by reference + @sa @ref value() for access by value with a default value + + @since version 1.0.0 + + @liveexample{The example below shows how object elements can be read and + written using `at()`. It also demonstrates the different exceptions that + can be thrown.,at__object_t_key_type} + */ + reference at(const typename object_t::key_type& key) + { + // at only works for objects + if (JSON_HEDLEY_LIKELY(is_object())) + { + JSON_TRY + { + return m_value.object->at(key); + } + JSON_CATCH (std::out_of_range&) + { + // create better exception explanation + JSON_THROW(out_of_range::create(403, "key '" + key + "' not found")); + } + } + else + { + JSON_THROW(type_error::create(304, "cannot use at() with " + std::string(type_name()))); + } + } + + /*! + @brief access specified object element with bounds checking + + Returns a const reference to the element at with specified key @a key, + with bounds checking. + + @param[in] key key of the element to access + + @return const reference to the element at key @a key + + @throw type_error.304 if the JSON value is not an object; in this case, + calling `at` with a key makes no sense. See example below. + @throw out_of_range.403 if the key @a key is is not stored in the object; + that is, `find(key) == end()`. See example below. + + @exceptionsafety Strong guarantee: if an exception is thrown, there are no + changes in the JSON value. + + @complexity Logarithmic in the size of the container. + + @sa @ref operator[](const typename object_t::key_type&) for unchecked + access by reference + @sa @ref value() for access by value with a default value + + @since version 1.0.0 + + @liveexample{The example below shows how object elements can be read using + `at()`. It also demonstrates the different exceptions that can be thrown., + at__object_t_key_type_const} + */ + const_reference at(const typename object_t::key_type& key) const + { + // at only works for objects + if (JSON_HEDLEY_LIKELY(is_object())) + { + JSON_TRY + { + return m_value.object->at(key); + } + JSON_CATCH (std::out_of_range&) + { + // create better exception explanation + JSON_THROW(out_of_range::create(403, "key '" + key + "' not found")); + } + } + else + { + JSON_THROW(type_error::create(304, "cannot use at() with " + std::string(type_name()))); + } + } + + /*! + @brief access specified array element + + Returns a reference to the element at specified location @a idx. + + @note If @a idx is beyond the range of the array (i.e., `idx >= size()`), + then the array is silently filled up with `null` values to make `idx` a + valid reference to the last stored element. + + @param[in] idx index of the element to access + + @return reference to the element at index @a idx + + @throw type_error.305 if the JSON value is not an array or null; in that + cases, using the [] operator with an index makes no sense. + + @complexity Constant if @a idx is in the range of the array. Otherwise + linear in `idx - size()`. + + @liveexample{The example below shows how array elements can be read and + written using `[]` operator. Note the addition of `null` + values.,operatorarray__size_type} + + @since version 1.0.0 + */ + reference operator[](size_type idx) + { + // implicitly convert null value to an empty array + if (is_null()) + { + m_type = value_t::array; + m_value.array = create(); + assert_invariant(); + } + + // operator[] only works for arrays + if (JSON_HEDLEY_LIKELY(is_array())) + { + // fill up array with null values if given idx is outside range + if (idx >= m_value.array->size()) + { + m_value.array->insert(m_value.array->end(), + idx - m_value.array->size() + 1, + basic_json()); + } + + return m_value.array->operator[](idx); + } + + JSON_THROW(type_error::create(305, "cannot use operator[] with a numeric argument with " + std::string(type_name()))); + } + + /*! + @brief access specified array element + + Returns a const reference to the element at specified location @a idx. + + @param[in] idx index of the element to access + + @return const reference to the element at index @a idx + + @throw type_error.305 if the JSON value is not an array; in that case, + using the [] operator with an index makes no sense. + + @complexity Constant. + + @liveexample{The example below shows how array elements can be read using + the `[]` operator.,operatorarray__size_type_const} + + @since version 1.0.0 + */ + const_reference operator[](size_type idx) const + { + // const operator[] only works for arrays + if (JSON_HEDLEY_LIKELY(is_array())) + { + return m_value.array->operator[](idx); + } + + JSON_THROW(type_error::create(305, "cannot use operator[] with a numeric argument with " + std::string(type_name()))); + } + + /*! + @brief access specified object element + + Returns a reference to the element at with specified key @a key. + + @note If @a key is not found in the object, then it is silently added to + the object and filled with a `null` value to make `key` a valid reference. + In case the value was `null` before, it is converted to an object. + + @param[in] key key of the element to access + + @return reference to the element at key @a key + + @throw type_error.305 if the JSON value is not an object or null; in that + cases, using the [] operator with a key makes no sense. + + @complexity Logarithmic in the size of the container. + + @liveexample{The example below shows how object elements can be read and + written using the `[]` operator.,operatorarray__key_type} + + @sa @ref at(const typename object_t::key_type&) for access by reference + with range checking + @sa @ref value() for access by value with a default value + + @since version 1.0.0 + */ + reference operator[](const typename object_t::key_type& key) + { + // implicitly convert null value to an empty object + if (is_null()) + { + m_type = value_t::object; + m_value.object = create(); + assert_invariant(); + } + + // operator[] only works for objects + if (JSON_HEDLEY_LIKELY(is_object())) + { + return m_value.object->operator[](key); + } + + JSON_THROW(type_error::create(305, "cannot use operator[] with a string argument with " + std::string(type_name()))); + } + + /*! + @brief read-only access specified object element + + Returns a const reference to the element at with specified key @a key. No + bounds checking is performed. + + @warning If the element with key @a key does not exist, the behavior is + undefined. + + @param[in] key key of the element to access + + @return const reference to the element at key @a key + + @pre The element with key @a key must exist. **This precondition is + enforced with an assertion.** + + @throw type_error.305 if the JSON value is not an object; in that case, + using the [] operator with a key makes no sense. + + @complexity Logarithmic in the size of the container. + + @liveexample{The example below shows how object elements can be read using + the `[]` operator.,operatorarray__key_type_const} + + @sa @ref at(const typename object_t::key_type&) for access by reference + with range checking + @sa @ref value() for access by value with a default value + + @since version 1.0.0 + */ + const_reference operator[](const typename object_t::key_type& key) const + { + // const operator[] only works for objects + if (JSON_HEDLEY_LIKELY(is_object())) + { + JSON_ASSERT(m_value.object->find(key) != m_value.object->end()); + return m_value.object->find(key)->second; + } + + JSON_THROW(type_error::create(305, "cannot use operator[] with a string argument with " + std::string(type_name()))); + } + + /*! + @brief access specified object element + + Returns a reference to the element at with specified key @a key. + + @note If @a key is not found in the object, then it is silently added to + the object and filled with a `null` value to make `key` a valid reference. + In case the value was `null` before, it is converted to an object. + + @param[in] key key of the element to access + + @return reference to the element at key @a key + + @throw type_error.305 if the JSON value is not an object or null; in that + cases, using the [] operator with a key makes no sense. + + @complexity Logarithmic in the size of the container. + + @liveexample{The example below shows how object elements can be read and + written using the `[]` operator.,operatorarray__key_type} + + @sa @ref at(const typename object_t::key_type&) for access by reference + with range checking + @sa @ref value() for access by value with a default value + + @since version 1.1.0 + */ + template + JSON_HEDLEY_NON_NULL(2) + reference operator[](T* key) + { + // implicitly convert null to object + if (is_null()) + { + m_type = value_t::object; + m_value = value_t::object; + assert_invariant(); + } + + // at only works for objects + if (JSON_HEDLEY_LIKELY(is_object())) + { + return m_value.object->operator[](key); + } + + JSON_THROW(type_error::create(305, "cannot use operator[] with a string argument with " + std::string(type_name()))); + } + + /*! + @brief read-only access specified object element + + Returns a const reference to the element at with specified key @a key. No + bounds checking is performed. + + @warning If the element with key @a key does not exist, the behavior is + undefined. + + @param[in] key key of the element to access + + @return const reference to the element at key @a key + + @pre The element with key @a key must exist. **This precondition is + enforced with an assertion.** + + @throw type_error.305 if the JSON value is not an object; in that case, + using the [] operator with a key makes no sense. + + @complexity Logarithmic in the size of the container. + + @liveexample{The example below shows how object elements can be read using + the `[]` operator.,operatorarray__key_type_const} + + @sa @ref at(const typename object_t::key_type&) for access by reference + with range checking + @sa @ref value() for access by value with a default value + + @since version 1.1.0 + */ + template + JSON_HEDLEY_NON_NULL(2) + const_reference operator[](T* key) const + { + // at only works for objects + if (JSON_HEDLEY_LIKELY(is_object())) + { + JSON_ASSERT(m_value.object->find(key) != m_value.object->end()); + return m_value.object->find(key)->second; + } + + JSON_THROW(type_error::create(305, "cannot use operator[] with a string argument with " + std::string(type_name()))); + } + + /*! + @brief access specified object element with default value + + Returns either a copy of an object's element at the specified key @a key + or a given default value if no element with key @a key exists. + + The function is basically equivalent to executing + @code {.cpp} + try { + return at(key); + } catch(out_of_range) { + return default_value; + } + @endcode + + @note Unlike @ref at(const typename object_t::key_type&), this function + does not throw if the given key @a key was not found. + + @note Unlike @ref operator[](const typename object_t::key_type& key), this + function does not implicitly add an element to the position defined by @a + key. This function is furthermore also applicable to const objects. + + @param[in] key key of the element to access + @param[in] default_value the value to return if @a key is not found + + @tparam ValueType type compatible to JSON values, for instance `int` for + JSON integer numbers, `bool` for JSON booleans, or `std::vector` types for + JSON arrays. Note the type of the expected value at @a key and the default + value @a default_value must be compatible. + + @return copy of the element at key @a key or @a default_value if @a key + is not found + + @throw type_error.302 if @a default_value does not match the type of the + value at @a key + @throw type_error.306 if the JSON value is not an object; in that case, + using `value()` with a key makes no sense. + + @complexity Logarithmic in the size of the container. + + @liveexample{The example below shows how object elements can be queried + with a default value.,basic_json__value} + + @sa @ref at(const typename object_t::key_type&) for access by reference + with range checking + @sa @ref operator[](const typename object_t::key_type&) for unchecked + access by reference + + @since version 1.0.0 + */ + // using std::is_convertible in a std::enable_if will fail when using explicit conversions + template < class ValueType, typename std::enable_if < + detail::is_getable::value + && !std::is_same::value, int >::type = 0 > + ValueType value(const typename object_t::key_type& key, const ValueType& default_value) const + { + // at only works for objects + if (JSON_HEDLEY_LIKELY(is_object())) + { + // if key is found, return value and given default value otherwise + const auto it = find(key); + if (it != end()) + { + return it->template get(); + } + + return default_value; + } + + JSON_THROW(type_error::create(306, "cannot use value() with " + std::string(type_name()))); + } + + /*! + @brief overload for a default value of type const char* + @copydoc basic_json::value(const typename object_t::key_type&, const ValueType&) const + */ + string_t value(const typename object_t::key_type& key, const char* default_value) const + { + return value(key, string_t(default_value)); + } + + /*! + @brief access specified object element via JSON Pointer with default value + + Returns either a copy of an object's element at the specified key @a key + or a given default value if no element with key @a key exists. + + The function is basically equivalent to executing + @code {.cpp} + try { + return at(ptr); + } catch(out_of_range) { + return default_value; + } + @endcode + + @note Unlike @ref at(const json_pointer&), this function does not throw + if the given key @a key was not found. + + @param[in] ptr a JSON pointer to the element to access + @param[in] default_value the value to return if @a ptr found no value + + @tparam ValueType type compatible to JSON values, for instance `int` for + JSON integer numbers, `bool` for JSON booleans, or `std::vector` types for + JSON arrays. Note the type of the expected value at @a key and the default + value @a default_value must be compatible. + + @return copy of the element at key @a key or @a default_value if @a key + is not found + + @throw type_error.302 if @a default_value does not match the type of the + value at @a ptr + @throw type_error.306 if the JSON value is not an object; in that case, + using `value()` with a key makes no sense. + + @complexity Logarithmic in the size of the container. + + @liveexample{The example below shows how object elements can be queried + with a default value.,basic_json__value_ptr} + + @sa @ref operator[](const json_pointer&) for unchecked access by reference + + @since version 2.0.2 + */ + template::value, int>::type = 0> + ValueType value(const json_pointer& ptr, const ValueType& default_value) const + { + // at only works for objects + if (JSON_HEDLEY_LIKELY(is_object())) + { + // if pointer resolves a value, return it or use default value + JSON_TRY + { + return ptr.get_checked(this).template get(); + } + JSON_INTERNAL_CATCH (out_of_range&) + { + return default_value; + } + } + + JSON_THROW(type_error::create(306, "cannot use value() with " + std::string(type_name()))); + } + + /*! + @brief overload for a default value of type const char* + @copydoc basic_json::value(const json_pointer&, ValueType) const + */ + JSON_HEDLEY_NON_NULL(3) + string_t value(const json_pointer& ptr, const char* default_value) const + { + return value(ptr, string_t(default_value)); + } + + /*! + @brief access the first element + + Returns a reference to the first element in the container. For a JSON + container `c`, the expression `c.front()` is equivalent to `*c.begin()`. + + @return In case of a structured type (array or object), a reference to the + first element is returned. In case of number, string, boolean, or binary + values, a reference to the value is returned. + + @complexity Constant. + + @pre The JSON value must not be `null` (would throw `std::out_of_range`) + or an empty array or object (undefined behavior, **guarded by + assertions**). + @post The JSON value remains unchanged. + + @throw invalid_iterator.214 when called on `null` value + + @liveexample{The following code shows an example for `front()`.,front} + + @sa @ref back() -- access the last element + + @since version 1.0.0 + */ + reference front() + { + return *begin(); + } + + /*! + @copydoc basic_json::front() + */ + const_reference front() const + { + return *cbegin(); + } + + /*! + @brief access the last element + + Returns a reference to the last element in the container. For a JSON + container `c`, the expression `c.back()` is equivalent to + @code {.cpp} + auto tmp = c.end(); + --tmp; + return *tmp; + @endcode + + @return In case of a structured type (array or object), a reference to the + last element is returned. In case of number, string, boolean, or binary + values, a reference to the value is returned. + + @complexity Constant. + + @pre The JSON value must not be `null` (would throw `std::out_of_range`) + or an empty array or object (undefined behavior, **guarded by + assertions**). + @post The JSON value remains unchanged. + + @throw invalid_iterator.214 when called on a `null` value. See example + below. + + @liveexample{The following code shows an example for `back()`.,back} + + @sa @ref front() -- access the first element + + @since version 1.0.0 + */ + reference back() + { + auto tmp = end(); + --tmp; + return *tmp; + } + + /*! + @copydoc basic_json::back() + */ + const_reference back() const + { + auto tmp = cend(); + --tmp; + return *tmp; + } + + /*! + @brief remove element given an iterator + + Removes the element specified by iterator @a pos. The iterator @a pos must + be valid and dereferenceable. Thus the `end()` iterator (which is valid, + but is not dereferenceable) cannot be used as a value for @a pos. + + If called on a primitive type other than `null`, the resulting JSON value + will be `null`. + + @param[in] pos iterator to the element to remove + @return Iterator following the last removed element. If the iterator @a + pos refers to the last element, the `end()` iterator is returned. + + @tparam IteratorType an @ref iterator or @ref const_iterator + + @post Invalidates iterators and references at or after the point of the + erase, including the `end()` iterator. + + @throw type_error.307 if called on a `null` value; example: `"cannot use + erase() with null"` + @throw invalid_iterator.202 if called on an iterator which does not belong + to the current JSON value; example: `"iterator does not fit current + value"` + @throw invalid_iterator.205 if called on a primitive type with invalid + iterator (i.e., any iterator which is not `begin()`); example: `"iterator + out of range"` + + @complexity The complexity depends on the type: + - objects: amortized constant + - arrays: linear in distance between @a pos and the end of the container + - strings and binary: linear in the length of the member + - other types: constant + + @liveexample{The example shows the result of `erase()` for different JSON + types.,erase__IteratorType} + + @sa @ref erase(IteratorType, IteratorType) -- removes the elements in + the given range + @sa @ref erase(const typename object_t::key_type&) -- removes the element + from an object at the given key + @sa @ref erase(const size_type) -- removes the element from an array at + the given index + + @since version 1.0.0 + */ + template < class IteratorType, typename std::enable_if < + std::is_same::value || + std::is_same::value, int >::type + = 0 > + IteratorType erase(IteratorType pos) + { + // make sure iterator fits the current value + if (JSON_HEDLEY_UNLIKELY(this != pos.m_object)) + { + JSON_THROW(invalid_iterator::create(202, "iterator does not fit current value")); + } + + IteratorType result = end(); + + switch (m_type) + { + case value_t::boolean: + case value_t::number_float: + case value_t::number_integer: + case value_t::number_unsigned: + case value_t::string: + case value_t::binary: + { + if (JSON_HEDLEY_UNLIKELY(!pos.m_it.primitive_iterator.is_begin())) + { + JSON_THROW(invalid_iterator::create(205, "iterator out of range")); + } + + if (is_string()) + { + AllocatorType alloc; + std::allocator_traits::destroy(alloc, m_value.string); + std::allocator_traits::deallocate(alloc, m_value.string, 1); + m_value.string = nullptr; + } + else if (is_binary()) + { + AllocatorType alloc; + std::allocator_traits::destroy(alloc, m_value.binary); + std::allocator_traits::deallocate(alloc, m_value.binary, 1); + m_value.binary = nullptr; + } + + m_type = value_t::null; + assert_invariant(); + break; + } + + case value_t::object: + { + result.m_it.object_iterator = m_value.object->erase(pos.m_it.object_iterator); + break; + } + + case value_t::array: + { + result.m_it.array_iterator = m_value.array->erase(pos.m_it.array_iterator); + break; + } + + default: + JSON_THROW(type_error::create(307, "cannot use erase() with " + std::string(type_name()))); + } + + return result; + } + + /*! + @brief remove elements given an iterator range + + Removes the element specified by the range `[first; last)`. The iterator + @a first does not need to be dereferenceable if `first == last`: erasing + an empty range is a no-op. + + If called on a primitive type other than `null`, the resulting JSON value + will be `null`. + + @param[in] first iterator to the beginning of the range to remove + @param[in] last iterator past the end of the range to remove + @return Iterator following the last removed element. If the iterator @a + second refers to the last element, the `end()` iterator is returned. + + @tparam IteratorType an @ref iterator or @ref const_iterator + + @post Invalidates iterators and references at or after the point of the + erase, including the `end()` iterator. + + @throw type_error.307 if called on a `null` value; example: `"cannot use + erase() with null"` + @throw invalid_iterator.203 if called on iterators which does not belong + to the current JSON value; example: `"iterators do not fit current value"` + @throw invalid_iterator.204 if called on a primitive type with invalid + iterators (i.e., if `first != begin()` and `last != end()`); example: + `"iterators out of range"` + + @complexity The complexity depends on the type: + - objects: `log(size()) + std::distance(first, last)` + - arrays: linear in the distance between @a first and @a last, plus linear + in the distance between @a last and end of the container + - strings and binary: linear in the length of the member + - other types: constant + + @liveexample{The example shows the result of `erase()` for different JSON + types.,erase__IteratorType_IteratorType} + + @sa @ref erase(IteratorType) -- removes the element at a given position + @sa @ref erase(const typename object_t::key_type&) -- removes the element + from an object at the given key + @sa @ref erase(const size_type) -- removes the element from an array at + the given index + + @since version 1.0.0 + */ + template < class IteratorType, typename std::enable_if < + std::is_same::value || + std::is_same::value, int >::type + = 0 > + IteratorType erase(IteratorType first, IteratorType last) + { + // make sure iterator fits the current value + if (JSON_HEDLEY_UNLIKELY(this != first.m_object || this != last.m_object)) + { + JSON_THROW(invalid_iterator::create(203, "iterators do not fit current value")); + } + + IteratorType result = end(); + + switch (m_type) + { + case value_t::boolean: + case value_t::number_float: + case value_t::number_integer: + case value_t::number_unsigned: + case value_t::string: + case value_t::binary: + { + if (JSON_HEDLEY_LIKELY(!first.m_it.primitive_iterator.is_begin() + || !last.m_it.primitive_iterator.is_end())) + { + JSON_THROW(invalid_iterator::create(204, "iterators out of range")); + } + + if (is_string()) + { + AllocatorType alloc; + std::allocator_traits::destroy(alloc, m_value.string); + std::allocator_traits::deallocate(alloc, m_value.string, 1); + m_value.string = nullptr; + } + else if (is_binary()) + { + AllocatorType alloc; + std::allocator_traits::destroy(alloc, m_value.binary); + std::allocator_traits::deallocate(alloc, m_value.binary, 1); + m_value.binary = nullptr; + } + + m_type = value_t::null; + assert_invariant(); + break; + } + + case value_t::object: + { + result.m_it.object_iterator = m_value.object->erase(first.m_it.object_iterator, + last.m_it.object_iterator); + break; + } + + case value_t::array: + { + result.m_it.array_iterator = m_value.array->erase(first.m_it.array_iterator, + last.m_it.array_iterator); + break; + } + + default: + JSON_THROW(type_error::create(307, "cannot use erase() with " + std::string(type_name()))); + } + + return result; + } + + /*! + @brief remove element from a JSON object given a key + + Removes elements from a JSON object with the key value @a key. + + @param[in] key value of the elements to remove + + @return Number of elements removed. If @a ObjectType is the default + `std::map` type, the return value will always be `0` (@a key was not + found) or `1` (@a key was found). + + @post References and iterators to the erased elements are invalidated. + Other references and iterators are not affected. + + @throw type_error.307 when called on a type other than JSON object; + example: `"cannot use erase() with null"` + + @complexity `log(size()) + count(key)` + + @liveexample{The example shows the effect of `erase()`.,erase__key_type} + + @sa @ref erase(IteratorType) -- removes the element at a given position + @sa @ref erase(IteratorType, IteratorType) -- removes the elements in + the given range + @sa @ref erase(const size_type) -- removes the element from an array at + the given index + + @since version 1.0.0 + */ + size_type erase(const typename object_t::key_type& key) + { + // this erase only works for objects + if (JSON_HEDLEY_LIKELY(is_object())) + { + return m_value.object->erase(key); + } + + JSON_THROW(type_error::create(307, "cannot use erase() with " + std::string(type_name()))); + } + + /*! + @brief remove element from a JSON array given an index + + Removes element from a JSON array at the index @a idx. + + @param[in] idx index of the element to remove + + @throw type_error.307 when called on a type other than JSON object; + example: `"cannot use erase() with null"` + @throw out_of_range.401 when `idx >= size()`; example: `"array index 17 + is out of range"` + + @complexity Linear in distance between @a idx and the end of the container. + + @liveexample{The example shows the effect of `erase()`.,erase__size_type} + + @sa @ref erase(IteratorType) -- removes the element at a given position + @sa @ref erase(IteratorType, IteratorType) -- removes the elements in + the given range + @sa @ref erase(const typename object_t::key_type&) -- removes the element + from an object at the given key + + @since version 1.0.0 + */ + void erase(const size_type idx) + { + // this erase only works for arrays + if (JSON_HEDLEY_LIKELY(is_array())) + { + if (JSON_HEDLEY_UNLIKELY(idx >= size())) + { + JSON_THROW(out_of_range::create(401, "array index " + std::to_string(idx) + " is out of range")); + } + + m_value.array->erase(m_value.array->begin() + static_cast(idx)); + } + else + { + JSON_THROW(type_error::create(307, "cannot use erase() with " + std::string(type_name()))); + } + } + + /// @} + + + //////////// + // lookup // + //////////// + + /// @name lookup + /// @{ + + /*! + @brief find an element in a JSON object + + Finds an element in a JSON object with key equivalent to @a key. If the + element is not found or the JSON value is not an object, end() is + returned. + + @note This method always returns @ref end() when executed on a JSON type + that is not an object. + + @param[in] key key value of the element to search for. + + @return Iterator to an element with key equivalent to @a key. If no such + element is found or the JSON value is not an object, past-the-end (see + @ref end()) iterator is returned. + + @complexity Logarithmic in the size of the JSON object. + + @liveexample{The example shows how `find()` is used.,find__key_type} + + @sa @ref contains(KeyT&&) const -- checks whether a key exists + + @since version 1.0.0 + */ + template + iterator find(KeyT&& key) + { + auto result = end(); + + if (is_object()) + { + result.m_it.object_iterator = m_value.object->find(std::forward(key)); + } + + return result; + } + + /*! + @brief find an element in a JSON object + @copydoc find(KeyT&&) + */ + template + const_iterator find(KeyT&& key) const + { + auto result = cend(); + + if (is_object()) + { + result.m_it.object_iterator = m_value.object->find(std::forward(key)); + } + + return result; + } + + /*! + @brief returns the number of occurrences of a key in a JSON object + + Returns the number of elements with key @a key. If ObjectType is the + default `std::map` type, the return value will always be `0` (@a key was + not found) or `1` (@a key was found). + + @note This method always returns `0` when executed on a JSON type that is + not an object. + + @param[in] key key value of the element to count + + @return Number of elements with key @a key. If the JSON value is not an + object, the return value will be `0`. + + @complexity Logarithmic in the size of the JSON object. + + @liveexample{The example shows how `count()` is used.,count} + + @since version 1.0.0 + */ + template + size_type count(KeyT&& key) const + { + // return 0 for all nonobject types + return is_object() ? m_value.object->count(std::forward(key)) : 0; + } + + /*! + @brief check the existence of an element in a JSON object + + Check whether an element exists in a JSON object with key equivalent to + @a key. If the element is not found or the JSON value is not an object, + false is returned. + + @note This method always returns false when executed on a JSON type + that is not an object. + + @param[in] key key value to check its existence. + + @return true if an element with specified @a key exists. If no such + element with such key is found or the JSON value is not an object, + false is returned. + + @complexity Logarithmic in the size of the JSON object. + + @liveexample{The following code shows an example for `contains()`.,contains} + + @sa @ref find(KeyT&&) -- returns an iterator to an object element + @sa @ref contains(const json_pointer&) const -- checks the existence for a JSON pointer + + @since version 3.6.0 + */ + template < typename KeyT, typename std::enable_if < + !std::is_same::type, json_pointer>::value, int >::type = 0 > + bool contains(KeyT && key) const + { + return is_object() && m_value.object->find(std::forward(key)) != m_value.object->end(); + } + + /*! + @brief check the existence of an element in a JSON object given a JSON pointer + + Check whether the given JSON pointer @a ptr can be resolved in the current + JSON value. + + @note This method can be executed on any JSON value type. + + @param[in] ptr JSON pointer to check its existence. + + @return true if the JSON pointer can be resolved to a stored value, false + otherwise. + + @post If `j.contains(ptr)` returns true, it is safe to call `j[ptr]`. + + @throw parse_error.106 if an array index begins with '0' + @throw parse_error.109 if an array index was not a number + + @complexity Logarithmic in the size of the JSON object. + + @liveexample{The following code shows an example for `contains()`.,contains_json_pointer} + + @sa @ref contains(KeyT &&) const -- checks the existence of a key + + @since version 3.7.0 + */ + bool contains(const json_pointer& ptr) const + { + return ptr.contains(this); + } + + /// @} + + + /////////////// + // iterators // + /////////////// + + /// @name iterators + /// @{ + + /*! + @brief returns an iterator to the first element + + Returns an iterator to the first element. + + @image html range-begin-end.svg "Illustration from cppreference.com" + + @return iterator to the first element + + @complexity Constant. + + @requirement This function helps `basic_json` satisfying the + [Container](https://en.cppreference.com/w/cpp/named_req/Container) + requirements: + - The complexity is constant. + + @liveexample{The following code shows an example for `begin()`.,begin} + + @sa @ref cbegin() -- returns a const iterator to the beginning + @sa @ref end() -- returns an iterator to the end + @sa @ref cend() -- returns a const iterator to the end + + @since version 1.0.0 + */ + iterator begin() noexcept + { + iterator result(this); + result.set_begin(); + return result; + } + + /*! + @copydoc basic_json::cbegin() + */ + const_iterator begin() const noexcept + { + return cbegin(); + } + + /*! + @brief returns a const iterator to the first element + + Returns a const iterator to the first element. + + @image html range-begin-end.svg "Illustration from cppreference.com" + + @return const iterator to the first element + + @complexity Constant. + + @requirement This function helps `basic_json` satisfying the + [Container](https://en.cppreference.com/w/cpp/named_req/Container) + requirements: + - The complexity is constant. + - Has the semantics of `const_cast(*this).begin()`. + + @liveexample{The following code shows an example for `cbegin()`.,cbegin} + + @sa @ref begin() -- returns an iterator to the beginning + @sa @ref end() -- returns an iterator to the end + @sa @ref cend() -- returns a const iterator to the end + + @since version 1.0.0 + */ + const_iterator cbegin() const noexcept + { + const_iterator result(this); + result.set_begin(); + return result; + } + + /*! + @brief returns an iterator to one past the last element + + Returns an iterator to one past the last element. + + @image html range-begin-end.svg "Illustration from cppreference.com" + + @return iterator one past the last element + + @complexity Constant. + + @requirement This function helps `basic_json` satisfying the + [Container](https://en.cppreference.com/w/cpp/named_req/Container) + requirements: + - The complexity is constant. + + @liveexample{The following code shows an example for `end()`.,end} + + @sa @ref cend() -- returns a const iterator to the end + @sa @ref begin() -- returns an iterator to the beginning + @sa @ref cbegin() -- returns a const iterator to the beginning + + @since version 1.0.0 + */ + iterator end() noexcept + { + iterator result(this); + result.set_end(); + return result; + } + + /*! + @copydoc basic_json::cend() + */ + const_iterator end() const noexcept + { + return cend(); + } + + /*! + @brief returns a const iterator to one past the last element + + Returns a const iterator to one past the last element. + + @image html range-begin-end.svg "Illustration from cppreference.com" + + @return const iterator one past the last element + + @complexity Constant. + + @requirement This function helps `basic_json` satisfying the + [Container](https://en.cppreference.com/w/cpp/named_req/Container) + requirements: + - The complexity is constant. + - Has the semantics of `const_cast(*this).end()`. + + @liveexample{The following code shows an example for `cend()`.,cend} + + @sa @ref end() -- returns an iterator to the end + @sa @ref begin() -- returns an iterator to the beginning + @sa @ref cbegin() -- returns a const iterator to the beginning + + @since version 1.0.0 + */ + const_iterator cend() const noexcept + { + const_iterator result(this); + result.set_end(); + return result; + } + + /*! + @brief returns an iterator to the reverse-beginning + + Returns an iterator to the reverse-beginning; that is, the last element. + + @image html range-rbegin-rend.svg "Illustration from cppreference.com" + + @complexity Constant. + + @requirement This function helps `basic_json` satisfying the + [ReversibleContainer](https://en.cppreference.com/w/cpp/named_req/ReversibleContainer) + requirements: + - The complexity is constant. + - Has the semantics of `reverse_iterator(end())`. + + @liveexample{The following code shows an example for `rbegin()`.,rbegin} + + @sa @ref crbegin() -- returns a const reverse iterator to the beginning + @sa @ref rend() -- returns a reverse iterator to the end + @sa @ref crend() -- returns a const reverse iterator to the end + + @since version 1.0.0 + */ + reverse_iterator rbegin() noexcept + { + return reverse_iterator(end()); + } + + /*! + @copydoc basic_json::crbegin() + */ + const_reverse_iterator rbegin() const noexcept + { + return crbegin(); + } + + /*! + @brief returns an iterator to the reverse-end + + Returns an iterator to the reverse-end; that is, one before the first + element. + + @image html range-rbegin-rend.svg "Illustration from cppreference.com" + + @complexity Constant. + + @requirement This function helps `basic_json` satisfying the + [ReversibleContainer](https://en.cppreference.com/w/cpp/named_req/ReversibleContainer) + requirements: + - The complexity is constant. + - Has the semantics of `reverse_iterator(begin())`. + + @liveexample{The following code shows an example for `rend()`.,rend} + + @sa @ref crend() -- returns a const reverse iterator to the end + @sa @ref rbegin() -- returns a reverse iterator to the beginning + @sa @ref crbegin() -- returns a const reverse iterator to the beginning + + @since version 1.0.0 + */ + reverse_iterator rend() noexcept + { + return reverse_iterator(begin()); + } + + /*! + @copydoc basic_json::crend() + */ + const_reverse_iterator rend() const noexcept + { + return crend(); + } + + /*! + @brief returns a const reverse iterator to the last element + + Returns a const iterator to the reverse-beginning; that is, the last + element. + + @image html range-rbegin-rend.svg "Illustration from cppreference.com" + + @complexity Constant. + + @requirement This function helps `basic_json` satisfying the + [ReversibleContainer](https://en.cppreference.com/w/cpp/named_req/ReversibleContainer) + requirements: + - The complexity is constant. + - Has the semantics of `const_cast(*this).rbegin()`. + + @liveexample{The following code shows an example for `crbegin()`.,crbegin} + + @sa @ref rbegin() -- returns a reverse iterator to the beginning + @sa @ref rend() -- returns a reverse iterator to the end + @sa @ref crend() -- returns a const reverse iterator to the end + + @since version 1.0.0 + */ + const_reverse_iterator crbegin() const noexcept + { + return const_reverse_iterator(cend()); + } + + /*! + @brief returns a const reverse iterator to one before the first + + Returns a const reverse iterator to the reverse-end; that is, one before + the first element. + + @image html range-rbegin-rend.svg "Illustration from cppreference.com" + + @complexity Constant. + + @requirement This function helps `basic_json` satisfying the + [ReversibleContainer](https://en.cppreference.com/w/cpp/named_req/ReversibleContainer) + requirements: + - The complexity is constant. + - Has the semantics of `const_cast(*this).rend()`. + + @liveexample{The following code shows an example for `crend()`.,crend} + + @sa @ref rend() -- returns a reverse iterator to the end + @sa @ref rbegin() -- returns a reverse iterator to the beginning + @sa @ref crbegin() -- returns a const reverse iterator to the beginning + + @since version 1.0.0 + */ + const_reverse_iterator crend() const noexcept + { + return const_reverse_iterator(cbegin()); + } + + public: + /*! + @brief wrapper to access iterator member functions in range-based for + + This function allows to access @ref iterator::key() and @ref + iterator::value() during range-based for loops. In these loops, a + reference to the JSON values is returned, so there is no access to the + underlying iterator. + + For loop without iterator_wrapper: + + @code{cpp} + for (auto it = j_object.begin(); it != j_object.end(); ++it) + { + std::cout << "key: " << it.key() << ", value:" << it.value() << '\n'; + } + @endcode + + Range-based for loop without iterator proxy: + + @code{cpp} + for (auto it : j_object) + { + // "it" is of type json::reference and has no key() member + std::cout << "value: " << it << '\n'; + } + @endcode + + Range-based for loop with iterator proxy: + + @code{cpp} + for (auto it : json::iterator_wrapper(j_object)) + { + std::cout << "key: " << it.key() << ", value:" << it.value() << '\n'; + } + @endcode + + @note When iterating over an array, `key()` will return the index of the + element as string (see example). + + @param[in] ref reference to a JSON value + @return iteration proxy object wrapping @a ref with an interface to use in + range-based for loops + + @liveexample{The following code shows how the wrapper is used,iterator_wrapper} + + @exceptionsafety Strong guarantee: if an exception is thrown, there are no + changes in the JSON value. + + @complexity Constant. + + @note The name of this function is not yet final and may change in the + future. + + @deprecated This stream operator is deprecated and will be removed in + future 4.0.0 of the library. Please use @ref items() instead; + that is, replace `json::iterator_wrapper(j)` with `j.items()`. + */ + JSON_HEDLEY_DEPRECATED_FOR(3.1.0, items()) + static iteration_proxy iterator_wrapper(reference ref) noexcept + { + return ref.items(); + } + + /*! + @copydoc iterator_wrapper(reference) + */ + JSON_HEDLEY_DEPRECATED_FOR(3.1.0, items()) + static iteration_proxy iterator_wrapper(const_reference ref) noexcept + { + return ref.items(); + } + + /*! + @brief helper to access iterator member functions in range-based for + + This function allows to access @ref iterator::key() and @ref + iterator::value() during range-based for loops. In these loops, a + reference to the JSON values is returned, so there is no access to the + underlying iterator. + + For loop without `items()` function: + + @code{cpp} + for (auto it = j_object.begin(); it != j_object.end(); ++it) + { + std::cout << "key: " << it.key() << ", value:" << it.value() << '\n'; + } + @endcode + + Range-based for loop without `items()` function: + + @code{cpp} + for (auto it : j_object) + { + // "it" is of type json::reference and has no key() member + std::cout << "value: " << it << '\n'; + } + @endcode + + Range-based for loop with `items()` function: + + @code{cpp} + for (auto& el : j_object.items()) + { + std::cout << "key: " << el.key() << ", value:" << el.value() << '\n'; + } + @endcode + + The `items()` function also allows to use + [structured bindings](https://en.cppreference.com/w/cpp/language/structured_binding) + (C++17): + + @code{cpp} + for (auto& [key, val] : j_object.items()) + { + std::cout << "key: " << key << ", value:" << val << '\n'; + } + @endcode + + @note When iterating over an array, `key()` will return the index of the + element as string (see example). For primitive types (e.g., numbers), + `key()` returns an empty string. + + @warning Using `items()` on temporary objects is dangerous. Make sure the + object's lifetime exeeds the iteration. See + for more + information. + + @return iteration proxy object wrapping @a ref with an interface to use in + range-based for loops + + @liveexample{The following code shows how the function is used.,items} + + @exceptionsafety Strong guarantee: if an exception is thrown, there are no + changes in the JSON value. + + @complexity Constant. + + @since version 3.1.0, structured bindings support since 3.5.0. + */ + iteration_proxy items() noexcept + { + return iteration_proxy(*this); + } + + /*! + @copydoc items() + */ + iteration_proxy items() const noexcept + { + return iteration_proxy(*this); + } + + /// @} + + + ////////////// + // capacity // + ////////////// + + /// @name capacity + /// @{ + + /*! + @brief checks whether the container is empty. + + Checks if a JSON value has no elements (i.e. whether its @ref size is `0`). + + @return The return value depends on the different types and is + defined as follows: + Value type | return value + ----------- | ------------- + null | `true` + boolean | `false` + string | `false` + number | `false` + binary | `false` + object | result of function `object_t::empty()` + array | result of function `array_t::empty()` + + @liveexample{The following code uses `empty()` to check if a JSON + object contains any elements.,empty} + + @complexity Constant, as long as @ref array_t and @ref object_t satisfy + the Container concept; that is, their `empty()` functions have constant + complexity. + + @iterators No changes. + + @exceptionsafety No-throw guarantee: this function never throws exceptions. + + @note This function does not return whether a string stored as JSON value + is empty - it returns whether the JSON container itself is empty which is + false in the case of a string. + + @requirement This function helps `basic_json` satisfying the + [Container](https://en.cppreference.com/w/cpp/named_req/Container) + requirements: + - The complexity is constant. + - Has the semantics of `begin() == end()`. + + @sa @ref size() -- returns the number of elements + + @since version 1.0.0 + */ + bool empty() const noexcept + { + switch (m_type) + { + case value_t::null: + { + // null values are empty + return true; + } + + case value_t::array: + { + // delegate call to array_t::empty() + return m_value.array->empty(); + } + + case value_t::object: + { + // delegate call to object_t::empty() + return m_value.object->empty(); + } + + default: + { + // all other types are nonempty + return false; + } + } + } + + /*! + @brief returns the number of elements + + Returns the number of elements in a JSON value. + + @return The return value depends on the different types and is + defined as follows: + Value type | return value + ----------- | ------------- + null | `0` + boolean | `1` + string | `1` + number | `1` + binary | `1` + object | result of function object_t::size() + array | result of function array_t::size() + + @liveexample{The following code calls `size()` on the different value + types.,size} + + @complexity Constant, as long as @ref array_t and @ref object_t satisfy + the Container concept; that is, their size() functions have constant + complexity. + + @iterators No changes. + + @exceptionsafety No-throw guarantee: this function never throws exceptions. + + @note This function does not return the length of a string stored as JSON + value - it returns the number of elements in the JSON value which is 1 in + the case of a string. + + @requirement This function helps `basic_json` satisfying the + [Container](https://en.cppreference.com/w/cpp/named_req/Container) + requirements: + - The complexity is constant. + - Has the semantics of `std::distance(begin(), end())`. + + @sa @ref empty() -- checks whether the container is empty + @sa @ref max_size() -- returns the maximal number of elements + + @since version 1.0.0 + */ + size_type size() const noexcept + { + switch (m_type) + { + case value_t::null: + { + // null values are empty + return 0; + } + + case value_t::array: + { + // delegate call to array_t::size() + return m_value.array->size(); + } + + case value_t::object: + { + // delegate call to object_t::size() + return m_value.object->size(); + } + + default: + { + // all other types have size 1 + return 1; + } + } + } + + /*! + @brief returns the maximum possible number of elements + + Returns the maximum number of elements a JSON value is able to hold due to + system or library implementation limitations, i.e. `std::distance(begin(), + end())` for the JSON value. + + @return The return value depends on the different types and is + defined as follows: + Value type | return value + ----------- | ------------- + null | `0` (same as `size()`) + boolean | `1` (same as `size()`) + string | `1` (same as `size()`) + number | `1` (same as `size()`) + binary | `1` (same as `size()`) + object | result of function `object_t::max_size()` + array | result of function `array_t::max_size()` + + @liveexample{The following code calls `max_size()` on the different value + types. Note the output is implementation specific.,max_size} + + @complexity Constant, as long as @ref array_t and @ref object_t satisfy + the Container concept; that is, their `max_size()` functions have constant + complexity. + + @iterators No changes. + + @exceptionsafety No-throw guarantee: this function never throws exceptions. + + @requirement This function helps `basic_json` satisfying the + [Container](https://en.cppreference.com/w/cpp/named_req/Container) + requirements: + - The complexity is constant. + - Has the semantics of returning `b.size()` where `b` is the largest + possible JSON value. + + @sa @ref size() -- returns the number of elements + + @since version 1.0.0 + */ + size_type max_size() const noexcept + { + switch (m_type) + { + case value_t::array: + { + // delegate call to array_t::max_size() + return m_value.array->max_size(); + } + + case value_t::object: + { + // delegate call to object_t::max_size() + return m_value.object->max_size(); + } + + default: + { + // all other types have max_size() == size() + return size(); + } + } + } + + /// @} + + + /////////////// + // modifiers // + /////////////// + + /// @name modifiers + /// @{ + + /*! + @brief clears the contents + + Clears the content of a JSON value and resets it to the default value as + if @ref basic_json(value_t) would have been called with the current value + type from @ref type(): + + Value type | initial value + ----------- | ------------- + null | `null` + boolean | `false` + string | `""` + number | `0` + binary | An empty byte vector + object | `{}` + array | `[]` + + @post Has the same effect as calling + @code {.cpp} + *this = basic_json(type()); + @endcode + + @liveexample{The example below shows the effect of `clear()` to different + JSON types.,clear} + + @complexity Linear in the size of the JSON value. + + @iterators All iterators, pointers and references related to this container + are invalidated. + + @exceptionsafety No-throw guarantee: this function never throws exceptions. + + @sa @ref basic_json(value_t) -- constructor that creates an object with the + same value than calling `clear()` + + @since version 1.0.0 + */ + void clear() noexcept + { + switch (m_type) + { + case value_t::number_integer: + { + m_value.number_integer = 0; + break; + } + + case value_t::number_unsigned: + { + m_value.number_unsigned = 0; + break; + } + + case value_t::number_float: + { + m_value.number_float = 0.0; + break; + } + + case value_t::boolean: + { + m_value.boolean = false; + break; + } + + case value_t::string: + { + m_value.string->clear(); + break; + } + + case value_t::binary: + { + m_value.binary->clear(); + break; + } + + case value_t::array: + { + m_value.array->clear(); + break; + } + + case value_t::object: + { + m_value.object->clear(); + break; + } + + default: + break; + } + } + + /*! + @brief add an object to an array + + Appends the given element @a val to the end of the JSON value. If the + function is called on a JSON null value, an empty array is created before + appending @a val. + + @param[in] val the value to add to the JSON array + + @throw type_error.308 when called on a type other than JSON array or + null; example: `"cannot use push_back() with number"` + + @complexity Amortized constant. + + @liveexample{The example shows how `push_back()` and `+=` can be used to + add elements to a JSON array. Note how the `null` value was silently + converted to a JSON array.,push_back} + + @since version 1.0.0 + */ + void push_back(basic_json&& val) + { + // push_back only works for null objects or arrays + if (JSON_HEDLEY_UNLIKELY(!(is_null() || is_array()))) + { + JSON_THROW(type_error::create(308, "cannot use push_back() with " + std::string(type_name()))); + } + + // transform null object into an array + if (is_null()) + { + m_type = value_t::array; + m_value = value_t::array; + assert_invariant(); + } + + // add element to array (move semantics) + m_value.array->push_back(std::move(val)); + // if val is moved from, basic_json move constructor marks it null so we do not call the destructor + } + + /*! + @brief add an object to an array + @copydoc push_back(basic_json&&) + */ + reference operator+=(basic_json&& val) + { + push_back(std::move(val)); + return *this; + } + + /*! + @brief add an object to an array + @copydoc push_back(basic_json&&) + */ + void push_back(const basic_json& val) + { + // push_back only works for null objects or arrays + if (JSON_HEDLEY_UNLIKELY(!(is_null() || is_array()))) + { + JSON_THROW(type_error::create(308, "cannot use push_back() with " + std::string(type_name()))); + } + + // transform null object into an array + if (is_null()) + { + m_type = value_t::array; + m_value = value_t::array; + assert_invariant(); + } + + // add element to array + m_value.array->push_back(val); + } + + /*! + @brief add an object to an array + @copydoc push_back(basic_json&&) + */ + reference operator+=(const basic_json& val) + { + push_back(val); + return *this; + } + + /*! + @brief add an object to an object + + Inserts the given element @a val to the JSON object. If the function is + called on a JSON null value, an empty object is created before inserting + @a val. + + @param[in] val the value to add to the JSON object + + @throw type_error.308 when called on a type other than JSON object or + null; example: `"cannot use push_back() with number"` + + @complexity Logarithmic in the size of the container, O(log(`size()`)). + + @liveexample{The example shows how `push_back()` and `+=` can be used to + add elements to a JSON object. Note how the `null` value was silently + converted to a JSON object.,push_back__object_t__value} + + @since version 1.0.0 + */ + void push_back(const typename object_t::value_type& val) + { + // push_back only works for null objects or objects + if (JSON_HEDLEY_UNLIKELY(!(is_null() || is_object()))) + { + JSON_THROW(type_error::create(308, "cannot use push_back() with " + std::string(type_name()))); + } + + // transform null object into an object + if (is_null()) + { + m_type = value_t::object; + m_value = value_t::object; + assert_invariant(); + } + + // add element to array + m_value.object->insert(val); + } + + /*! + @brief add an object to an object + @copydoc push_back(const typename object_t::value_type&) + */ + reference operator+=(const typename object_t::value_type& val) + { + push_back(val); + return *this; + } + + /*! + @brief add an object to an object + + This function allows to use `push_back` with an initializer list. In case + + 1. the current value is an object, + 2. the initializer list @a init contains only two elements, and + 3. the first element of @a init is a string, + + @a init is converted into an object element and added using + @ref push_back(const typename object_t::value_type&). Otherwise, @a init + is converted to a JSON value and added using @ref push_back(basic_json&&). + + @param[in] init an initializer list + + @complexity Linear in the size of the initializer list @a init. + + @note This function is required to resolve an ambiguous overload error, + because pairs like `{"key", "value"}` can be both interpreted as + `object_t::value_type` or `std::initializer_list`, see + https://github.com/nlohmann/json/issues/235 for more information. + + @liveexample{The example shows how initializer lists are treated as + objects when possible.,push_back__initializer_list} + */ + void push_back(initializer_list_t init) + { + if (is_object() && init.size() == 2 && (*init.begin())->is_string()) + { + basic_json&& key = init.begin()->moved_or_copied(); + push_back(typename object_t::value_type( + std::move(key.get_ref()), (init.begin() + 1)->moved_or_copied())); + } + else + { + push_back(basic_json(init)); + } + } + + /*! + @brief add an object to an object + @copydoc push_back(initializer_list_t) + */ + reference operator+=(initializer_list_t init) + { + push_back(init); + return *this; + } + + /*! + @brief add an object to an array + + Creates a JSON value from the passed parameters @a args to the end of the + JSON value. If the function is called on a JSON null value, an empty array + is created before appending the value created from @a args. + + @param[in] args arguments to forward to a constructor of @ref basic_json + @tparam Args compatible types to create a @ref basic_json object + + @return reference to the inserted element + + @throw type_error.311 when called on a type other than JSON array or + null; example: `"cannot use emplace_back() with number"` + + @complexity Amortized constant. + + @liveexample{The example shows how `push_back()` can be used to add + elements to a JSON array. Note how the `null` value was silently converted + to a JSON array.,emplace_back} + + @since version 2.0.8, returns reference since 3.7.0 + */ + template + reference emplace_back(Args&& ... args) + { + // emplace_back only works for null objects or arrays + if (JSON_HEDLEY_UNLIKELY(!(is_null() || is_array()))) + { + JSON_THROW(type_error::create(311, "cannot use emplace_back() with " + std::string(type_name()))); + } + + // transform null object into an array + if (is_null()) + { + m_type = value_t::array; + m_value = value_t::array; + assert_invariant(); + } + + // add element to array (perfect forwarding) +#ifdef JSON_HAS_CPP_17 + return m_value.array->emplace_back(std::forward(args)...); +#else + m_value.array->emplace_back(std::forward(args)...); + return m_value.array->back(); +#endif + } + + /*! + @brief add an object to an object if key does not exist + + Inserts a new element into a JSON object constructed in-place with the + given @a args if there is no element with the key in the container. If the + function is called on a JSON null value, an empty object is created before + appending the value created from @a args. + + @param[in] args arguments to forward to a constructor of @ref basic_json + @tparam Args compatible types to create a @ref basic_json object + + @return a pair consisting of an iterator to the inserted element, or the + already-existing element if no insertion happened, and a bool + denoting whether the insertion took place. + + @throw type_error.311 when called on a type other than JSON object or + null; example: `"cannot use emplace() with number"` + + @complexity Logarithmic in the size of the container, O(log(`size()`)). + + @liveexample{The example shows how `emplace()` can be used to add elements + to a JSON object. Note how the `null` value was silently converted to a + JSON object. Further note how no value is added if there was already one + value stored with the same key.,emplace} + + @since version 2.0.8 + */ + template + std::pair emplace(Args&& ... args) + { + // emplace only works for null objects or arrays + if (JSON_HEDLEY_UNLIKELY(!(is_null() || is_object()))) + { + JSON_THROW(type_error::create(311, "cannot use emplace() with " + std::string(type_name()))); + } + + // transform null object into an object + if (is_null()) + { + m_type = value_t::object; + m_value = value_t::object; + assert_invariant(); + } + + // add element to array (perfect forwarding) + auto res = m_value.object->emplace(std::forward(args)...); + // create result iterator and set iterator to the result of emplace + auto it = begin(); + it.m_it.object_iterator = res.first; + + // return pair of iterator and boolean + return {it, res.second}; + } + + /// Helper for insertion of an iterator + /// @note: This uses std::distance to support GCC 4.8, + /// see https://github.com/nlohmann/json/pull/1257 + template + iterator insert_iterator(const_iterator pos, Args&& ... args) + { + iterator result(this); + JSON_ASSERT(m_value.array != nullptr); + + auto insert_pos = std::distance(m_value.array->begin(), pos.m_it.array_iterator); + m_value.array->insert(pos.m_it.array_iterator, std::forward(args)...); + result.m_it.array_iterator = m_value.array->begin() + insert_pos; + + // This could have been written as: + // result.m_it.array_iterator = m_value.array->insert(pos.m_it.array_iterator, cnt, val); + // but the return value of insert is missing in GCC 4.8, so it is written this way instead. + + return result; + } + + /*! + @brief inserts element + + Inserts element @a val before iterator @a pos. + + @param[in] pos iterator before which the content will be inserted; may be + the end() iterator + @param[in] val element to insert + @return iterator pointing to the inserted @a val. + + @throw type_error.309 if called on JSON values other than arrays; + example: `"cannot use insert() with string"` + @throw invalid_iterator.202 if @a pos is not an iterator of *this; + example: `"iterator does not fit current value"` + + @complexity Constant plus linear in the distance between @a pos and end of + the container. + + @liveexample{The example shows how `insert()` is used.,insert} + + @since version 1.0.0 + */ + iterator insert(const_iterator pos, const basic_json& val) + { + // insert only works for arrays + if (JSON_HEDLEY_LIKELY(is_array())) + { + // check if iterator pos fits to this JSON value + if (JSON_HEDLEY_UNLIKELY(pos.m_object != this)) + { + JSON_THROW(invalid_iterator::create(202, "iterator does not fit current value")); + } + + // insert to array and return iterator + return insert_iterator(pos, val); + } + + JSON_THROW(type_error::create(309, "cannot use insert() with " + std::string(type_name()))); + } + + /*! + @brief inserts element + @copydoc insert(const_iterator, const basic_json&) + */ + iterator insert(const_iterator pos, basic_json&& val) + { + return insert(pos, val); + } + + /*! + @brief inserts elements + + Inserts @a cnt copies of @a val before iterator @a pos. + + @param[in] pos iterator before which the content will be inserted; may be + the end() iterator + @param[in] cnt number of copies of @a val to insert + @param[in] val element to insert + @return iterator pointing to the first element inserted, or @a pos if + `cnt==0` + + @throw type_error.309 if called on JSON values other than arrays; example: + `"cannot use insert() with string"` + @throw invalid_iterator.202 if @a pos is not an iterator of *this; + example: `"iterator does not fit current value"` + + @complexity Linear in @a cnt plus linear in the distance between @a pos + and end of the container. + + @liveexample{The example shows how `insert()` is used.,insert__count} + + @since version 1.0.0 + */ + iterator insert(const_iterator pos, size_type cnt, const basic_json& val) + { + // insert only works for arrays + if (JSON_HEDLEY_LIKELY(is_array())) + { + // check if iterator pos fits to this JSON value + if (JSON_HEDLEY_UNLIKELY(pos.m_object != this)) + { + JSON_THROW(invalid_iterator::create(202, "iterator does not fit current value")); + } + + // insert to array and return iterator + return insert_iterator(pos, cnt, val); + } + + JSON_THROW(type_error::create(309, "cannot use insert() with " + std::string(type_name()))); + } + + /*! + @brief inserts elements + + Inserts elements from range `[first, last)` before iterator @a pos. + + @param[in] pos iterator before which the content will be inserted; may be + the end() iterator + @param[in] first begin of the range of elements to insert + @param[in] last end of the range of elements to insert + + @throw type_error.309 if called on JSON values other than arrays; example: + `"cannot use insert() with string"` + @throw invalid_iterator.202 if @a pos is not an iterator of *this; + example: `"iterator does not fit current value"` + @throw invalid_iterator.210 if @a first and @a last do not belong to the + same JSON value; example: `"iterators do not fit"` + @throw invalid_iterator.211 if @a first or @a last are iterators into + container for which insert is called; example: `"passed iterators may not + belong to container"` + + @return iterator pointing to the first element inserted, or @a pos if + `first==last` + + @complexity Linear in `std::distance(first, last)` plus linear in the + distance between @a pos and end of the container. + + @liveexample{The example shows how `insert()` is used.,insert__range} + + @since version 1.0.0 + */ + iterator insert(const_iterator pos, const_iterator first, const_iterator last) + { + // insert only works for arrays + if (JSON_HEDLEY_UNLIKELY(!is_array())) + { + JSON_THROW(type_error::create(309, "cannot use insert() with " + std::string(type_name()))); + } + + // check if iterator pos fits to this JSON value + if (JSON_HEDLEY_UNLIKELY(pos.m_object != this)) + { + JSON_THROW(invalid_iterator::create(202, "iterator does not fit current value")); + } + + // check if range iterators belong to the same JSON object + if (JSON_HEDLEY_UNLIKELY(first.m_object != last.m_object)) + { + JSON_THROW(invalid_iterator::create(210, "iterators do not fit")); + } + + if (JSON_HEDLEY_UNLIKELY(first.m_object == this)) + { + JSON_THROW(invalid_iterator::create(211, "passed iterators may not belong to container")); + } + + // insert to array and return iterator + return insert_iterator(pos, first.m_it.array_iterator, last.m_it.array_iterator); + } + + /*! + @brief inserts elements + + Inserts elements from initializer list @a ilist before iterator @a pos. + + @param[in] pos iterator before which the content will be inserted; may be + the end() iterator + @param[in] ilist initializer list to insert the values from + + @throw type_error.309 if called on JSON values other than arrays; example: + `"cannot use insert() with string"` + @throw invalid_iterator.202 if @a pos is not an iterator of *this; + example: `"iterator does not fit current value"` + + @return iterator pointing to the first element inserted, or @a pos if + `ilist` is empty + + @complexity Linear in `ilist.size()` plus linear in the distance between + @a pos and end of the container. + + @liveexample{The example shows how `insert()` is used.,insert__ilist} + + @since version 1.0.0 + */ + iterator insert(const_iterator pos, initializer_list_t ilist) + { + // insert only works for arrays + if (JSON_HEDLEY_UNLIKELY(!is_array())) + { + JSON_THROW(type_error::create(309, "cannot use insert() with " + std::string(type_name()))); + } + + // check if iterator pos fits to this JSON value + if (JSON_HEDLEY_UNLIKELY(pos.m_object != this)) + { + JSON_THROW(invalid_iterator::create(202, "iterator does not fit current value")); + } + + // insert to array and return iterator + return insert_iterator(pos, ilist.begin(), ilist.end()); + } + + /*! + @brief inserts elements + + Inserts elements from range `[first, last)`. + + @param[in] first begin of the range of elements to insert + @param[in] last end of the range of elements to insert + + @throw type_error.309 if called on JSON values other than objects; example: + `"cannot use insert() with string"` + @throw invalid_iterator.202 if iterator @a first or @a last does does not + point to an object; example: `"iterators first and last must point to + objects"` + @throw invalid_iterator.210 if @a first and @a last do not belong to the + same JSON value; example: `"iterators do not fit"` + + @complexity Logarithmic: `O(N*log(size() + N))`, where `N` is the number + of elements to insert. + + @liveexample{The example shows how `insert()` is used.,insert__range_object} + + @since version 3.0.0 + */ + void insert(const_iterator first, const_iterator last) + { + // insert only works for objects + if (JSON_HEDLEY_UNLIKELY(!is_object())) + { + JSON_THROW(type_error::create(309, "cannot use insert() with " + std::string(type_name()))); + } + + // check if range iterators belong to the same JSON object + if (JSON_HEDLEY_UNLIKELY(first.m_object != last.m_object)) + { + JSON_THROW(invalid_iterator::create(210, "iterators do not fit")); + } + + // passed iterators must belong to objects + if (JSON_HEDLEY_UNLIKELY(!first.m_object->is_object())) + { + JSON_THROW(invalid_iterator::create(202, "iterators first and last must point to objects")); + } + + m_value.object->insert(first.m_it.object_iterator, last.m_it.object_iterator); + } + + /*! + @brief updates a JSON object from another object, overwriting existing keys + + Inserts all values from JSON object @a j and overwrites existing keys. + + @param[in] j JSON object to read values from + + @throw type_error.312 if called on JSON values other than objects; example: + `"cannot use update() with string"` + + @complexity O(N*log(size() + N)), where N is the number of elements to + insert. + + @liveexample{The example shows how `update()` is used.,update} + + @sa https://docs.python.org/3.6/library/stdtypes.html#dict.update + + @since version 3.0.0 + */ + void update(const_reference j) + { + // implicitly convert null value to an empty object + if (is_null()) + { + m_type = value_t::object; + m_value.object = create(); + assert_invariant(); + } + + if (JSON_HEDLEY_UNLIKELY(!is_object())) + { + JSON_THROW(type_error::create(312, "cannot use update() with " + std::string(type_name()))); + } + if (JSON_HEDLEY_UNLIKELY(!j.is_object())) + { + JSON_THROW(type_error::create(312, "cannot use update() with " + std::string(j.type_name()))); + } + + for (auto it = j.cbegin(); it != j.cend(); ++it) + { + m_value.object->operator[](it.key()) = it.value(); + } + } + + /*! + @brief updates a JSON object from another object, overwriting existing keys + + Inserts all values from from range `[first, last)` and overwrites existing + keys. + + @param[in] first begin of the range of elements to insert + @param[in] last end of the range of elements to insert + + @throw type_error.312 if called on JSON values other than objects; example: + `"cannot use update() with string"` + @throw invalid_iterator.202 if iterator @a first or @a last does does not + point to an object; example: `"iterators first and last must point to + objects"` + @throw invalid_iterator.210 if @a first and @a last do not belong to the + same JSON value; example: `"iterators do not fit"` + + @complexity O(N*log(size() + N)), where N is the number of elements to + insert. + + @liveexample{The example shows how `update()` is used__range.,update} + + @sa https://docs.python.org/3.6/library/stdtypes.html#dict.update + + @since version 3.0.0 + */ + void update(const_iterator first, const_iterator last) + { + // implicitly convert null value to an empty object + if (is_null()) + { + m_type = value_t::object; + m_value.object = create(); + assert_invariant(); + } + + if (JSON_HEDLEY_UNLIKELY(!is_object())) + { + JSON_THROW(type_error::create(312, "cannot use update() with " + std::string(type_name()))); + } + + // check if range iterators belong to the same JSON object + if (JSON_HEDLEY_UNLIKELY(first.m_object != last.m_object)) + { + JSON_THROW(invalid_iterator::create(210, "iterators do not fit")); + } + + // passed iterators must belong to objects + if (JSON_HEDLEY_UNLIKELY(!first.m_object->is_object() + || !last.m_object->is_object())) + { + JSON_THROW(invalid_iterator::create(202, "iterators first and last must point to objects")); + } + + for (auto it = first; it != last; ++it) + { + m_value.object->operator[](it.key()) = it.value(); + } + } + + /*! + @brief exchanges the values + + Exchanges the contents of the JSON value with those of @a other. Does not + invoke any move, copy, or swap operations on individual elements. All + iterators and references remain valid. The past-the-end iterator is + invalidated. + + @param[in,out] other JSON value to exchange the contents with + + @complexity Constant. + + @liveexample{The example below shows how JSON values can be swapped with + `swap()`.,swap__reference} + + @since version 1.0.0 + */ + void swap(reference other) noexcept ( + std::is_nothrow_move_constructible::value&& + std::is_nothrow_move_assignable::value&& + std::is_nothrow_move_constructible::value&& + std::is_nothrow_move_assignable::value + ) + { + std::swap(m_type, other.m_type); + std::swap(m_value, other.m_value); + assert_invariant(); + } + + /*! + @brief exchanges the values + + Exchanges the contents of the JSON value from @a left with those of @a right. Does not + invoke any move, copy, or swap operations on individual elements. All + iterators and references remain valid. The past-the-end iterator is + invalidated. implemented as a friend function callable via ADL. + + @param[in,out] left JSON value to exchange the contents with + @param[in,out] right JSON value to exchange the contents with + + @complexity Constant. + + @liveexample{The example below shows how JSON values can be swapped with + `swap()`.,swap__reference} + + @since version 1.0.0 + */ + friend void swap(reference left, reference right) noexcept ( + std::is_nothrow_move_constructible::value&& + std::is_nothrow_move_assignable::value&& + std::is_nothrow_move_constructible::value&& + std::is_nothrow_move_assignable::value + ) + { + left.swap(right); + } + + /*! + @brief exchanges the values + + Exchanges the contents of a JSON array with those of @a other. Does not + invoke any move, copy, or swap operations on individual elements. All + iterators and references remain valid. The past-the-end iterator is + invalidated. + + @param[in,out] other array to exchange the contents with + + @throw type_error.310 when JSON value is not an array; example: `"cannot + use swap() with string"` + + @complexity Constant. + + @liveexample{The example below shows how arrays can be swapped with + `swap()`.,swap__array_t} + + @since version 1.0.0 + */ + void swap(array_t& other) + { + // swap only works for arrays + if (JSON_HEDLEY_LIKELY(is_array())) + { + std::swap(*(m_value.array), other); + } + else + { + JSON_THROW(type_error::create(310, "cannot use swap() with " + std::string(type_name()))); + } + } + + /*! + @brief exchanges the values + + Exchanges the contents of a JSON object with those of @a other. Does not + invoke any move, copy, or swap operations on individual elements. All + iterators and references remain valid. The past-the-end iterator is + invalidated. + + @param[in,out] other object to exchange the contents with + + @throw type_error.310 when JSON value is not an object; example: + `"cannot use swap() with string"` + + @complexity Constant. + + @liveexample{The example below shows how objects can be swapped with + `swap()`.,swap__object_t} + + @since version 1.0.0 + */ + void swap(object_t& other) + { + // swap only works for objects + if (JSON_HEDLEY_LIKELY(is_object())) + { + std::swap(*(m_value.object), other); + } + else + { + JSON_THROW(type_error::create(310, "cannot use swap() with " + std::string(type_name()))); + } + } + + /*! + @brief exchanges the values + + Exchanges the contents of a JSON string with those of @a other. Does not + invoke any move, copy, or swap operations on individual elements. All + iterators and references remain valid. The past-the-end iterator is + invalidated. + + @param[in,out] other string to exchange the contents with + + @throw type_error.310 when JSON value is not a string; example: `"cannot + use swap() with boolean"` + + @complexity Constant. + + @liveexample{The example below shows how strings can be swapped with + `swap()`.,swap__string_t} + + @since version 1.0.0 + */ + void swap(string_t& other) + { + // swap only works for strings + if (JSON_HEDLEY_LIKELY(is_string())) + { + std::swap(*(m_value.string), other); + } + else + { + JSON_THROW(type_error::create(310, "cannot use swap() with " + std::string(type_name()))); + } + } + + /*! + @brief exchanges the values + + Exchanges the contents of a JSON string with those of @a other. Does not + invoke any move, copy, or swap operations on individual elements. All + iterators and references remain valid. The past-the-end iterator is + invalidated. + + @param[in,out] other binary to exchange the contents with + + @throw type_error.310 when JSON value is not a string; example: `"cannot + use swap() with boolean"` + + @complexity Constant. + + @liveexample{The example below shows how strings can be swapped with + `swap()`.,swap__binary_t} + + @since version 3.8.0 + */ + void swap(binary_t& other) + { + // swap only works for strings + if (JSON_HEDLEY_LIKELY(is_binary())) + { + std::swap(*(m_value.binary), other); + } + else + { + JSON_THROW(type_error::create(310, "cannot use swap() with " + std::string(type_name()))); + } + } + + /// @copydoc swap(binary_t) + void swap(typename binary_t::container_type& other) + { + // swap only works for strings + if (JSON_HEDLEY_LIKELY(is_binary())) + { + std::swap(*(m_value.binary), other); + } + else + { + JSON_THROW(type_error::create(310, "cannot use swap() with " + std::string(type_name()))); + } + } + + /// @} + + public: + ////////////////////////////////////////// + // lexicographical comparison operators // + ////////////////////////////////////////// + + /// @name lexicographical comparison operators + /// @{ + + /*! + @brief comparison: equal + + Compares two JSON values for equality according to the following rules: + - Two JSON values are equal if (1) they are from the same type and (2) + their stored values are the same according to their respective + `operator==`. + - Integer and floating-point numbers are automatically converted before + comparison. Note that two NaN values are always treated as unequal. + - Two JSON null values are equal. + + @note Floating-point inside JSON values numbers are compared with + `json::number_float_t::operator==` which is `double::operator==` by + default. To compare floating-point while respecting an epsilon, an alternative + [comparison function](https://github.com/mariokonrad/marnav/blob/master/include/marnav/math/floatingpoint.hpp#L34-#L39) + could be used, for instance + @code {.cpp} + template::value, T>::type> + inline bool is_same(T a, T b, T epsilon = std::numeric_limits::epsilon()) noexcept + { + return std::abs(a - b) <= epsilon; + } + @endcode + Or you can self-defined operator equal function like this: + @code {.cpp} + bool my_equal(const_reference lhs, const_reference rhs) { + const auto lhs_type lhs.type(); + const auto rhs_type rhs.type(); + if (lhs_type == rhs_type) { + switch(lhs_type) + // self_defined case + case value_t::number_float: + return std::abs(lhs - rhs) <= std::numeric_limits::epsilon(); + // other cases remain the same with the original + ... + } + ... + } + @endcode + + @note NaN values never compare equal to themselves or to other NaN values. + + @param[in] lhs first JSON value to consider + @param[in] rhs second JSON value to consider + @return whether the values @a lhs and @a rhs are equal + + @exceptionsafety No-throw guarantee: this function never throws exceptions. + + @complexity Linear. + + @liveexample{The example demonstrates comparing several JSON + types.,operator__equal} + + @since version 1.0.0 + */ + friend bool operator==(const_reference lhs, const_reference rhs) noexcept + { + const auto lhs_type = lhs.type(); + const auto rhs_type = rhs.type(); + + if (lhs_type == rhs_type) + { + switch (lhs_type) + { + case value_t::array: + return *lhs.m_value.array == *rhs.m_value.array; + + case value_t::object: + return *lhs.m_value.object == *rhs.m_value.object; + + case value_t::null: + return true; + + case value_t::string: + return *lhs.m_value.string == *rhs.m_value.string; + + case value_t::boolean: + return lhs.m_value.boolean == rhs.m_value.boolean; + + case value_t::number_integer: + return lhs.m_value.number_integer == rhs.m_value.number_integer; + + case value_t::number_unsigned: + return lhs.m_value.number_unsigned == rhs.m_value.number_unsigned; + + case value_t::number_float: + return lhs.m_value.number_float == rhs.m_value.number_float; + + case value_t::binary: + return *lhs.m_value.binary == *rhs.m_value.binary; + + default: + return false; + } + } + else if (lhs_type == value_t::number_integer && rhs_type == value_t::number_float) + { + return static_cast(lhs.m_value.number_integer) == rhs.m_value.number_float; + } + else if (lhs_type == value_t::number_float && rhs_type == value_t::number_integer) + { + return lhs.m_value.number_float == static_cast(rhs.m_value.number_integer); + } + else if (lhs_type == value_t::number_unsigned && rhs_type == value_t::number_float) + { + return static_cast(lhs.m_value.number_unsigned) == rhs.m_value.number_float; + } + else if (lhs_type == value_t::number_float && rhs_type == value_t::number_unsigned) + { + return lhs.m_value.number_float == static_cast(rhs.m_value.number_unsigned); + } + else if (lhs_type == value_t::number_unsigned && rhs_type == value_t::number_integer) + { + return static_cast(lhs.m_value.number_unsigned) == rhs.m_value.number_integer; + } + else if (lhs_type == value_t::number_integer && rhs_type == value_t::number_unsigned) + { + return lhs.m_value.number_integer == static_cast(rhs.m_value.number_unsigned); + } + + return false; + } + + /*! + @brief comparison: equal + @copydoc operator==(const_reference, const_reference) + */ + template::value, int>::type = 0> + friend bool operator==(const_reference lhs, const ScalarType rhs) noexcept + { + return lhs == basic_json(rhs); + } + + /*! + @brief comparison: equal + @copydoc operator==(const_reference, const_reference) + */ + template::value, int>::type = 0> + friend bool operator==(const ScalarType lhs, const_reference rhs) noexcept + { + return basic_json(lhs) == rhs; + } + + /*! + @brief comparison: not equal + + Compares two JSON values for inequality by calculating `not (lhs == rhs)`. + + @param[in] lhs first JSON value to consider + @param[in] rhs second JSON value to consider + @return whether the values @a lhs and @a rhs are not equal + + @complexity Linear. + + @exceptionsafety No-throw guarantee: this function never throws exceptions. + + @liveexample{The example demonstrates comparing several JSON + types.,operator__notequal} + + @since version 1.0.0 + */ + friend bool operator!=(const_reference lhs, const_reference rhs) noexcept + { + return !(lhs == rhs); + } + + /*! + @brief comparison: not equal + @copydoc operator!=(const_reference, const_reference) + */ + template::value, int>::type = 0> + friend bool operator!=(const_reference lhs, const ScalarType rhs) noexcept + { + return lhs != basic_json(rhs); + } + + /*! + @brief comparison: not equal + @copydoc operator!=(const_reference, const_reference) + */ + template::value, int>::type = 0> + friend bool operator!=(const ScalarType lhs, const_reference rhs) noexcept + { + return basic_json(lhs) != rhs; + } + + /*! + @brief comparison: less than + + Compares whether one JSON value @a lhs is less than another JSON value @a + rhs according to the following rules: + - If @a lhs and @a rhs have the same type, the values are compared using + the default `<` operator. + - Integer and floating-point numbers are automatically converted before + comparison + - In case @a lhs and @a rhs have different types, the values are ignored + and the order of the types is considered, see + @ref operator<(const value_t, const value_t). + + @param[in] lhs first JSON value to consider + @param[in] rhs second JSON value to consider + @return whether @a lhs is less than @a rhs + + @complexity Linear. + + @exceptionsafety No-throw guarantee: this function never throws exceptions. + + @liveexample{The example demonstrates comparing several JSON + types.,operator__less} + + @since version 1.0.0 + */ + friend bool operator<(const_reference lhs, const_reference rhs) noexcept + { + const auto lhs_type = lhs.type(); + const auto rhs_type = rhs.type(); + + if (lhs_type == rhs_type) + { + switch (lhs_type) + { + case value_t::array: + // note parentheses are necessary, see + // https://github.com/nlohmann/json/issues/1530 + return (*lhs.m_value.array) < (*rhs.m_value.array); + + case value_t::object: + return (*lhs.m_value.object) < (*rhs.m_value.object); + + case value_t::null: + return false; + + case value_t::string: + return (*lhs.m_value.string) < (*rhs.m_value.string); + + case value_t::boolean: + return (lhs.m_value.boolean) < (rhs.m_value.boolean); + + case value_t::number_integer: + return (lhs.m_value.number_integer) < (rhs.m_value.number_integer); + + case value_t::number_unsigned: + return (lhs.m_value.number_unsigned) < (rhs.m_value.number_unsigned); + + case value_t::number_float: + return (lhs.m_value.number_float) < (rhs.m_value.number_float); + + case value_t::binary: + return (*lhs.m_value.binary) < (*rhs.m_value.binary); + + default: + return false; + } + } + else if (lhs_type == value_t::number_integer && rhs_type == value_t::number_float) + { + return static_cast(lhs.m_value.number_integer) < rhs.m_value.number_float; + } + else if (lhs_type == value_t::number_float && rhs_type == value_t::number_integer) + { + return lhs.m_value.number_float < static_cast(rhs.m_value.number_integer); + } + else if (lhs_type == value_t::number_unsigned && rhs_type == value_t::number_float) + { + return static_cast(lhs.m_value.number_unsigned) < rhs.m_value.number_float; + } + else if (lhs_type == value_t::number_float && rhs_type == value_t::number_unsigned) + { + return lhs.m_value.number_float < static_cast(rhs.m_value.number_unsigned); + } + else if (lhs_type == value_t::number_integer && rhs_type == value_t::number_unsigned) + { + return lhs.m_value.number_integer < static_cast(rhs.m_value.number_unsigned); + } + else if (lhs_type == value_t::number_unsigned && rhs_type == value_t::number_integer) + { + return static_cast(lhs.m_value.number_unsigned) < rhs.m_value.number_integer; + } + + // We only reach this line if we cannot compare values. In that case, + // we compare types. Note we have to call the operator explicitly, + // because MSVC has problems otherwise. + return operator<(lhs_type, rhs_type); + } + + /*! + @brief comparison: less than + @copydoc operator<(const_reference, const_reference) + */ + template::value, int>::type = 0> + friend bool operator<(const_reference lhs, const ScalarType rhs) noexcept + { + return lhs < basic_json(rhs); + } + + /*! + @brief comparison: less than + @copydoc operator<(const_reference, const_reference) + */ + template::value, int>::type = 0> + friend bool operator<(const ScalarType lhs, const_reference rhs) noexcept + { + return basic_json(lhs) < rhs; + } + + /*! + @brief comparison: less than or equal + + Compares whether one JSON value @a lhs is less than or equal to another + JSON value by calculating `not (rhs < lhs)`. + + @param[in] lhs first JSON value to consider + @param[in] rhs second JSON value to consider + @return whether @a lhs is less than or equal to @a rhs + + @complexity Linear. + + @exceptionsafety No-throw guarantee: this function never throws exceptions. + + @liveexample{The example demonstrates comparing several JSON + types.,operator__greater} + + @since version 1.0.0 + */ + friend bool operator<=(const_reference lhs, const_reference rhs) noexcept + { + return !(rhs < lhs); + } + + /*! + @brief comparison: less than or equal + @copydoc operator<=(const_reference, const_reference) + */ + template::value, int>::type = 0> + friend bool operator<=(const_reference lhs, const ScalarType rhs) noexcept + { + return lhs <= basic_json(rhs); + } + + /*! + @brief comparison: less than or equal + @copydoc operator<=(const_reference, const_reference) + */ + template::value, int>::type = 0> + friend bool operator<=(const ScalarType lhs, const_reference rhs) noexcept + { + return basic_json(lhs) <= rhs; + } + + /*! + @brief comparison: greater than + + Compares whether one JSON value @a lhs is greater than another + JSON value by calculating `not (lhs <= rhs)`. + + @param[in] lhs first JSON value to consider + @param[in] rhs second JSON value to consider + @return whether @a lhs is greater than to @a rhs + + @complexity Linear. + + @exceptionsafety No-throw guarantee: this function never throws exceptions. + + @liveexample{The example demonstrates comparing several JSON + types.,operator__lessequal} + + @since version 1.0.0 + */ + friend bool operator>(const_reference lhs, const_reference rhs) noexcept + { + return !(lhs <= rhs); + } + + /*! + @brief comparison: greater than + @copydoc operator>(const_reference, const_reference) + */ + template::value, int>::type = 0> + friend bool operator>(const_reference lhs, const ScalarType rhs) noexcept + { + return lhs > basic_json(rhs); + } + + /*! + @brief comparison: greater than + @copydoc operator>(const_reference, const_reference) + */ + template::value, int>::type = 0> + friend bool operator>(const ScalarType lhs, const_reference rhs) noexcept + { + return basic_json(lhs) > rhs; + } + + /*! + @brief comparison: greater than or equal + + Compares whether one JSON value @a lhs is greater than or equal to another + JSON value by calculating `not (lhs < rhs)`. + + @param[in] lhs first JSON value to consider + @param[in] rhs second JSON value to consider + @return whether @a lhs is greater than or equal to @a rhs + + @complexity Linear. + + @exceptionsafety No-throw guarantee: this function never throws exceptions. + + @liveexample{The example demonstrates comparing several JSON + types.,operator__greaterequal} + + @since version 1.0.0 + */ + friend bool operator>=(const_reference lhs, const_reference rhs) noexcept + { + return !(lhs < rhs); + } + + /*! + @brief comparison: greater than or equal + @copydoc operator>=(const_reference, const_reference) + */ + template::value, int>::type = 0> + friend bool operator>=(const_reference lhs, const ScalarType rhs) noexcept + { + return lhs >= basic_json(rhs); + } + + /*! + @brief comparison: greater than or equal + @copydoc operator>=(const_reference, const_reference) + */ + template::value, int>::type = 0> + friend bool operator>=(const ScalarType lhs, const_reference rhs) noexcept + { + return basic_json(lhs) >= rhs; + } + + /// @} + + /////////////////// + // serialization // + /////////////////// + + /// @name serialization + /// @{ + + /*! + @brief serialize to stream + + Serialize the given JSON value @a j to the output stream @a o. The JSON + value will be serialized using the @ref dump member function. + + - The indentation of the output can be controlled with the member variable + `width` of the output stream @a o. For instance, using the manipulator + `std::setw(4)` on @a o sets the indentation level to `4` and the + serialization result is the same as calling `dump(4)`. + + - The indentation character can be controlled with the member variable + `fill` of the output stream @a o. For instance, the manipulator + `std::setfill('\\t')` sets indentation to use a tab character rather than + the default space character. + + @param[in,out] o stream to serialize to + @param[in] j JSON value to serialize + + @return the stream @a o + + @throw type_error.316 if a string stored inside the JSON value is not + UTF-8 encoded + + @complexity Linear. + + @liveexample{The example below shows the serialization with different + parameters to `width` to adjust the indentation level.,operator_serialize} + + @since version 1.0.0; indentation character added in version 3.0.0 + */ + friend std::ostream& operator<<(std::ostream& o, const basic_json& j) + { + // read width member and use it as indentation parameter if nonzero + const bool pretty_print = o.width() > 0; + const auto indentation = pretty_print ? o.width() : 0; + + // reset width to 0 for subsequent calls to this stream + o.width(0); + + // do the actual serialization + serializer s(detail::output_adapter(o), o.fill()); + s.dump(j, pretty_print, false, static_cast(indentation)); + return o; + } + + /*! + @brief serialize to stream + @deprecated This stream operator is deprecated and will be removed in + future 4.0.0 of the library. Please use + @ref operator<<(std::ostream&, const basic_json&) + instead; that is, replace calls like `j >> o;` with `o << j;`. + @since version 1.0.0; deprecated since version 3.0.0 + */ + JSON_HEDLEY_DEPRECATED_FOR(3.0.0, operator<<(std::ostream&, const basic_json&)) + friend std::ostream& operator>>(const basic_json& j, std::ostream& o) + { + return o << j; + } + + /// @} + + + ///////////////////// + // deserialization // + ///////////////////// + + /// @name deserialization + /// @{ + + /*! + @brief deserialize from a compatible input + + @tparam InputType A compatible input, for instance + - an std::istream object + - a FILE pointer + - a C-style array of characters + - a pointer to a null-terminated string of single byte characters + - an object obj for which begin(obj) and end(obj) produces a valid pair of + iterators. + + @param[in] i input to read from + @param[in] cb a parser callback function of type @ref parser_callback_t + which is used to control the deserialization by filtering unwanted values + (optional) + @param[in] allow_exceptions whether to throw exceptions in case of a + parse error (optional, true by default) + @param[in] ignore_comments whether comments should be ignored and treated + like whitespace (true) or yield a parse error (true); (optional, false by + default) + + @return deserialized JSON value; in case of a parse error and + @a allow_exceptions set to `false`, the return value will be + value_t::discarded. + + @throw parse_error.101 if a parse error occurs; example: `""unexpected end + of input; expected string literal""` + @throw parse_error.102 if to_unicode fails or surrogate error + @throw parse_error.103 if to_unicode fails + + @complexity Linear in the length of the input. The parser is a predictive + LL(1) parser. The complexity can be higher if the parser callback function + @a cb or reading from the input @a i has a super-linear complexity. + + @note A UTF-8 byte order mark is silently ignored. + + @liveexample{The example below demonstrates the `parse()` function reading + from an array.,parse__array__parser_callback_t} + + @liveexample{The example below demonstrates the `parse()` function with + and without callback function.,parse__string__parser_callback_t} + + @liveexample{The example below demonstrates the `parse()` function with + and without callback function.,parse__istream__parser_callback_t} + + @liveexample{The example below demonstrates the `parse()` function reading + from a contiguous container.,parse__contiguouscontainer__parser_callback_t} + + @since version 2.0.3 (contiguous containers); version 3.9.0 allowed to + ignore comments. + */ + template + JSON_HEDLEY_WARN_UNUSED_RESULT + static basic_json parse(InputType&& i, + const parser_callback_t cb = nullptr, + const bool allow_exceptions = true, + const bool ignore_comments = false) + { + basic_json result; + parser(detail::input_adapter(std::forward(i)), cb, allow_exceptions, ignore_comments).parse(true, result); + return result; + } + + /*! + @brief deserialize from a pair of character iterators + + The value_type of the iterator must be a integral type with size of 1, 2 or + 4 bytes, which will be interpreted respectively as UTF-8, UTF-16 and UTF-32. + + @param[in] first iterator to start of character range + @param[in] last iterator to end of character range + @param[in] cb a parser callback function of type @ref parser_callback_t + which is used to control the deserialization by filtering unwanted values + (optional) + @param[in] allow_exceptions whether to throw exceptions in case of a + parse error (optional, true by default) + @param[in] ignore_comments whether comments should be ignored and treated + like whitespace (true) or yield a parse error (true); (optional, false by + default) + + @return deserialized JSON value; in case of a parse error and + @a allow_exceptions set to `false`, the return value will be + value_t::discarded. + + @throw parse_error.101 if a parse error occurs; example: `""unexpected end + of input; expected string literal""` + @throw parse_error.102 if to_unicode fails or surrogate error + @throw parse_error.103 if to_unicode fails + */ + template + JSON_HEDLEY_WARN_UNUSED_RESULT + static basic_json parse(IteratorType first, + IteratorType last, + const parser_callback_t cb = nullptr, + const bool allow_exceptions = true, + const bool ignore_comments = false) + { + basic_json result; + parser(detail::input_adapter(std::move(first), std::move(last)), cb, allow_exceptions, ignore_comments).parse(true, result); + return result; + } + + JSON_HEDLEY_WARN_UNUSED_RESULT + JSON_HEDLEY_DEPRECATED_FOR(3.8.0, parse(ptr, ptr + len)) + static basic_json parse(detail::span_input_adapter&& i, + const parser_callback_t cb = nullptr, + const bool allow_exceptions = true, + const bool ignore_comments = false) + { + basic_json result; + parser(i.get(), cb, allow_exceptions, ignore_comments).parse(true, result); + return result; + } + + /*! + @brief check if the input is valid JSON + + Unlike the @ref parse(InputType&&, const parser_callback_t,const bool) + function, this function neither throws an exception in case of invalid JSON + input (i.e., a parse error) nor creates diagnostic information. + + @tparam InputType A compatible input, for instance + - an std::istream object + - a FILE pointer + - a C-style array of characters + - a pointer to a null-terminated string of single byte characters + - an object obj for which begin(obj) and end(obj) produces a valid pair of + iterators. + + @param[in] i input to read from + @param[in] ignore_comments whether comments should be ignored and treated + like whitespace (true) or yield a parse error (true); (optional, false by + default) + + @return Whether the input read from @a i is valid JSON. + + @complexity Linear in the length of the input. The parser is a predictive + LL(1) parser. + + @note A UTF-8 byte order mark is silently ignored. + + @liveexample{The example below demonstrates the `accept()` function reading + from a string.,accept__string} + */ + template + static bool accept(InputType&& i, + const bool ignore_comments = false) + { + return parser(detail::input_adapter(std::forward(i)), nullptr, false, ignore_comments).accept(true); + } + + template + static bool accept(IteratorType first, IteratorType last, + const bool ignore_comments = false) + { + return parser(detail::input_adapter(std::move(first), std::move(last)), nullptr, false, ignore_comments).accept(true); + } + + JSON_HEDLEY_WARN_UNUSED_RESULT + JSON_HEDLEY_DEPRECATED_FOR(3.8.0, accept(ptr, ptr + len)) + static bool accept(detail::span_input_adapter&& i, + const bool ignore_comments = false) + { + return parser(i.get(), nullptr, false, ignore_comments).accept(true); + } + + /*! + @brief generate SAX events + + The SAX event lister must follow the interface of @ref json_sax. + + This function reads from a compatible input. Examples are: + - an std::istream object + - a FILE pointer + - a C-style array of characters + - a pointer to a null-terminated string of single byte characters + - an object obj for which begin(obj) and end(obj) produces a valid pair of + iterators. + + @param[in] i input to read from + @param[in,out] sax SAX event listener + @param[in] format the format to parse (JSON, CBOR, MessagePack, or UBJSON) + @param[in] strict whether the input has to be consumed completely + @param[in] ignore_comments whether comments should be ignored and treated + like whitespace (true) or yield a parse error (true); (optional, false by + default); only applies to the JSON file format. + + @return return value of the last processed SAX event + + @throw parse_error.101 if a parse error occurs; example: `""unexpected end + of input; expected string literal""` + @throw parse_error.102 if to_unicode fails or surrogate error + @throw parse_error.103 if to_unicode fails + + @complexity Linear in the length of the input. The parser is a predictive + LL(1) parser. The complexity can be higher if the SAX consumer @a sax has + a super-linear complexity. + + @note A UTF-8 byte order mark is silently ignored. + + @liveexample{The example below demonstrates the `sax_parse()` function + reading from string and processing the events with a user-defined SAX + event consumer.,sax_parse} + + @since version 3.2.0 + */ + template + JSON_HEDLEY_NON_NULL(2) + static bool sax_parse(InputType&& i, SAX* sax, + input_format_t format = input_format_t::json, + const bool strict = true, + const bool ignore_comments = false) + { + auto ia = detail::input_adapter(std::forward(i)); + return format == input_format_t::json + ? parser(std::move(ia), nullptr, true, ignore_comments).sax_parse(sax, strict) + : detail::binary_reader(std::move(ia)).sax_parse(format, sax, strict); + } + + template + JSON_HEDLEY_NON_NULL(3) + static bool sax_parse(IteratorType first, IteratorType last, SAX* sax, + input_format_t format = input_format_t::json, + const bool strict = true, + const bool ignore_comments = false) + { + auto ia = detail::input_adapter(std::move(first), std::move(last)); + return format == input_format_t::json + ? parser(std::move(ia), nullptr, true, ignore_comments).sax_parse(sax, strict) + : detail::binary_reader(std::move(ia)).sax_parse(format, sax, strict); + } + + template + JSON_HEDLEY_DEPRECATED_FOR(3.8.0, sax_parse(ptr, ptr + len, ...)) + JSON_HEDLEY_NON_NULL(2) + static bool sax_parse(detail::span_input_adapter&& i, SAX* sax, + input_format_t format = input_format_t::json, + const bool strict = true, + const bool ignore_comments = false) + { + auto ia = i.get(); + return format == input_format_t::json + ? parser(std::move(ia), nullptr, true, ignore_comments).sax_parse(sax, strict) + : detail::binary_reader(std::move(ia)).sax_parse(format, sax, strict); + } + + /*! + @brief deserialize from stream + @deprecated This stream operator is deprecated and will be removed in + version 4.0.0 of the library. Please use + @ref operator>>(std::istream&, basic_json&) + instead; that is, replace calls like `j << i;` with `i >> j;`. + @since version 1.0.0; deprecated since version 3.0.0 + */ + JSON_HEDLEY_DEPRECATED_FOR(3.0.0, operator>>(std::istream&, basic_json&)) + friend std::istream& operator<<(basic_json& j, std::istream& i) + { + return operator>>(i, j); + } + + /*! + @brief deserialize from stream + + Deserializes an input stream to a JSON value. + + @param[in,out] i input stream to read a serialized JSON value from + @param[in,out] j JSON value to write the deserialized input to + + @throw parse_error.101 in case of an unexpected token + @throw parse_error.102 if to_unicode fails or surrogate error + @throw parse_error.103 if to_unicode fails + + @complexity Linear in the length of the input. The parser is a predictive + LL(1) parser. + + @note A UTF-8 byte order mark is silently ignored. + + @liveexample{The example below shows how a JSON value is constructed by + reading a serialization from a stream.,operator_deserialize} + + @sa parse(std::istream&, const parser_callback_t) for a variant with a + parser callback function to filter values while parsing + + @since version 1.0.0 + */ + friend std::istream& operator>>(std::istream& i, basic_json& j) + { + parser(detail::input_adapter(i)).parse(false, j); + return i; + } + + /// @} + + /////////////////////////// + // convenience functions // + /////////////////////////// + + /*! + @brief return the type as string + + Returns the type name as string to be used in error messages - usually to + indicate that a function was called on a wrong JSON type. + + @return a string representation of a the @a m_type member: + Value type | return value + ----------- | ------------- + null | `"null"` + boolean | `"boolean"` + string | `"string"` + number | `"number"` (for all number types) + object | `"object"` + array | `"array"` + binary | `"binary"` + discarded | `"discarded"` + + @exceptionsafety No-throw guarantee: this function never throws exceptions. + + @complexity Constant. + + @liveexample{The following code exemplifies `type_name()` for all JSON + types.,type_name} + + @sa @ref type() -- return the type of the JSON value + @sa @ref operator value_t() -- return the type of the JSON value (implicit) + + @since version 1.0.0, public since 2.1.0, `const char*` and `noexcept` + since 3.0.0 + */ + JSON_HEDLEY_RETURNS_NON_NULL + const char* type_name() const noexcept + { + { + switch (m_type) + { + case value_t::null: + return "null"; + case value_t::object: + return "object"; + case value_t::array: + return "array"; + case value_t::string: + return "string"; + case value_t::boolean: + return "boolean"; + case value_t::binary: + return "binary"; + case value_t::discarded: + return "discarded"; + default: + return "number"; + } + } + } + + + private: + ////////////////////// + // member variables // + ////////////////////// + + /// the type of the current element + value_t m_type = value_t::null; + + /// the value of the current element + json_value m_value = {}; + + ////////////////////////////////////////// + // binary serialization/deserialization // + ////////////////////////////////////////// + + /// @name binary serialization/deserialization support + /// @{ + + public: + /*! + @brief create a CBOR serialization of a given JSON value + + Serializes a given JSON value @a j to a byte vector using the CBOR (Concise + Binary Object Representation) serialization format. CBOR is a binary + serialization format which aims to be more compact than JSON itself, yet + more efficient to parse. + + The library uses the following mapping from JSON values types to + CBOR types according to the CBOR specification (RFC 7049): + + JSON value type | value/range | CBOR type | first byte + --------------- | ------------------------------------------ | ---------------------------------- | --------------- + null | `null` | Null | 0xF6 + boolean | `true` | True | 0xF5 + boolean | `false` | False | 0xF4 + number_integer | -9223372036854775808..-2147483649 | Negative integer (8 bytes follow) | 0x3B + number_integer | -2147483648..-32769 | Negative integer (4 bytes follow) | 0x3A + number_integer | -32768..-129 | Negative integer (2 bytes follow) | 0x39 + number_integer | -128..-25 | Negative integer (1 byte follow) | 0x38 + number_integer | -24..-1 | Negative integer | 0x20..0x37 + number_integer | 0..23 | Integer | 0x00..0x17 + number_integer | 24..255 | Unsigned integer (1 byte follow) | 0x18 + number_integer | 256..65535 | Unsigned integer (2 bytes follow) | 0x19 + number_integer | 65536..4294967295 | Unsigned integer (4 bytes follow) | 0x1A + number_integer | 4294967296..18446744073709551615 | Unsigned integer (8 bytes follow) | 0x1B + number_unsigned | 0..23 | Integer | 0x00..0x17 + number_unsigned | 24..255 | Unsigned integer (1 byte follow) | 0x18 + number_unsigned | 256..65535 | Unsigned integer (2 bytes follow) | 0x19 + number_unsigned | 65536..4294967295 | Unsigned integer (4 bytes follow) | 0x1A + number_unsigned | 4294967296..18446744073709551615 | Unsigned integer (8 bytes follow) | 0x1B + number_float | *any value representable by a float* | Single-Precision Float | 0xFA + number_float | *any value NOT representable by a float* | Double-Precision Float | 0xFB + string | *length*: 0..23 | UTF-8 string | 0x60..0x77 + string | *length*: 23..255 | UTF-8 string (1 byte follow) | 0x78 + string | *length*: 256..65535 | UTF-8 string (2 bytes follow) | 0x79 + string | *length*: 65536..4294967295 | UTF-8 string (4 bytes follow) | 0x7A + string | *length*: 4294967296..18446744073709551615 | UTF-8 string (8 bytes follow) | 0x7B + array | *size*: 0..23 | array | 0x80..0x97 + array | *size*: 23..255 | array (1 byte follow) | 0x98 + array | *size*: 256..65535 | array (2 bytes follow) | 0x99 + array | *size*: 65536..4294967295 | array (4 bytes follow) | 0x9A + array | *size*: 4294967296..18446744073709551615 | array (8 bytes follow) | 0x9B + object | *size*: 0..23 | map | 0xA0..0xB7 + object | *size*: 23..255 | map (1 byte follow) | 0xB8 + object | *size*: 256..65535 | map (2 bytes follow) | 0xB9 + object | *size*: 65536..4294967295 | map (4 bytes follow) | 0xBA + object | *size*: 4294967296..18446744073709551615 | map (8 bytes follow) | 0xBB + binary | *size*: 0..23 | byte string | 0x40..0x57 + binary | *size*: 23..255 | byte string (1 byte follow) | 0x58 + binary | *size*: 256..65535 | byte string (2 bytes follow) | 0x59 + binary | *size*: 65536..4294967295 | byte string (4 bytes follow) | 0x5A + binary | *size*: 4294967296..18446744073709551615 | byte string (8 bytes follow) | 0x5B + + @note The mapping is **complete** in the sense that any JSON value type + can be converted to a CBOR value. + + @note If NaN or Infinity are stored inside a JSON number, they are + serialized properly. This behavior differs from the @ref dump() + function which serializes NaN or Infinity to `null`. + + @note The following CBOR types are not used in the conversion: + - UTF-8 strings terminated by "break" (0x7F) + - arrays terminated by "break" (0x9F) + - maps terminated by "break" (0xBF) + - byte strings terminated by "break" (0x5F) + - date/time (0xC0..0xC1) + - bignum (0xC2..0xC3) + - decimal fraction (0xC4) + - bigfloat (0xC5) + - expected conversions (0xD5..0xD7) + - simple values (0xE0..0xF3, 0xF8) + - undefined (0xF7) + - half-precision floats (0xF9) + - break (0xFF) + + @param[in] j JSON value to serialize + @return CBOR serialization as byte vector + + @complexity Linear in the size of the JSON value @a j. + + @liveexample{The example shows the serialization of a JSON value to a byte + vector in CBOR format.,to_cbor} + + @sa http://cbor.io + @sa @ref from_cbor(detail::input_adapter&&, const bool, const bool, const cbor_tag_handler_t) for the + analogous deserialization + @sa @ref to_msgpack(const basic_json&) for the related MessagePack format + @sa @ref to_ubjson(const basic_json&, const bool, const bool) for the + related UBJSON format + + @since version 2.0.9; compact representation of floating-point numbers + since version 3.8.0 + */ + static std::vector to_cbor(const basic_json& j) + { + std::vector result; + to_cbor(j, result); + return result; + } + + static void to_cbor(const basic_json& j, detail::output_adapter o) + { + binary_writer(o).write_cbor(j); + } + + static void to_cbor(const basic_json& j, detail::output_adapter o) + { + binary_writer(o).write_cbor(j); + } + + /*! + @brief create a MessagePack serialization of a given JSON value + + Serializes a given JSON value @a j to a byte vector using the MessagePack + serialization format. MessagePack is a binary serialization format which + aims to be more compact than JSON itself, yet more efficient to parse. + + The library uses the following mapping from JSON values types to + MessagePack types according to the MessagePack specification: + + JSON value type | value/range | MessagePack type | first byte + --------------- | --------------------------------- | ---------------- | ---------- + null | `null` | nil | 0xC0 + boolean | `true` | true | 0xC3 + boolean | `false` | false | 0xC2 + number_integer | -9223372036854775808..-2147483649 | int64 | 0xD3 + number_integer | -2147483648..-32769 | int32 | 0xD2 + number_integer | -32768..-129 | int16 | 0xD1 + number_integer | -128..-33 | int8 | 0xD0 + number_integer | -32..-1 | negative fixint | 0xE0..0xFF + number_integer | 0..127 | positive fixint | 0x00..0x7F + number_integer | 128..255 | uint 8 | 0xCC + number_integer | 256..65535 | uint 16 | 0xCD + number_integer | 65536..4294967295 | uint 32 | 0xCE + number_integer | 4294967296..18446744073709551615 | uint 64 | 0xCF + number_unsigned | 0..127 | positive fixint | 0x00..0x7F + number_unsigned | 128..255 | uint 8 | 0xCC + number_unsigned | 256..65535 | uint 16 | 0xCD + number_unsigned | 65536..4294967295 | uint 32 | 0xCE + number_unsigned | 4294967296..18446744073709551615 | uint 64 | 0xCF + number_float | *any value representable by a float* | float 32 | 0xCA + number_float | *any value NOT representable by a float* | float 64 | 0xCB + string | *length*: 0..31 | fixstr | 0xA0..0xBF + string | *length*: 32..255 | str 8 | 0xD9 + string | *length*: 256..65535 | str 16 | 0xDA + string | *length*: 65536..4294967295 | str 32 | 0xDB + array | *size*: 0..15 | fixarray | 0x90..0x9F + array | *size*: 16..65535 | array 16 | 0xDC + array | *size*: 65536..4294967295 | array 32 | 0xDD + object | *size*: 0..15 | fix map | 0x80..0x8F + object | *size*: 16..65535 | map 16 | 0xDE + object | *size*: 65536..4294967295 | map 32 | 0xDF + binary | *size*: 0..255 | bin 8 | 0xC4 + binary | *size*: 256..65535 | bin 16 | 0xC5 + binary | *size*: 65536..4294967295 | bin 32 | 0xC6 + + @note The mapping is **complete** in the sense that any JSON value type + can be converted to a MessagePack value. + + @note The following values can **not** be converted to a MessagePack value: + - strings with more than 4294967295 bytes + - byte strings with more than 4294967295 bytes + - arrays with more than 4294967295 elements + - objects with more than 4294967295 elements + + @note Any MessagePack output created @ref to_msgpack can be successfully + parsed by @ref from_msgpack. + + @note If NaN or Infinity are stored inside a JSON number, they are + serialized properly. This behavior differs from the @ref dump() + function which serializes NaN or Infinity to `null`. + + @param[in] j JSON value to serialize + @return MessagePack serialization as byte vector + + @complexity Linear in the size of the JSON value @a j. + + @liveexample{The example shows the serialization of a JSON value to a byte + vector in MessagePack format.,to_msgpack} + + @sa http://msgpack.org + @sa @ref from_msgpack for the analogous deserialization + @sa @ref to_cbor(const basic_json& for the related CBOR format + @sa @ref to_ubjson(const basic_json&, const bool, const bool) for the + related UBJSON format + + @since version 2.0.9 + */ + static std::vector to_msgpack(const basic_json& j) + { + std::vector result; + to_msgpack(j, result); + return result; + } + + static void to_msgpack(const basic_json& j, detail::output_adapter o) + { + binary_writer(o).write_msgpack(j); + } + + static void to_msgpack(const basic_json& j, detail::output_adapter o) + { + binary_writer(o).write_msgpack(j); + } + + /*! + @brief create a UBJSON serialization of a given JSON value + + Serializes a given JSON value @a j to a byte vector using the UBJSON + (Universal Binary JSON) serialization format. UBJSON aims to be more compact + than JSON itself, yet more efficient to parse. + + The library uses the following mapping from JSON values types to + UBJSON types according to the UBJSON specification: + + JSON value type | value/range | UBJSON type | marker + --------------- | --------------------------------- | ----------- | ------ + null | `null` | null | `Z` + boolean | `true` | true | `T` + boolean | `false` | false | `F` + number_integer | -9223372036854775808..-2147483649 | int64 | `L` + number_integer | -2147483648..-32769 | int32 | `l` + number_integer | -32768..-129 | int16 | `I` + number_integer | -128..127 | int8 | `i` + number_integer | 128..255 | uint8 | `U` + number_integer | 256..32767 | int16 | `I` + number_integer | 32768..2147483647 | int32 | `l` + number_integer | 2147483648..9223372036854775807 | int64 | `L` + number_unsigned | 0..127 | int8 | `i` + number_unsigned | 128..255 | uint8 | `U` + number_unsigned | 256..32767 | int16 | `I` + number_unsigned | 32768..2147483647 | int32 | `l` + number_unsigned | 2147483648..9223372036854775807 | int64 | `L` + number_unsigned | 2147483649..18446744073709551615 | high-precision | `H` + number_float | *any value* | float64 | `D` + string | *with shortest length indicator* | string | `S` + array | *see notes on optimized format* | array | `[` + object | *see notes on optimized format* | map | `{` + + @note The mapping is **complete** in the sense that any JSON value type + can be converted to a UBJSON value. + + @note The following values can **not** be converted to a UBJSON value: + - strings with more than 9223372036854775807 bytes (theoretical) + + @note The following markers are not used in the conversion: + - `Z`: no-op values are not created. + - `C`: single-byte strings are serialized with `S` markers. + + @note Any UBJSON output created @ref to_ubjson can be successfully parsed + by @ref from_ubjson. + + @note If NaN or Infinity are stored inside a JSON number, they are + serialized properly. This behavior differs from the @ref dump() + function which serializes NaN or Infinity to `null`. + + @note The optimized formats for containers are supported: Parameter + @a use_size adds size information to the beginning of a container and + removes the closing marker. Parameter @a use_type further checks + whether all elements of a container have the same type and adds the + type marker to the beginning of the container. The @a use_type + parameter must only be used together with @a use_size = true. Note + that @a use_size = true alone may result in larger representations - + the benefit of this parameter is that the receiving side is + immediately informed on the number of elements of the container. + + @note If the JSON data contains the binary type, the value stored is a list + of integers, as suggested by the UBJSON documentation. In particular, + this means that serialization and the deserialization of a JSON + containing binary values into UBJSON and back will result in a + different JSON object. + + @param[in] j JSON value to serialize + @param[in] use_size whether to add size annotations to container types + @param[in] use_type whether to add type annotations to container types + (must be combined with @a use_size = true) + @return UBJSON serialization as byte vector + + @complexity Linear in the size of the JSON value @a j. + + @liveexample{The example shows the serialization of a JSON value to a byte + vector in UBJSON format.,to_ubjson} + + @sa http://ubjson.org + @sa @ref from_ubjson(detail::input_adapter&&, const bool, const bool) for the + analogous deserialization + @sa @ref to_cbor(const basic_json& for the related CBOR format + @sa @ref to_msgpack(const basic_json&) for the related MessagePack format + + @since version 3.1.0 + */ + static std::vector to_ubjson(const basic_json& j, + const bool use_size = false, + const bool use_type = false) + { + std::vector result; + to_ubjson(j, result, use_size, use_type); + return result; + } + + static void to_ubjson(const basic_json& j, detail::output_adapter o, + const bool use_size = false, const bool use_type = false) + { + binary_writer(o).write_ubjson(j, use_size, use_type); + } + + static void to_ubjson(const basic_json& j, detail::output_adapter o, + const bool use_size = false, const bool use_type = false) + { + binary_writer(o).write_ubjson(j, use_size, use_type); + } + + + /*! + @brief Serializes the given JSON object `j` to BSON and returns a vector + containing the corresponding BSON-representation. + + BSON (Binary JSON) is a binary format in which zero or more ordered key/value pairs are + stored as a single entity (a so-called document). + + The library uses the following mapping from JSON values types to BSON types: + + JSON value type | value/range | BSON type | marker + --------------- | --------------------------------- | ----------- | ------ + null | `null` | null | 0x0A + boolean | `true`, `false` | boolean | 0x08 + number_integer | -9223372036854775808..-2147483649 | int64 | 0x12 + number_integer | -2147483648..2147483647 | int32 | 0x10 + number_integer | 2147483648..9223372036854775807 | int64 | 0x12 + number_unsigned | 0..2147483647 | int32 | 0x10 + number_unsigned | 2147483648..9223372036854775807 | int64 | 0x12 + number_unsigned | 9223372036854775808..18446744073709551615| -- | -- + number_float | *any value* | double | 0x01 + string | *any value* | string | 0x02 + array | *any value* | document | 0x04 + object | *any value* | document | 0x03 + binary | *any value* | binary | 0x05 + + @warning The mapping is **incomplete**, since only JSON-objects (and things + contained therein) can be serialized to BSON. + Also, integers larger than 9223372036854775807 cannot be serialized to BSON, + and the keys may not contain U+0000, since they are serialized a + zero-terminated c-strings. + + @throw out_of_range.407 if `j.is_number_unsigned() && j.get() > 9223372036854775807` + @throw out_of_range.409 if a key in `j` contains a NULL (U+0000) + @throw type_error.317 if `!j.is_object()` + + @pre The input `j` is required to be an object: `j.is_object() == true`. + + @note Any BSON output created via @ref to_bson can be successfully parsed + by @ref from_bson. + + @param[in] j JSON value to serialize + @return BSON serialization as byte vector + + @complexity Linear in the size of the JSON value @a j. + + @liveexample{The example shows the serialization of a JSON value to a byte + vector in BSON format.,to_bson} + + @sa http://bsonspec.org/spec.html + @sa @ref from_bson(detail::input_adapter&&, const bool strict) for the + analogous deserialization + @sa @ref to_ubjson(const basic_json&, const bool, const bool) for the + related UBJSON format + @sa @ref to_cbor(const basic_json&) for the related CBOR format + @sa @ref to_msgpack(const basic_json&) for the related MessagePack format + */ + static std::vector to_bson(const basic_json& j) + { + std::vector result; + to_bson(j, result); + return result; + } + + /*! + @brief Serializes the given JSON object `j` to BSON and forwards the + corresponding BSON-representation to the given output_adapter `o`. + @param j The JSON object to convert to BSON. + @param o The output adapter that receives the binary BSON representation. + @pre The input `j` shall be an object: `j.is_object() == true` + @sa @ref to_bson(const basic_json&) + */ + static void to_bson(const basic_json& j, detail::output_adapter o) + { + binary_writer(o).write_bson(j); + } + + /*! + @copydoc to_bson(const basic_json&, detail::output_adapter) + */ + static void to_bson(const basic_json& j, detail::output_adapter o) + { + binary_writer(o).write_bson(j); + } + + + /*! + @brief create a JSON value from an input in CBOR format + + Deserializes a given input @a i to a JSON value using the CBOR (Concise + Binary Object Representation) serialization format. + + The library maps CBOR types to JSON value types as follows: + + CBOR type | JSON value type | first byte + ---------------------- | --------------- | ---------- + Integer | number_unsigned | 0x00..0x17 + Unsigned integer | number_unsigned | 0x18 + Unsigned integer | number_unsigned | 0x19 + Unsigned integer | number_unsigned | 0x1A + Unsigned integer | number_unsigned | 0x1B + Negative integer | number_integer | 0x20..0x37 + Negative integer | number_integer | 0x38 + Negative integer | number_integer | 0x39 + Negative integer | number_integer | 0x3A + Negative integer | number_integer | 0x3B + Byte string | binary | 0x40..0x57 + Byte string | binary | 0x58 + Byte string | binary | 0x59 + Byte string | binary | 0x5A + Byte string | binary | 0x5B + UTF-8 string | string | 0x60..0x77 + UTF-8 string | string | 0x78 + UTF-8 string | string | 0x79 + UTF-8 string | string | 0x7A + UTF-8 string | string | 0x7B + UTF-8 string | string | 0x7F + array | array | 0x80..0x97 + array | array | 0x98 + array | array | 0x99 + array | array | 0x9A + array | array | 0x9B + array | array | 0x9F + map | object | 0xA0..0xB7 + map | object | 0xB8 + map | object | 0xB9 + map | object | 0xBA + map | object | 0xBB + map | object | 0xBF + False | `false` | 0xF4 + True | `true` | 0xF5 + Null | `null` | 0xF6 + Half-Precision Float | number_float | 0xF9 + Single-Precision Float | number_float | 0xFA + Double-Precision Float | number_float | 0xFB + + @warning The mapping is **incomplete** in the sense that not all CBOR + types can be converted to a JSON value. The following CBOR types + are not supported and will yield parse errors (parse_error.112): + - date/time (0xC0..0xC1) + - bignum (0xC2..0xC3) + - decimal fraction (0xC4) + - bigfloat (0xC5) + - expected conversions (0xD5..0xD7) + - simple values (0xE0..0xF3, 0xF8) + - undefined (0xF7) + + @warning CBOR allows map keys of any type, whereas JSON only allows + strings as keys in object values. Therefore, CBOR maps with keys + other than UTF-8 strings are rejected (parse_error.113). + + @note Any CBOR output created @ref to_cbor can be successfully parsed by + @ref from_cbor. + + @param[in] i an input in CBOR format convertible to an input adapter + @param[in] strict whether to expect the input to be consumed until EOF + (true by default) + @param[in] allow_exceptions whether to throw exceptions in case of a + parse error (optional, true by default) + @param[in] tag_handler how to treat CBOR tags (optional, error by default) + + @return deserialized JSON value; in case of a parse error and + @a allow_exceptions set to `false`, the return value will be + value_t::discarded. + + @throw parse_error.110 if the given input ends prematurely or the end of + file was not reached when @a strict was set to true + @throw parse_error.112 if unsupported features from CBOR were + used in the given input @a v or if the input is not valid CBOR + @throw parse_error.113 if a string was expected as map key, but not found + + @complexity Linear in the size of the input @a i. + + @liveexample{The example shows the deserialization of a byte vector in CBOR + format to a JSON value.,from_cbor} + + @sa http://cbor.io + @sa @ref to_cbor(const basic_json&) for the analogous serialization + @sa @ref from_msgpack(detail::input_adapter&&, const bool, const bool) for the + related MessagePack format + @sa @ref from_ubjson(detail::input_adapter&&, const bool, const bool) for the + related UBJSON format + + @since version 2.0.9; parameter @a start_index since 2.1.1; changed to + consume input adapters, removed start_index parameter, and added + @a strict parameter since 3.0.0; added @a allow_exceptions parameter + since 3.2.0; added @a tag_handler parameter since 3.9.0. + */ + template + JSON_HEDLEY_WARN_UNUSED_RESULT + static basic_json from_cbor(InputType&& i, + const bool strict = true, + const bool allow_exceptions = true, + const cbor_tag_handler_t tag_handler = cbor_tag_handler_t::error) + { + basic_json result; + detail::json_sax_dom_parser sdp(result, allow_exceptions); + auto ia = detail::input_adapter(std::forward(i)); + const bool res = binary_reader(std::move(ia)).sax_parse(input_format_t::cbor, &sdp, strict, tag_handler); + return res ? result : basic_json(value_t::discarded); + } + + /*! + @copydoc from_cbor(detail::input_adapter&&, const bool, const bool, const cbor_tag_handler_t) + */ + template + JSON_HEDLEY_WARN_UNUSED_RESULT + static basic_json from_cbor(IteratorType first, IteratorType last, + const bool strict = true, + const bool allow_exceptions = true, + const cbor_tag_handler_t tag_handler = cbor_tag_handler_t::error) + { + basic_json result; + detail::json_sax_dom_parser sdp(result, allow_exceptions); + auto ia = detail::input_adapter(std::move(first), std::move(last)); + const bool res = binary_reader(std::move(ia)).sax_parse(input_format_t::cbor, &sdp, strict, tag_handler); + return res ? result : basic_json(value_t::discarded); + } + + template + JSON_HEDLEY_WARN_UNUSED_RESULT + JSON_HEDLEY_DEPRECATED_FOR(3.8.0, from_cbor(ptr, ptr + len)) + static basic_json from_cbor(const T* ptr, std::size_t len, + const bool strict = true, + const bool allow_exceptions = true, + const cbor_tag_handler_t tag_handler = cbor_tag_handler_t::error) + { + return from_cbor(ptr, ptr + len, strict, allow_exceptions, tag_handler); + } + + + JSON_HEDLEY_WARN_UNUSED_RESULT + JSON_HEDLEY_DEPRECATED_FOR(3.8.0, from_cbor(ptr, ptr + len)) + static basic_json from_cbor(detail::span_input_adapter&& i, + const bool strict = true, + const bool allow_exceptions = true, + const cbor_tag_handler_t tag_handler = cbor_tag_handler_t::error) + { + basic_json result; + detail::json_sax_dom_parser sdp(result, allow_exceptions); + auto ia = i.get(); + const bool res = binary_reader(std::move(ia)).sax_parse(input_format_t::cbor, &sdp, strict, tag_handler); + return res ? result : basic_json(value_t::discarded); + } + + /*! + @brief create a JSON value from an input in MessagePack format + + Deserializes a given input @a i to a JSON value using the MessagePack + serialization format. + + The library maps MessagePack types to JSON value types as follows: + + MessagePack type | JSON value type | first byte + ---------------- | --------------- | ---------- + positive fixint | number_unsigned | 0x00..0x7F + fixmap | object | 0x80..0x8F + fixarray | array | 0x90..0x9F + fixstr | string | 0xA0..0xBF + nil | `null` | 0xC0 + false | `false` | 0xC2 + true | `true` | 0xC3 + float 32 | number_float | 0xCA + float 64 | number_float | 0xCB + uint 8 | number_unsigned | 0xCC + uint 16 | number_unsigned | 0xCD + uint 32 | number_unsigned | 0xCE + uint 64 | number_unsigned | 0xCF + int 8 | number_integer | 0xD0 + int 16 | number_integer | 0xD1 + int 32 | number_integer | 0xD2 + int 64 | number_integer | 0xD3 + str 8 | string | 0xD9 + str 16 | string | 0xDA + str 32 | string | 0xDB + array 16 | array | 0xDC + array 32 | array | 0xDD + map 16 | object | 0xDE + map 32 | object | 0xDF + bin 8 | binary | 0xC4 + bin 16 | binary | 0xC5 + bin 32 | binary | 0xC6 + ext 8 | binary | 0xC7 + ext 16 | binary | 0xC8 + ext 32 | binary | 0xC9 + fixext 1 | binary | 0xD4 + fixext 2 | binary | 0xD5 + fixext 4 | binary | 0xD6 + fixext 8 | binary | 0xD7 + fixext 16 | binary | 0xD8 + negative fixint | number_integer | 0xE0-0xFF + + @note Any MessagePack output created @ref to_msgpack can be successfully + parsed by @ref from_msgpack. + + @param[in] i an input in MessagePack format convertible to an input + adapter + @param[in] strict whether to expect the input to be consumed until EOF + (true by default) + @param[in] allow_exceptions whether to throw exceptions in case of a + parse error (optional, true by default) + + @return deserialized JSON value; in case of a parse error and + @a allow_exceptions set to `false`, the return value will be + value_t::discarded. + + @throw parse_error.110 if the given input ends prematurely or the end of + file was not reached when @a strict was set to true + @throw parse_error.112 if unsupported features from MessagePack were + used in the given input @a i or if the input is not valid MessagePack + @throw parse_error.113 if a string was expected as map key, but not found + + @complexity Linear in the size of the input @a i. + + @liveexample{The example shows the deserialization of a byte vector in + MessagePack format to a JSON value.,from_msgpack} + + @sa http://msgpack.org + @sa @ref to_msgpack(const basic_json&) for the analogous serialization + @sa @ref from_cbor(detail::input_adapter&&, const bool, const bool, const cbor_tag_handler_t) for the + related CBOR format + @sa @ref from_ubjson(detail::input_adapter&&, const bool, const bool) for + the related UBJSON format + @sa @ref from_bson(detail::input_adapter&&, const bool, const bool) for + the related BSON format + + @since version 2.0.9; parameter @a start_index since 2.1.1; changed to + consume input adapters, removed start_index parameter, and added + @a strict parameter since 3.0.0; added @a allow_exceptions parameter + since 3.2.0 + */ + template + JSON_HEDLEY_WARN_UNUSED_RESULT + static basic_json from_msgpack(InputType&& i, + const bool strict = true, + const bool allow_exceptions = true) + { + basic_json result; + detail::json_sax_dom_parser sdp(result, allow_exceptions); + auto ia = detail::input_adapter(std::forward(i)); + const bool res = binary_reader(std::move(ia)).sax_parse(input_format_t::msgpack, &sdp, strict); + return res ? result : basic_json(value_t::discarded); + } + + /*! + @copydoc from_msgpack(detail::input_adapter&&, const bool, const bool) + */ + template + JSON_HEDLEY_WARN_UNUSED_RESULT + static basic_json from_msgpack(IteratorType first, IteratorType last, + const bool strict = true, + const bool allow_exceptions = true) + { + basic_json result; + detail::json_sax_dom_parser sdp(result, allow_exceptions); + auto ia = detail::input_adapter(std::move(first), std::move(last)); + const bool res = binary_reader(std::move(ia)).sax_parse(input_format_t::msgpack, &sdp, strict); + return res ? result : basic_json(value_t::discarded); + } + + + template + JSON_HEDLEY_WARN_UNUSED_RESULT + JSON_HEDLEY_DEPRECATED_FOR(3.8.0, from_msgpack(ptr, ptr + len)) + static basic_json from_msgpack(const T* ptr, std::size_t len, + const bool strict = true, + const bool allow_exceptions = true) + { + return from_msgpack(ptr, ptr + len, strict, allow_exceptions); + } + + JSON_HEDLEY_WARN_UNUSED_RESULT + JSON_HEDLEY_DEPRECATED_FOR(3.8.0, from_msgpack(ptr, ptr + len)) + static basic_json from_msgpack(detail::span_input_adapter&& i, + const bool strict = true, + const bool allow_exceptions = true) + { + basic_json result; + detail::json_sax_dom_parser sdp(result, allow_exceptions); + auto ia = i.get(); + const bool res = binary_reader(std::move(ia)).sax_parse(input_format_t::msgpack, &sdp, strict); + return res ? result : basic_json(value_t::discarded); + } + + + /*! + @brief create a JSON value from an input in UBJSON format + + Deserializes a given input @a i to a JSON value using the UBJSON (Universal + Binary JSON) serialization format. + + The library maps UBJSON types to JSON value types as follows: + + UBJSON type | JSON value type | marker + ----------- | --------------------------------------- | ------ + no-op | *no value, next value is read* | `N` + null | `null` | `Z` + false | `false` | `F` + true | `true` | `T` + float32 | number_float | `d` + float64 | number_float | `D` + uint8 | number_unsigned | `U` + int8 | number_integer | `i` + int16 | number_integer | `I` + int32 | number_integer | `l` + int64 | number_integer | `L` + high-precision number | number_integer, number_unsigned, or number_float - depends on number string | 'H' + string | string | `S` + char | string | `C` + array | array (optimized values are supported) | `[` + object | object (optimized values are supported) | `{` + + @note The mapping is **complete** in the sense that any UBJSON value can + be converted to a JSON value. + + @param[in] i an input in UBJSON format convertible to an input adapter + @param[in] strict whether to expect the input to be consumed until EOF + (true by default) + @param[in] allow_exceptions whether to throw exceptions in case of a + parse error (optional, true by default) + + @return deserialized JSON value; in case of a parse error and + @a allow_exceptions set to `false`, the return value will be + value_t::discarded. + + @throw parse_error.110 if the given input ends prematurely or the end of + file was not reached when @a strict was set to true + @throw parse_error.112 if a parse error occurs + @throw parse_error.113 if a string could not be parsed successfully + + @complexity Linear in the size of the input @a i. + + @liveexample{The example shows the deserialization of a byte vector in + UBJSON format to a JSON value.,from_ubjson} + + @sa http://ubjson.org + @sa @ref to_ubjson(const basic_json&, const bool, const bool) for the + analogous serialization + @sa @ref from_cbor(detail::input_adapter&&, const bool, const bool, const cbor_tag_handler_t) for the + related CBOR format + @sa @ref from_msgpack(detail::input_adapter&&, const bool, const bool) for + the related MessagePack format + @sa @ref from_bson(detail::input_adapter&&, const bool, const bool) for + the related BSON format + + @since version 3.1.0; added @a allow_exceptions parameter since 3.2.0 + */ + template + JSON_HEDLEY_WARN_UNUSED_RESULT + static basic_json from_ubjson(InputType&& i, + const bool strict = true, + const bool allow_exceptions = true) + { + basic_json result; + detail::json_sax_dom_parser sdp(result, allow_exceptions); + auto ia = detail::input_adapter(std::forward(i)); + const bool res = binary_reader(std::move(ia)).sax_parse(input_format_t::ubjson, &sdp, strict); + return res ? result : basic_json(value_t::discarded); + } + + /*! + @copydoc from_ubjson(detail::input_adapter&&, const bool, const bool) + */ + template + JSON_HEDLEY_WARN_UNUSED_RESULT + static basic_json from_ubjson(IteratorType first, IteratorType last, + const bool strict = true, + const bool allow_exceptions = true) + { + basic_json result; + detail::json_sax_dom_parser sdp(result, allow_exceptions); + auto ia = detail::input_adapter(std::move(first), std::move(last)); + const bool res = binary_reader(std::move(ia)).sax_parse(input_format_t::ubjson, &sdp, strict); + return res ? result : basic_json(value_t::discarded); + } + + template + JSON_HEDLEY_WARN_UNUSED_RESULT + JSON_HEDLEY_DEPRECATED_FOR(3.8.0, from_ubjson(ptr, ptr + len)) + static basic_json from_ubjson(const T* ptr, std::size_t len, + const bool strict = true, + const bool allow_exceptions = true) + { + return from_ubjson(ptr, ptr + len, strict, allow_exceptions); + } + + JSON_HEDLEY_WARN_UNUSED_RESULT + JSON_HEDLEY_DEPRECATED_FOR(3.8.0, from_ubjson(ptr, ptr + len)) + static basic_json from_ubjson(detail::span_input_adapter&& i, + const bool strict = true, + const bool allow_exceptions = true) + { + basic_json result; + detail::json_sax_dom_parser sdp(result, allow_exceptions); + auto ia = i.get(); + const bool res = binary_reader(std::move(ia)).sax_parse(input_format_t::ubjson, &sdp, strict); + return res ? result : basic_json(value_t::discarded); + } + + + /*! + @brief Create a JSON value from an input in BSON format + + Deserializes a given input @a i to a JSON value using the BSON (Binary JSON) + serialization format. + + The library maps BSON record types to JSON value types as follows: + + BSON type | BSON marker byte | JSON value type + --------------- | ---------------- | --------------------------- + double | 0x01 | number_float + string | 0x02 | string + document | 0x03 | object + array | 0x04 | array + binary | 0x05 | still unsupported + undefined | 0x06 | still unsupported + ObjectId | 0x07 | still unsupported + boolean | 0x08 | boolean + UTC Date-Time | 0x09 | still unsupported + null | 0x0A | null + Regular Expr. | 0x0B | still unsupported + DB Pointer | 0x0C | still unsupported + JavaScript Code | 0x0D | still unsupported + Symbol | 0x0E | still unsupported + JavaScript Code | 0x0F | still unsupported + int32 | 0x10 | number_integer + Timestamp | 0x11 | still unsupported + 128-bit decimal float | 0x13 | still unsupported + Max Key | 0x7F | still unsupported + Min Key | 0xFF | still unsupported + + @warning The mapping is **incomplete**. The unsupported mappings + are indicated in the table above. + + @param[in] i an input in BSON format convertible to an input adapter + @param[in] strict whether to expect the input to be consumed until EOF + (true by default) + @param[in] allow_exceptions whether to throw exceptions in case of a + parse error (optional, true by default) + + @return deserialized JSON value; in case of a parse error and + @a allow_exceptions set to `false`, the return value will be + value_t::discarded. + + @throw parse_error.114 if an unsupported BSON record type is encountered + + @complexity Linear in the size of the input @a i. + + @liveexample{The example shows the deserialization of a byte vector in + BSON format to a JSON value.,from_bson} + + @sa http://bsonspec.org/spec.html + @sa @ref to_bson(const basic_json&) for the analogous serialization + @sa @ref from_cbor(detail::input_adapter&&, const bool, const bool, const cbor_tag_handler_t) for the + related CBOR format + @sa @ref from_msgpack(detail::input_adapter&&, const bool, const bool) for + the related MessagePack format + @sa @ref from_ubjson(detail::input_adapter&&, const bool, const bool) for the + related UBJSON format + */ + template + JSON_HEDLEY_WARN_UNUSED_RESULT + static basic_json from_bson(InputType&& i, + const bool strict = true, + const bool allow_exceptions = true) + { + basic_json result; + detail::json_sax_dom_parser sdp(result, allow_exceptions); + auto ia = detail::input_adapter(std::forward(i)); + const bool res = binary_reader(std::move(ia)).sax_parse(input_format_t::bson, &sdp, strict); + return res ? result : basic_json(value_t::discarded); + } + + /*! + @copydoc from_bson(detail::input_adapter&&, const bool, const bool) + */ + template + JSON_HEDLEY_WARN_UNUSED_RESULT + static basic_json from_bson(IteratorType first, IteratorType last, + const bool strict = true, + const bool allow_exceptions = true) + { + basic_json result; + detail::json_sax_dom_parser sdp(result, allow_exceptions); + auto ia = detail::input_adapter(std::move(first), std::move(last)); + const bool res = binary_reader(std::move(ia)).sax_parse(input_format_t::bson, &sdp, strict); + return res ? result : basic_json(value_t::discarded); + } + + template + JSON_HEDLEY_WARN_UNUSED_RESULT + JSON_HEDLEY_DEPRECATED_FOR(3.8.0, from_bson(ptr, ptr + len)) + static basic_json from_bson(const T* ptr, std::size_t len, + const bool strict = true, + const bool allow_exceptions = true) + { + return from_bson(ptr, ptr + len, strict, allow_exceptions); + } + + JSON_HEDLEY_WARN_UNUSED_RESULT + JSON_HEDLEY_DEPRECATED_FOR(3.8.0, from_bson(ptr, ptr + len)) + static basic_json from_bson(detail::span_input_adapter&& i, + const bool strict = true, + const bool allow_exceptions = true) + { + basic_json result; + detail::json_sax_dom_parser sdp(result, allow_exceptions); + auto ia = i.get(); + const bool res = binary_reader(std::move(ia)).sax_parse(input_format_t::bson, &sdp, strict); + return res ? result : basic_json(value_t::discarded); + } + /// @} + + ////////////////////////// + // JSON Pointer support // + ////////////////////////// + + /// @name JSON Pointer functions + /// @{ + + /*! + @brief access specified element via JSON Pointer + + Uses a JSON pointer to retrieve a reference to the respective JSON value. + No bound checking is performed. Similar to @ref operator[](const typename + object_t::key_type&), `null` values are created in arrays and objects if + necessary. + + In particular: + - If the JSON pointer points to an object key that does not exist, it + is created an filled with a `null` value before a reference to it + is returned. + - If the JSON pointer points to an array index that does not exist, it + is created an filled with a `null` value before a reference to it + is returned. All indices between the current maximum and the given + index are also filled with `null`. + - The special value `-` is treated as a synonym for the index past the + end. + + @param[in] ptr a JSON pointer + + @return reference to the element pointed to by @a ptr + + @complexity Constant. + + @throw parse_error.106 if an array index begins with '0' + @throw parse_error.109 if an array index was not a number + @throw out_of_range.404 if the JSON pointer can not be resolved + + @liveexample{The behavior is shown in the example.,operatorjson_pointer} + + @since version 2.0.0 + */ + reference operator[](const json_pointer& ptr) + { + return ptr.get_unchecked(this); + } + + /*! + @brief access specified element via JSON Pointer + + Uses a JSON pointer to retrieve a reference to the respective JSON value. + No bound checking is performed. The function does not change the JSON + value; no `null` values are created. In particular, the special value + `-` yields an exception. + + @param[in] ptr JSON pointer to the desired element + + @return const reference to the element pointed to by @a ptr + + @complexity Constant. + + @throw parse_error.106 if an array index begins with '0' + @throw parse_error.109 if an array index was not a number + @throw out_of_range.402 if the array index '-' is used + @throw out_of_range.404 if the JSON pointer can not be resolved + + @liveexample{The behavior is shown in the example.,operatorjson_pointer_const} + + @since version 2.0.0 + */ + const_reference operator[](const json_pointer& ptr) const + { + return ptr.get_unchecked(this); + } + + /*! + @brief access specified element via JSON Pointer + + Returns a reference to the element at with specified JSON pointer @a ptr, + with bounds checking. + + @param[in] ptr JSON pointer to the desired element + + @return reference to the element pointed to by @a ptr + + @throw parse_error.106 if an array index in the passed JSON pointer @a ptr + begins with '0'. See example below. + + @throw parse_error.109 if an array index in the passed JSON pointer @a ptr + is not a number. See example below. + + @throw out_of_range.401 if an array index in the passed JSON pointer @a ptr + is out of range. See example below. + + @throw out_of_range.402 if the array index '-' is used in the passed JSON + pointer @a ptr. As `at` provides checked access (and no elements are + implicitly inserted), the index '-' is always invalid. See example below. + + @throw out_of_range.403 if the JSON pointer describes a key of an object + which cannot be found. See example below. + + @throw out_of_range.404 if the JSON pointer @a ptr can not be resolved. + See example below. + + @exceptionsafety Strong guarantee: if an exception is thrown, there are no + changes in the JSON value. + + @complexity Constant. + + @since version 2.0.0 + + @liveexample{The behavior is shown in the example.,at_json_pointer} + */ + reference at(const json_pointer& ptr) + { + return ptr.get_checked(this); + } + + /*! + @brief access specified element via JSON Pointer + + Returns a const reference to the element at with specified JSON pointer @a + ptr, with bounds checking. + + @param[in] ptr JSON pointer to the desired element + + @return reference to the element pointed to by @a ptr + + @throw parse_error.106 if an array index in the passed JSON pointer @a ptr + begins with '0'. See example below. + + @throw parse_error.109 if an array index in the passed JSON pointer @a ptr + is not a number. See example below. + + @throw out_of_range.401 if an array index in the passed JSON pointer @a ptr + is out of range. See example below. + + @throw out_of_range.402 if the array index '-' is used in the passed JSON + pointer @a ptr. As `at` provides checked access (and no elements are + implicitly inserted), the index '-' is always invalid. See example below. + + @throw out_of_range.403 if the JSON pointer describes a key of an object + which cannot be found. See example below. + + @throw out_of_range.404 if the JSON pointer @a ptr can not be resolved. + See example below. + + @exceptionsafety Strong guarantee: if an exception is thrown, there are no + changes in the JSON value. + + @complexity Constant. + + @since version 2.0.0 + + @liveexample{The behavior is shown in the example.,at_json_pointer_const} + */ + const_reference at(const json_pointer& ptr) const + { + return ptr.get_checked(this); + } + + /*! + @brief return flattened JSON value + + The function creates a JSON object whose keys are JSON pointers (see [RFC + 6901](https://tools.ietf.org/html/rfc6901)) and whose values are all + primitive. The original JSON value can be restored using the @ref + unflatten() function. + + @return an object that maps JSON pointers to primitive values + + @note Empty objects and arrays are flattened to `null` and will not be + reconstructed correctly by the @ref unflatten() function. + + @complexity Linear in the size the JSON value. + + @liveexample{The following code shows how a JSON object is flattened to an + object whose keys consist of JSON pointers.,flatten} + + @sa @ref unflatten() for the reverse function + + @since version 2.0.0 + */ + basic_json flatten() const + { + basic_json result(value_t::object); + json_pointer::flatten("", *this, result); + return result; + } + + /*! + @brief unflatten a previously flattened JSON value + + The function restores the arbitrary nesting of a JSON value that has been + flattened before using the @ref flatten() function. The JSON value must + meet certain constraints: + 1. The value must be an object. + 2. The keys must be JSON pointers (see + [RFC 6901](https://tools.ietf.org/html/rfc6901)) + 3. The mapped values must be primitive JSON types. + + @return the original JSON from a flattened version + + @note Empty objects and arrays are flattened by @ref flatten() to `null` + values and can not unflattened to their original type. Apart from + this example, for a JSON value `j`, the following is always true: + `j == j.flatten().unflatten()`. + + @complexity Linear in the size the JSON value. + + @throw type_error.314 if value is not an object + @throw type_error.315 if object values are not primitive + + @liveexample{The following code shows how a flattened JSON object is + unflattened into the original nested JSON object.,unflatten} + + @sa @ref flatten() for the reverse function + + @since version 2.0.0 + */ + basic_json unflatten() const + { + return json_pointer::unflatten(*this); + } + + /// @} + + ////////////////////////// + // JSON Patch functions // + ////////////////////////// + + /// @name JSON Patch functions + /// @{ + + /*! + @brief applies a JSON patch + + [JSON Patch](http://jsonpatch.com) defines a JSON document structure for + expressing a sequence of operations to apply to a JSON) document. With + this function, a JSON Patch is applied to the current JSON value by + executing all operations from the patch. + + @param[in] json_patch JSON patch document + @return patched document + + @note The application of a patch is atomic: Either all operations succeed + and the patched document is returned or an exception is thrown. In + any case, the original value is not changed: the patch is applied + to a copy of the value. + + @throw parse_error.104 if the JSON patch does not consist of an array of + objects + + @throw parse_error.105 if the JSON patch is malformed (e.g., mandatory + attributes are missing); example: `"operation add must have member path"` + + @throw out_of_range.401 if an array index is out of range. + + @throw out_of_range.403 if a JSON pointer inside the patch could not be + resolved successfully in the current JSON value; example: `"key baz not + found"` + + @throw out_of_range.405 if JSON pointer has no parent ("add", "remove", + "move") + + @throw other_error.501 if "test" operation was unsuccessful + + @complexity Linear in the size of the JSON value and the length of the + JSON patch. As usually only a fraction of the JSON value is affected by + the patch, the complexity can usually be neglected. + + @liveexample{The following code shows how a JSON patch is applied to a + value.,patch} + + @sa @ref diff -- create a JSON patch by comparing two JSON values + + @sa [RFC 6902 (JSON Patch)](https://tools.ietf.org/html/rfc6902) + @sa [RFC 6901 (JSON Pointer)](https://tools.ietf.org/html/rfc6901) + + @since version 2.0.0 + */ + basic_json patch(const basic_json& json_patch) const + { + // make a working copy to apply the patch to + basic_json result = *this; + + // the valid JSON Patch operations + enum class patch_operations {add, remove, replace, move, copy, test, invalid}; + + const auto get_op = [](const std::string & op) + { + if (op == "add") + { + return patch_operations::add; + } + if (op == "remove") + { + return patch_operations::remove; + } + if (op == "replace") + { + return patch_operations::replace; + } + if (op == "move") + { + return patch_operations::move; + } + if (op == "copy") + { + return patch_operations::copy; + } + if (op == "test") + { + return patch_operations::test; + } + + return patch_operations::invalid; + }; + + // wrapper for "add" operation; add value at ptr + const auto operation_add = [&result](json_pointer & ptr, basic_json val) + { + // adding to the root of the target document means replacing it + if (ptr.empty()) + { + result = val; + return; + } + + // make sure the top element of the pointer exists + json_pointer top_pointer = ptr.top(); + if (top_pointer != ptr) + { + result.at(top_pointer); + } + + // get reference to parent of JSON pointer ptr + const auto last_path = ptr.back(); + ptr.pop_back(); + basic_json& parent = result[ptr]; + + switch (parent.m_type) + { + case value_t::null: + case value_t::object: + { + // use operator[] to add value + parent[last_path] = val; + break; + } + + case value_t::array: + { + if (last_path == "-") + { + // special case: append to back + parent.push_back(val); + } + else + { + const auto idx = json_pointer::array_index(last_path); + if (JSON_HEDLEY_UNLIKELY(idx > parent.size())) + { + // avoid undefined behavior + JSON_THROW(out_of_range::create(401, "array index " + std::to_string(idx) + " is out of range")); + } + + // default case: insert add offset + parent.insert(parent.begin() + static_cast(idx), val); + } + break; + } + + // if there exists a parent it cannot be primitive + default: // LCOV_EXCL_LINE + JSON_ASSERT(false); // LCOV_EXCL_LINE + } + }; + + // wrapper for "remove" operation; remove value at ptr + const auto operation_remove = [&result](json_pointer & ptr) + { + // get reference to parent of JSON pointer ptr + const auto last_path = ptr.back(); + ptr.pop_back(); + basic_json& parent = result.at(ptr); + + // remove child + if (parent.is_object()) + { + // perform range check + auto it = parent.find(last_path); + if (JSON_HEDLEY_LIKELY(it != parent.end())) + { + parent.erase(it); + } + else + { + JSON_THROW(out_of_range::create(403, "key '" + last_path + "' not found")); + } + } + else if (parent.is_array()) + { + // note erase performs range check + parent.erase(json_pointer::array_index(last_path)); + } + }; + + // type check: top level value must be an array + if (JSON_HEDLEY_UNLIKELY(!json_patch.is_array())) + { + JSON_THROW(parse_error::create(104, 0, "JSON patch must be an array of objects")); + } + + // iterate and apply the operations + for (const auto& val : json_patch) + { + // wrapper to get a value for an operation + const auto get_value = [&val](const std::string & op, + const std::string & member, + bool string_type) -> basic_json & + { + // find value + auto it = val.m_value.object->find(member); + + // context-sensitive error message + const auto error_msg = (op == "op") ? "operation" : "operation '" + op + "'"; + + // check if desired value is present + if (JSON_HEDLEY_UNLIKELY(it == val.m_value.object->end())) + { + JSON_THROW(parse_error::create(105, 0, error_msg + " must have member '" + member + "'")); + } + + // check if result is of type string + if (JSON_HEDLEY_UNLIKELY(string_type && !it->second.is_string())) + { + JSON_THROW(parse_error::create(105, 0, error_msg + " must have string member '" + member + "'")); + } + + // no error: return value + return it->second; + }; + + // type check: every element of the array must be an object + if (JSON_HEDLEY_UNLIKELY(!val.is_object())) + { + JSON_THROW(parse_error::create(104, 0, "JSON patch must be an array of objects")); + } + + // collect mandatory members + const auto op = get_value("op", "op", true).template get(); + const auto path = get_value(op, "path", true).template get(); + json_pointer ptr(path); + + switch (get_op(op)) + { + case patch_operations::add: + { + operation_add(ptr, get_value("add", "value", false)); + break; + } + + case patch_operations::remove: + { + operation_remove(ptr); + break; + } + + case patch_operations::replace: + { + // the "path" location must exist - use at() + result.at(ptr) = get_value("replace", "value", false); + break; + } + + case patch_operations::move: + { + const auto from_path = get_value("move", "from", true).template get(); + json_pointer from_ptr(from_path); + + // the "from" location must exist - use at() + basic_json v = result.at(from_ptr); + + // The move operation is functionally identical to a + // "remove" operation on the "from" location, followed + // immediately by an "add" operation at the target + // location with the value that was just removed. + operation_remove(from_ptr); + operation_add(ptr, v); + break; + } + + case patch_operations::copy: + { + const auto from_path = get_value("copy", "from", true).template get(); + const json_pointer from_ptr(from_path); + + // the "from" location must exist - use at() + basic_json v = result.at(from_ptr); + + // The copy is functionally identical to an "add" + // operation at the target location using the value + // specified in the "from" member. + operation_add(ptr, v); + break; + } + + case patch_operations::test: + { + bool success = false; + JSON_TRY + { + // check if "value" matches the one at "path" + // the "path" location must exist - use at() + success = (result.at(ptr) == get_value("test", "value", false)); + } + JSON_INTERNAL_CATCH (out_of_range&) + { + // ignore out of range errors: success remains false + } + + // throw an exception if test fails + if (JSON_HEDLEY_UNLIKELY(!success)) + { + JSON_THROW(other_error::create(501, "unsuccessful: " + val.dump())); + } + + break; + } + + default: + { + // op must be "add", "remove", "replace", "move", "copy", or + // "test" + JSON_THROW(parse_error::create(105, 0, "operation value '" + op + "' is invalid")); + } + } + } + + return result; + } + + /*! + @brief creates a diff as a JSON patch + + Creates a [JSON Patch](http://jsonpatch.com) so that value @a source can + be changed into the value @a target by calling @ref patch function. + + @invariant For two JSON values @a source and @a target, the following code + yields always `true`: + @code {.cpp} + source.patch(diff(source, target)) == target; + @endcode + + @note Currently, only `remove`, `add`, and `replace` operations are + generated. + + @param[in] source JSON value to compare from + @param[in] target JSON value to compare against + @param[in] path helper value to create JSON pointers + + @return a JSON patch to convert the @a source to @a target + + @complexity Linear in the lengths of @a source and @a target. + + @liveexample{The following code shows how a JSON patch is created as a + diff for two JSON values.,diff} + + @sa @ref patch -- apply a JSON patch + @sa @ref merge_patch -- apply a JSON Merge Patch + + @sa [RFC 6902 (JSON Patch)](https://tools.ietf.org/html/rfc6902) + + @since version 2.0.0 + */ + JSON_HEDLEY_WARN_UNUSED_RESULT + static basic_json diff(const basic_json& source, const basic_json& target, + const std::string& path = "") + { + // the patch + basic_json result(value_t::array); + + // if the values are the same, return empty patch + if (source == target) + { + return result; + } + + if (source.type() != target.type()) + { + // different types: replace value + result.push_back( + { + {"op", "replace"}, {"path", path}, {"value", target} + }); + return result; + } + + switch (source.type()) + { + case value_t::array: + { + // first pass: traverse common elements + std::size_t i = 0; + while (i < source.size() && i < target.size()) + { + // recursive call to compare array values at index i + auto temp_diff = diff(source[i], target[i], path + "/" + std::to_string(i)); + result.insert(result.end(), temp_diff.begin(), temp_diff.end()); + ++i; + } + + // i now reached the end of at least one array + // in a second pass, traverse the remaining elements + + // remove my remaining elements + const auto end_index = static_cast(result.size()); + while (i < source.size()) + { + // add operations in reverse order to avoid invalid + // indices + result.insert(result.begin() + end_index, object( + { + {"op", "remove"}, + {"path", path + "/" + std::to_string(i)} + })); + ++i; + } + + // add other remaining elements + while (i < target.size()) + { + result.push_back( + { + {"op", "add"}, + {"path", path + "/-"}, + {"value", target[i]} + }); + ++i; + } + + break; + } + + case value_t::object: + { + // first pass: traverse this object's elements + for (auto it = source.cbegin(); it != source.cend(); ++it) + { + // escape the key name to be used in a JSON patch + const auto key = json_pointer::escape(it.key()); + + if (target.find(it.key()) != target.end()) + { + // recursive call to compare object values at key it + auto temp_diff = diff(it.value(), target[it.key()], path + "/" + key); + result.insert(result.end(), temp_diff.begin(), temp_diff.end()); + } + else + { + // found a key that is not in o -> remove it + result.push_back(object( + { + {"op", "remove"}, {"path", path + "/" + key} + })); + } + } + + // second pass: traverse other object's elements + for (auto it = target.cbegin(); it != target.cend(); ++it) + { + if (source.find(it.key()) == source.end()) + { + // found a key that is not in this -> add it + const auto key = json_pointer::escape(it.key()); + result.push_back( + { + {"op", "add"}, {"path", path + "/" + key}, + {"value", it.value()} + }); + } + } + + break; + } + + default: + { + // both primitive type: replace value + result.push_back( + { + {"op", "replace"}, {"path", path}, {"value", target} + }); + break; + } + } + + return result; + } + + /// @} + + //////////////////////////////// + // JSON Merge Patch functions // + //////////////////////////////// + + /// @name JSON Merge Patch functions + /// @{ + + /*! + @brief applies a JSON Merge Patch + + The merge patch format is primarily intended for use with the HTTP PATCH + method as a means of describing a set of modifications to a target + resource's content. This function applies a merge patch to the current + JSON value. + + The function implements the following algorithm from Section 2 of + [RFC 7396 (JSON Merge Patch)](https://tools.ietf.org/html/rfc7396): + + ``` + define MergePatch(Target, Patch): + if Patch is an Object: + if Target is not an Object: + Target = {} // Ignore the contents and set it to an empty Object + for each Name/Value pair in Patch: + if Value is null: + if Name exists in Target: + remove the Name/Value pair from Target + else: + Target[Name] = MergePatch(Target[Name], Value) + return Target + else: + return Patch + ``` + + Thereby, `Target` is the current object; that is, the patch is applied to + the current value. + + @param[in] apply_patch the patch to apply + + @complexity Linear in the lengths of @a patch. + + @liveexample{The following code shows how a JSON Merge Patch is applied to + a JSON document.,merge_patch} + + @sa @ref patch -- apply a JSON patch + @sa [RFC 7396 (JSON Merge Patch)](https://tools.ietf.org/html/rfc7396) + + @since version 3.0.0 + */ + void merge_patch(const basic_json& apply_patch) + { + if (apply_patch.is_object()) + { + if (!is_object()) + { + *this = object(); + } + for (auto it = apply_patch.begin(); it != apply_patch.end(); ++it) + { + if (it.value().is_null()) + { + erase(it.key()); + } + else + { + operator[](it.key()).merge_patch(it.value()); + } + } + } + else + { + *this = apply_patch; + } + } + + /// @} +}; + +/*! +@brief user-defined to_string function for JSON values + +This function implements a user-defined to_string for JSON objects. + +@param[in] j a JSON object +@return a std::string object +*/ + +NLOHMANN_BASIC_JSON_TPL_DECLARATION +std::string to_string(const NLOHMANN_BASIC_JSON_TPL& j) +{ + return j.dump(); +} +} // namespace nlohmann + +/////////////////////// +// nonmember support // +/////////////////////// + +// specialization of std::swap, and std::hash +namespace std +{ + +/// hash value for JSON objects +template<> +struct hash +{ + /*! + @brief return a hash value for a JSON object + + @since version 1.0.0 + */ + std::size_t operator()(const nlohmann::json& j) const + { + return nlohmann::detail::hash(j); + } +}; + +/// specialization for std::less +/// @note: do not remove the space after '<', +/// see https://github.com/nlohmann/json/pull/679 +template<> +struct less<::nlohmann::detail::value_t> +{ + /*! + @brief compare two value_t enum values + @since version 3.0.0 + */ + bool operator()(nlohmann::detail::value_t lhs, + nlohmann::detail::value_t rhs) const noexcept + { + return nlohmann::detail::operator<(lhs, rhs); + } +}; + +// C++20 prohibit function specialization in the std namespace. +#ifndef JSON_HAS_CPP_20 + +/*! +@brief exchanges the values of two JSON objects + +@since version 1.0.0 +*/ +template<> +inline void swap(nlohmann::json& j1, nlohmann::json& j2) noexcept( + is_nothrow_move_constructible::value&& + is_nothrow_move_assignable::value + ) +{ + j1.swap(j2); +} + +#endif + +} // namespace std + +/*! +@brief user-defined string literal for JSON values + +This operator implements a user-defined string literal for JSON objects. It +can be used by adding `"_json"` to a string literal and returns a JSON object +if no parse error occurred. + +@param[in] s a string representation of a JSON object +@param[in] n the length of string @a s +@return a JSON object + +@since version 1.0.0 +*/ +JSON_HEDLEY_NON_NULL(1) +inline nlohmann::json operator "" _json(const char* s, std::size_t n) +{ + return nlohmann::json::parse(s, s + n); +} + +/*! +@brief user-defined string literal for JSON pointer + +This operator implements a user-defined string literal for JSON Pointers. It +can be used by adding `"_json_pointer"` to a string literal and returns a JSON pointer +object if no parse error occurred. + +@param[in] s a string representation of a JSON Pointer +@param[in] n the length of string @a s +@return a JSON pointer object + +@since version 2.0.0 +*/ +JSON_HEDLEY_NON_NULL(1) +inline nlohmann::json::json_pointer operator "" _json_pointer(const char* s, std::size_t n) +{ + return nlohmann::json::json_pointer(std::string(s, n)); +} + +// #include + + +// restore GCC/clang diagnostic settings +#if defined(__clang__) || defined(__GNUC__) || defined(__GNUG__) + #pragma GCC diagnostic pop +#endif +#if defined(__clang__) + #pragma GCC diagnostic pop +#endif + +// clean up +#undef JSON_ASSERT +#undef JSON_INTERNAL_CATCH +#undef JSON_CATCH +#undef JSON_THROW +#undef JSON_TRY +#undef JSON_HAS_CPP_14 +#undef JSON_HAS_CPP_17 +#undef NLOHMANN_BASIC_JSON_TPL_DECLARATION +#undef NLOHMANN_BASIC_JSON_TPL +#undef JSON_EXPLICIT + +// #include +#undef JSON_HEDLEY_ALWAYS_INLINE +#undef JSON_HEDLEY_ARM_VERSION +#undef JSON_HEDLEY_ARM_VERSION_CHECK +#undef JSON_HEDLEY_ARRAY_PARAM +#undef JSON_HEDLEY_ASSUME +#undef JSON_HEDLEY_BEGIN_C_DECLS +#undef JSON_HEDLEY_CLANG_HAS_ATTRIBUTE +#undef JSON_HEDLEY_CLANG_HAS_BUILTIN +#undef JSON_HEDLEY_CLANG_HAS_CPP_ATTRIBUTE +#undef JSON_HEDLEY_CLANG_HAS_DECLSPEC_DECLSPEC_ATTRIBUTE +#undef JSON_HEDLEY_CLANG_HAS_EXTENSION +#undef JSON_HEDLEY_CLANG_HAS_FEATURE +#undef JSON_HEDLEY_CLANG_HAS_WARNING +#undef JSON_HEDLEY_COMPCERT_VERSION +#undef JSON_HEDLEY_COMPCERT_VERSION_CHECK +#undef JSON_HEDLEY_CONCAT +#undef JSON_HEDLEY_CONCAT3 +#undef JSON_HEDLEY_CONCAT3_EX +#undef JSON_HEDLEY_CONCAT_EX +#undef JSON_HEDLEY_CONST +#undef JSON_HEDLEY_CONSTEXPR +#undef JSON_HEDLEY_CONST_CAST +#undef JSON_HEDLEY_CPP_CAST +#undef JSON_HEDLEY_CRAY_VERSION +#undef JSON_HEDLEY_CRAY_VERSION_CHECK +#undef JSON_HEDLEY_C_DECL +#undef JSON_HEDLEY_DEPRECATED +#undef JSON_HEDLEY_DEPRECATED_FOR +#undef JSON_HEDLEY_DIAGNOSTIC_DISABLE_CAST_QUAL +#undef JSON_HEDLEY_DIAGNOSTIC_DISABLE_CPP98_COMPAT_WRAP_ +#undef JSON_HEDLEY_DIAGNOSTIC_DISABLE_DEPRECATED +#undef JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_CPP_ATTRIBUTES +#undef JSON_HEDLEY_DIAGNOSTIC_DISABLE_UNKNOWN_PRAGMAS +#undef JSON_HEDLEY_DIAGNOSTIC_POP +#undef JSON_HEDLEY_DIAGNOSTIC_PUSH +#undef JSON_HEDLEY_DMC_VERSION +#undef JSON_HEDLEY_DMC_VERSION_CHECK +#undef JSON_HEDLEY_EMPTY_BASES +#undef JSON_HEDLEY_EMSCRIPTEN_VERSION +#undef JSON_HEDLEY_EMSCRIPTEN_VERSION_CHECK +#undef JSON_HEDLEY_END_C_DECLS +#undef JSON_HEDLEY_FLAGS +#undef JSON_HEDLEY_FLAGS_CAST +#undef JSON_HEDLEY_GCC_HAS_ATTRIBUTE +#undef JSON_HEDLEY_GCC_HAS_BUILTIN +#undef JSON_HEDLEY_GCC_HAS_CPP_ATTRIBUTE +#undef JSON_HEDLEY_GCC_HAS_DECLSPEC_ATTRIBUTE +#undef JSON_HEDLEY_GCC_HAS_EXTENSION +#undef JSON_HEDLEY_GCC_HAS_FEATURE +#undef JSON_HEDLEY_GCC_HAS_WARNING +#undef JSON_HEDLEY_GCC_NOT_CLANG_VERSION_CHECK +#undef JSON_HEDLEY_GCC_VERSION +#undef JSON_HEDLEY_GCC_VERSION_CHECK +#undef JSON_HEDLEY_GNUC_HAS_ATTRIBUTE +#undef JSON_HEDLEY_GNUC_HAS_BUILTIN +#undef JSON_HEDLEY_GNUC_HAS_CPP_ATTRIBUTE +#undef JSON_HEDLEY_GNUC_HAS_DECLSPEC_ATTRIBUTE +#undef JSON_HEDLEY_GNUC_HAS_EXTENSION +#undef JSON_HEDLEY_GNUC_HAS_FEATURE +#undef JSON_HEDLEY_GNUC_HAS_WARNING +#undef JSON_HEDLEY_GNUC_VERSION +#undef JSON_HEDLEY_GNUC_VERSION_CHECK +#undef JSON_HEDLEY_HAS_ATTRIBUTE +#undef JSON_HEDLEY_HAS_BUILTIN +#undef JSON_HEDLEY_HAS_CPP_ATTRIBUTE +#undef JSON_HEDLEY_HAS_CPP_ATTRIBUTE_NS +#undef JSON_HEDLEY_HAS_DECLSPEC_ATTRIBUTE +#undef JSON_HEDLEY_HAS_EXTENSION +#undef JSON_HEDLEY_HAS_FEATURE +#undef JSON_HEDLEY_HAS_WARNING +#undef JSON_HEDLEY_IAR_VERSION +#undef JSON_HEDLEY_IAR_VERSION_CHECK +#undef JSON_HEDLEY_IBM_VERSION +#undef JSON_HEDLEY_IBM_VERSION_CHECK +#undef JSON_HEDLEY_IMPORT +#undef JSON_HEDLEY_INLINE +#undef JSON_HEDLEY_INTEL_VERSION +#undef JSON_HEDLEY_INTEL_VERSION_CHECK +#undef JSON_HEDLEY_IS_CONSTANT +#undef JSON_HEDLEY_IS_CONSTEXPR_ +#undef JSON_HEDLEY_LIKELY +#undef JSON_HEDLEY_MALLOC +#undef JSON_HEDLEY_MESSAGE +#undef JSON_HEDLEY_MSVC_VERSION +#undef JSON_HEDLEY_MSVC_VERSION_CHECK +#undef JSON_HEDLEY_NEVER_INLINE +#undef JSON_HEDLEY_NON_NULL +#undef JSON_HEDLEY_NO_ESCAPE +#undef JSON_HEDLEY_NO_RETURN +#undef JSON_HEDLEY_NO_THROW +#undef JSON_HEDLEY_NULL +#undef JSON_HEDLEY_PELLES_VERSION +#undef JSON_HEDLEY_PELLES_VERSION_CHECK +#undef JSON_HEDLEY_PGI_VERSION +#undef JSON_HEDLEY_PGI_VERSION_CHECK +#undef JSON_HEDLEY_PREDICT +#undef JSON_HEDLEY_PRINTF_FORMAT +#undef JSON_HEDLEY_PRIVATE +#undef JSON_HEDLEY_PUBLIC +#undef JSON_HEDLEY_PURE +#undef JSON_HEDLEY_REINTERPRET_CAST +#undef JSON_HEDLEY_REQUIRE +#undef JSON_HEDLEY_REQUIRE_CONSTEXPR +#undef JSON_HEDLEY_REQUIRE_MSG +#undef JSON_HEDLEY_RESTRICT +#undef JSON_HEDLEY_RETURNS_NON_NULL +#undef JSON_HEDLEY_SENTINEL +#undef JSON_HEDLEY_STATIC_ASSERT +#undef JSON_HEDLEY_STATIC_CAST +#undef JSON_HEDLEY_STRINGIFY +#undef JSON_HEDLEY_STRINGIFY_EX +#undef JSON_HEDLEY_SUNPRO_VERSION +#undef JSON_HEDLEY_SUNPRO_VERSION_CHECK +#undef JSON_HEDLEY_TINYC_VERSION +#undef JSON_HEDLEY_TINYC_VERSION_CHECK +#undef JSON_HEDLEY_TI_ARMCL_VERSION +#undef JSON_HEDLEY_TI_ARMCL_VERSION_CHECK +#undef JSON_HEDLEY_TI_CL2000_VERSION +#undef JSON_HEDLEY_TI_CL2000_VERSION_CHECK +#undef JSON_HEDLEY_TI_CL430_VERSION +#undef JSON_HEDLEY_TI_CL430_VERSION_CHECK +#undef JSON_HEDLEY_TI_CL6X_VERSION +#undef JSON_HEDLEY_TI_CL6X_VERSION_CHECK +#undef JSON_HEDLEY_TI_CL7X_VERSION +#undef JSON_HEDLEY_TI_CL7X_VERSION_CHECK +#undef JSON_HEDLEY_TI_CLPRU_VERSION +#undef JSON_HEDLEY_TI_CLPRU_VERSION_CHECK +#undef JSON_HEDLEY_TI_VERSION +#undef JSON_HEDLEY_TI_VERSION_CHECK +#undef JSON_HEDLEY_UNAVAILABLE +#undef JSON_HEDLEY_UNLIKELY +#undef JSON_HEDLEY_UNPREDICTABLE +#undef JSON_HEDLEY_UNREACHABLE +#undef JSON_HEDLEY_UNREACHABLE_RETURN +#undef JSON_HEDLEY_VERSION +#undef JSON_HEDLEY_VERSION_DECODE_MAJOR +#undef JSON_HEDLEY_VERSION_DECODE_MINOR +#undef JSON_HEDLEY_VERSION_DECODE_REVISION +#undef JSON_HEDLEY_VERSION_ENCODE +#undef JSON_HEDLEY_WARNING +#undef JSON_HEDLEY_WARN_UNUSED_RESULT +#undef JSON_HEDLEY_WARN_UNUSED_RESULT_MSG +#undef JSON_HEDLEY_FALL_THROUGH + + + +#endif // INCLUDE_NLOHMANN_JSON_HPP_ diff --git a/dep/jwt-cpp/include/jwt-cpp/picojson.h b/dep/jwt-cpp/include/picojson/picojson.h similarity index 96% rename from dep/jwt-cpp/include/jwt-cpp/picojson.h rename to dep/jwt-cpp/include/picojson/picojson.h index 24a60c5be..76742fe06 100644 --- a/dep/jwt-cpp/include/jwt-cpp/picojson.h +++ b/dep/jwt-cpp/include/picojson/picojson.h @@ -76,8 +76,14 @@ extern "C" { // experimental support for int64_t (see README.mkdn for detail) #ifdef PICOJSON_USE_INT64 #define __STDC_FORMAT_MACROS -#include +#include +#if __cplusplus >= 201103L +#include +#else +extern "C" { #include +} +#endif #endif // to disable the use of localeconv(3), set PICOJSON_USE_LOCALE to 0 @@ -104,6 +110,7 @@ extern "C" { #pragma warning(disable : 4244) // conversion from int to char #pragma warning(disable : 4127) // conditional expression is constant #pragma warning(disable : 4702) // unreachable code +#pragma warning(disable : 4706) // assignment within conditional expression #else #define SNPRINTF snprintf #endif @@ -123,7 +130,7 @@ enum { #endif }; -enum { INDENT_WIDTH = 2 }; +enum { INDENT_WIDTH = 2, DEFAULT_MAX_DEPTHS = 100 }; struct null {}; @@ -377,7 +384,7 @@ GET(array, *u_.array_) GET(object, *u_.object_) #ifdef PICOJSON_USE_INT64 GET(double, - (type_ == int64_type && (const_cast(this)->type_ = number_type, const_cast(this)->u_.number_ = u_.int64_), + (type_ == int64_type && (const_cast(this)->type_ = number_type, (const_cast(this)->u_.number_ = u_.int64_)), u_.number_)) GET(int64_t, u_.int64_) #else @@ -832,7 +839,7 @@ template inline bool _parse_object(Context &ct return false; } if (in.expect('}')) { - return true; + return ctx.parse_object_stop(); } do { std::string key; @@ -843,7 +850,7 @@ template inline bool _parse_object(Context &ct return false; } } while (in.expect(',')); - return in.expect('}'); + return in.expect('}') && ctx.parse_object_stop(); } template inline std::string _parse_number(input &in) { @@ -959,9 +966,10 @@ public: class default_parse_context { protected: value *out_; + size_t depths_; public: - default_parse_context(value *out) : out_(out) { + default_parse_context(value *out, size_t depths = DEFAULT_MAX_DEPTHS) : out_(out), depths_(depths) { } bool set_null() { *out_ = value(); @@ -986,27 +994,37 @@ public: return _parse_string(out_->get(), in); } bool parse_array_start() { + if (depths_ == 0) + return false; + --depths_; *out_ = value(array_type, false); return true; } template bool parse_array_item(input &in, size_t) { array &a = out_->get(); a.push_back(value()); - default_parse_context ctx(&a.back()); + default_parse_context ctx(&a.back(), depths_); return _parse(ctx, in); } bool parse_array_stop(size_t) { + ++depths_; return true; } bool parse_object_start() { + if (depths_ == 0) + return false; *out_ = value(object_type, false); return true; } template bool parse_object_item(input &in, const std::string &key) { object &o = out_->get(); - default_parse_context ctx(&o[key]); + default_parse_context ctx(&o[key], depths_); return _parse(ctx, in); } + bool parse_object_stop() { + ++depths_; + return true; + } private: default_parse_context(const default_parse_context &); @@ -1014,6 +1032,9 @@ private: }; class null_parse_context { +protected: + size_t depths_; + public: struct dummy_str { void push_back(int) { @@ -1021,7 +1042,7 @@ public: }; public: - null_parse_context() { + null_parse_context(size_t depths = DEFAULT_MAX_DEPTHS) : depths_(depths) { } bool set_null() { return true; @@ -1042,20 +1063,31 @@ public: return _parse_string(s, in); } bool parse_array_start() { + if (depths_ == 0) + return false; + --depths_; return true; } template bool parse_array_item(input &in, size_t) { return _parse(*this, in); } bool parse_array_stop(size_t) { + ++depths_; return true; } bool parse_object_start() { + if (depths_ == 0) + return false; + --depths_; return true; } template bool parse_object_item(input &in, const std::string &) { + ++depths_; return _parse(*this, in); } + bool parse_object_stop() { + return true; + } private: null_parse_context(const null_parse_context &); @@ -1165,4 +1197,4 @@ inline std::ostream &operator<<(std::ostream &os, const picojson::value &x) { #pragma warning(pop) #endif -#endif \ No newline at end of file +#endif diff --git a/dep/jwt-cpp/jwt-cpp.sln b/dep/jwt-cpp/jwt-cpp.sln deleted file mode 100644 index ef5abc2a6..000000000 --- a/dep/jwt-cpp/jwt-cpp.sln +++ /dev/null @@ -1,28 +0,0 @@ - -Microsoft Visual Studio Solution File, Format Version 12.00 -# Visual Studio 14 -VisualStudioVersion = 14.0.25420.1 -MinimumVisualStudioVersion = 10.0.40219.1 -Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "jwt-cpp", "jwt-cpp.vcxproj", "{1CA8C676-7F8E-434C-9069-8F20A562E6E9}" -EndProject -Global - GlobalSection(SolutionConfigurationPlatforms) = preSolution - Debug|x64 = Debug|x64 - Debug|x86 = Debug|x86 - Release|x64 = Release|x64 - Release|x86 = Release|x86 - EndGlobalSection - GlobalSection(ProjectConfigurationPlatforms) = postSolution - {1CA8C676-7F8E-434C-9069-8F20A562E6E9}.Debug|x64.ActiveCfg = Debug|x64 - {1CA8C676-7F8E-434C-9069-8F20A562E6E9}.Debug|x64.Build.0 = Debug|x64 - {1CA8C676-7F8E-434C-9069-8F20A562E6E9}.Debug|x86.ActiveCfg = Debug|Win32 - {1CA8C676-7F8E-434C-9069-8F20A562E6E9}.Debug|x86.Build.0 = Debug|Win32 - {1CA8C676-7F8E-434C-9069-8F20A562E6E9}.Release|x64.ActiveCfg = Release|x64 - {1CA8C676-7F8E-434C-9069-8F20A562E6E9}.Release|x64.Build.0 = Release|x64 - {1CA8C676-7F8E-434C-9069-8F20A562E6E9}.Release|x86.ActiveCfg = Release|Win32 - {1CA8C676-7F8E-434C-9069-8F20A562E6E9}.Release|x86.Build.0 = Release|Win32 - EndGlobalSection - GlobalSection(SolutionProperties) = preSolution - HideSolutionNode = FALSE - EndGlobalSection -EndGlobal diff --git a/dep/jwt-cpp/jwt-cpp.vcxproj b/dep/jwt-cpp/jwt-cpp.vcxproj deleted file mode 100644 index 7d791a4d4..000000000 --- a/dep/jwt-cpp/jwt-cpp.vcxproj +++ /dev/null @@ -1,160 +0,0 @@ - - - - - Debug - Win32 - - - Release - Win32 - - - Debug - x64 - - - Release - x64 - - - - {1CA8C676-7F8E-434C-9069-8F20A562E6E9} - Win32Proj - jwtcpp - 8.1 - - - - Application - true - v140 - Unicode - - - Application - false - v140 - true - Unicode - - - Application - true - v140 - Unicode - - - Application - false - v140 - true - Unicode - - - - - - - - - - - - - - - - - - - - - true - - - true - - - false - - - false - - - - - - Level3 - Disabled - WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions) - - - Console - true - gtest.lib;gtest_main.lib;%(AdditionalDependencies) - - - - - - - Level3 - Disabled - _DEBUG;_CONSOLE;%(PreprocessorDefinitions) - - - Console - true - gtest.lib;gtest_main.lib;%(AdditionalDependencies) - - - - - Level3 - - - MaxSpeed - true - true - WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions) - - - Console - true - true - true - gtest.lib;gtest_main.lib;%(AdditionalDependencies) - - - - - Level3 - - - MaxSpeed - true - true - NDEBUG;_CONSOLE;%(PreprocessorDefinitions) - - - Console - true - true - true - gtest.lib;gtest_main.lib;%(AdditionalDependencies) - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/dep/jwt-cpp/jwt-cpp.vcxproj.filters b/dep/jwt-cpp/jwt-cpp.vcxproj.filters deleted file mode 100644 index 6419b2d22..000000000 --- a/dep/jwt-cpp/jwt-cpp.vcxproj.filters +++ /dev/null @@ -1,36 +0,0 @@ - - - - - {4FC737F1-C7A5-4376-A066-2A32D752A2FF} - cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx - - - {93995380-89BD-4b04-88EB-625FBE52EBFB} - h;hh;hpp;hxx;hm;inl;inc;xsd - - - {67DA6AB6-F800-4c08-8B7A-83BB121AAD01} - rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx;tiff;tif;png;wav;mfcribbon-ms - - - - - Headerdateien - - - Headerdateien - - - Headerdateien - - - - - Quelldateien - - - Quelldateien - - - \ No newline at end of file diff --git a/dep/jwt-cpp/vcpkg/CONTROL b/dep/jwt-cpp/vcpkg/CONTROL deleted file mode 100644 index d29834fc6..000000000 --- a/dep/jwt-cpp/vcpkg/CONTROL +++ /dev/null @@ -1,3 +0,0 @@ -Source: jwt-cpp -Version: 2019-04-20 -Description: A header only library for creating and validating json web tokens in c++ \ No newline at end of file diff --git a/dep/jwt-cpp/vcpkg/fix-picojson.patch b/dep/jwt-cpp/vcpkg/fix-picojson.patch deleted file mode 100644 index 44c04fe58..000000000 --- a/dep/jwt-cpp/vcpkg/fix-picojson.patch +++ /dev/null @@ -1,12 +0,0 @@ -diff --git a/include/jwt-cpp/jwt.h b/include/jwt-cpp/jwt.h -index ec56810..a26fd97 100644 ---- a/include/jwt-cpp/jwt.h -+++ b/include/jwt-cpp/jwt.h -@@ -1,6 +1,6 @@ - #pragma once - #define PICOJSON_USE_INT64 --#include "picojson.h" -+#include "picojson/picojson.h" - #include "base.h" - #include - #include diff --git a/dep/jwt-cpp/vcpkg/fix-warning.patch b/dep/jwt-cpp/vcpkg/fix-warning.patch deleted file mode 100644 index d013a7782..000000000 --- a/dep/jwt-cpp/vcpkg/fix-warning.patch +++ /dev/null @@ -1,31 +0,0 @@ -diff --git a/include/jwt-cpp/base.h b/include/jwt-cpp/base.h -index dfca7fc..4d05c0b 100644 ---- a/include/jwt-cpp/base.h -+++ b/include/jwt-cpp/base.h -@@ -2,6 +2,10 @@ - #include - #include - -+#ifdef _MSC_VER -+#pragma warning(disable : 4267) -+#endif -+ - namespace jwt { - namespace alphabet { - struct base64 { -diff --git a/include/jwt-cpp/jwt.h b/include/jwt-cpp/jwt.h -index ec56810..313cef2 100644 ---- a/include/jwt-cpp/jwt.h -+++ b/include/jwt-cpp/jwt.h -@@ -12,6 +12,11 @@ - #include - #include - -+#ifdef _MSC_VER -+#pragma warning(disable : 4267) -+#pragma warning(disable : 4067) -+#endif -+ - //If openssl version less than 1.1 - #if OPENSSL_VERSION_NUMBER < 269484032 - #define OPENSSL10 diff --git a/dep/jwt-cpp/vcpkg/portfile.cmake b/dep/jwt-cpp/vcpkg/portfile.cmake deleted file mode 100644 index 1e10e3c21..000000000 --- a/dep/jwt-cpp/vcpkg/portfile.cmake +++ /dev/null @@ -1,23 +0,0 @@ -#header-only library -include(vcpkg_common_functions) - -set(SOURCE_PATH ${CURRENT_BUILDTREES_DIR}/src/jwt-cpp) - -vcpkg_from_github(OUT_SOURCE_PATH SOURCE_PATH - REPO Thalhammer/jwt-cpp - REF f0e37a79f605312686065405dd720fc197cc3df0 - SHA512 ae83c205dbb340dedc58d0d3f0e2453c4edcf5ce43b401f49d02692dc8a2a4b7260f1ced05ddfa7c1d5d6f92446e232629ddbdf67a58a119b50c5c8163591598 - HEAD_REF master - PATCHES fix-picojson.patch - fix-warning.patch) - -# Copy the constexpr header files -file(GLOB HEADER_FILES ${SOURCE_PATH}/include/jwt-cpp/*) -file(COPY ${HEADER_FILES} - DESTINATION ${CURRENT_PACKAGES_DIR}/include/jwt-cpp - REGEX "\.(gitattributes|gitignore|picojson.h)$" EXCLUDE) - -# Put the licence file where vcpkg expects it -file(COPY ${SOURCE_PATH}/LICENSE - DESTINATION ${CURRENT_PACKAGES_DIR}/share/jwt-cpp) -file(RENAME ${CURRENT_PACKAGES_DIR}/share/jwt-cpp/LICENSE ${CURRENT_PACKAGES_DIR}/share/jwt-cpp/copyright) \ No newline at end of file diff --git a/src/zm_crypt.cpp b/src/zm_crypt.cpp index 0682970c1..9a63e4593 100644 --- a/src/zm_crypt.cpp +++ b/src/zm_crypt.cpp @@ -8,7 +8,7 @@ #if HAVE_LIBJWT #include #else -#include "jwt_cpp.h" +#include #endif #if HAVE_LIBCRYPTO From 243d028118e2d4d91bdab701d1ff5d71bc461340 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 2 Mar 2021 08:24:54 -0500 Subject: [PATCH 0057/1277] Add liblivemedia dependencies to build rtsp server --- distros/ubuntu2004/control | 2 ++ 1 file changed, 2 insertions(+) diff --git a/distros/ubuntu2004/control b/distros/ubuntu2004/control index 28c9374a1..22bd33cf2 100644 --- a/distros/ubuntu2004/control +++ b/distros/ubuntu2004/control @@ -32,6 +32,7 @@ Build-Depends: debhelper (>= 12), sphinx-doc, python3-sphinx, dh-linktree, dh-ap ,libcrypt-eksblowfish-perl ,libdata-entropy-perl ,libvncserver-dev + ,liblivemedia-dev Standards-Version: 4.5.0 Homepage: https://www.zoneminder.com/ @@ -74,6 +75,7 @@ Depends: ${shlibs:Depends}, ${misc:Depends}, ${perl:Depends} ,libcrypt-eksblowfish-perl ,libdata-entropy-perl ,libvncclient1|libvncclient0 + ,liblivemedia62|liblivemedia77 Recommends: ${misc:Recommends} ,libapache2-mod-php | php-fpm ,default-mysql-server | mariadb-server | virtual-mysql-server From 9beaf613bfda9b85b45747c8e378c05df36632ce Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 2 Mar 2021 09:05:51 -0500 Subject: [PATCH 0058/1277] Add liblivemedia as a dependency to build rtsp_server --- distros/ubuntu1604/control | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/distros/ubuntu1604/control b/distros/ubuntu1604/control index d43b50bd0..310cbac7c 100644 --- a/distros/ubuntu1604/control +++ b/distros/ubuntu1604/control @@ -33,8 +33,7 @@ Build-Depends: debhelper (>= 9), dh-systemd, python3-sphinx, apache2-dev, dh-lin ,libcrypt-eksblowfish-perl ,libdata-entropy-perl ,libvncserver-dev -# Unbundled (dh_linktree): - ,libjs-jquery + ,liblivemedia-dev Standards-Version: 3.9.8 Homepage: http://www.zoneminder.com/ Vcs-Browser: http://anonscm.debian.org/cgit/collab-maint/zoneminder.git @@ -81,6 +80,7 @@ Depends: ${shlibs:Depends}, ${misc:Depends}, ${perl:Depends} ,libcrypt-eksblowfish-perl ,libdata-entropy-perl ,libvncclient1|libvncclient0 + ,liblivemedia50|liblivemedia62 Recommends: ${misc:Recommends} ,libapache2-mod-php5 | libapache2-mod-php | php5-fpm | php-fpm ,mysql-server | mariadb-server | virtual-mysql-server From 832eabbd7962134696aab3dcfada75465e2ae7e9 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 2 Mar 2021 09:59:32 -0500 Subject: [PATCH 0059/1277] Don't both updating analysis fps unless we are doing motion detection --- src/zm_monitor.cpp | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index b6d5b834b..6fe46d91e 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -2267,7 +2267,8 @@ bool Monitor::Analyse() { // Only do these if it's a video packet. shared_data->last_read_index = snap->image_index; analysis_image_count++; - UpdateAnalysisFPS(); + if ( function == MODECT or function == MOCORD ) + UpdateAnalysisFPS(); } shared_data->last_read_time = time(nullptr); packetqueue.clearPackets(snap); @@ -2543,23 +2544,25 @@ int Monitor::Capture() { Debug(2, "Have packet stream_index:%d ?= videostream_id:(%d) q.vpktcount(%d) event?(%d) ", packet->packet.stream_index, video_stream_id, packetqueue.packet_count(video_stream_id), ( event ? 1 : 0 ) ); - if (packet->packet.stream_index == video_stream_id) { - if (video_fifo) { - if ( packet->keyframe ) { - // avcodec strips out important nals that describe the stream and - // stick them in extradata. Need to send them along with keyframes - AVStream *stream = camera->get_VideoStream(); - video_fifo->write( - static_cast(stream->codecpar->extradata), - stream->codecpar->extradata_size, - packet->pts); + if (packet->packet.stream_index == video_stream_id) { + if (video_fifo) { + if ( packet->keyframe ) { + // avcodec strips out important nals that describe the stream and + // stick them in extradata. Need to send them along with keyframes + AVStream *stream = camera->get_VideoStream(); +#if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) + video_fifo->write( + static_cast(stream->codecpar->extradata), + stream->codecpar->extradata_size, + packet->pts); +#endif + } + video_fifo->writePacket(*packet); + } + } else if (packet->packet.stream_index == audio_stream_id) { + if (audio_fifo) + audio_fifo->writePacket(*packet); } - video_fifo->writePacket(*packet); - } - } else if (packet->packet.stream_index == audio_stream_id) { - if (audio_fifo) - audio_fifo->writePacket(*packet); - } if ( (packet->packet.stream_index != video_stream_id) and ! packet->image ) { // Only queue if we have some video packets in there. Should push this logic into packetqueue From 46743ebaab9c5802d97f115ec85edb49f1280f70 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 2 Mar 2021 10:00:48 -0500 Subject: [PATCH 0060/1277] Silence compile warning --- src/zm_monitor.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 6fe46d91e..483cb35a9 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -1023,7 +1023,7 @@ bool Monitor::connect() { trigger_data->trigger_cause[0] = 0; trigger_data->trigger_text[0] = 0; trigger_data->trigger_showtext[0] = 0; - video_store_data->recording = (struct timeval){0}; + video_store_data->recording = (struct timeval){0,0}; // Uh, why nothing? Why not nullptr? snprintf(video_store_data->event_file, sizeof(video_store_data->event_file), "nothing"); video_store_data->size = sizeof(VideoStoreData); From 205ed4c510a1e8402bb63e23089a2147bc6f6cf5 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 2 Mar 2021 10:14:25 -0500 Subject: [PATCH 0061/1277] EAGAIN happens when no one is listening. Make it a debug --- src/zm_fifo.cpp | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/src/zm_fifo.cpp b/src/zm_fifo.cpp index 7f737426e..83876b740 100644 --- a/src/zm_fifo.cpp +++ b/src/zm_fifo.cpp @@ -101,7 +101,10 @@ bool Fifo::writePacket(ZMPacket &packet) { Debug(1, "Writing header ZM %u %" PRId64, packet.packet.size, packet.pts); // Going to write a brief header if ( fprintf(outfile, "ZM %u %" PRId64 "\n", packet.packet.size, packet.pts) < 0 ) { - Error("Problem during writing: %s", strerror(errno)); + if (errno != EAGAIN) + Error("Problem during writing: %s", strerror(errno)); + else + Debug(1, "Problem during writing: %s", strerror(errno)); return false; } @@ -147,7 +150,10 @@ bool Fifo::write(uint8_t *data, size_t bytes, int64_t pts) { // Going to write a brief header Debug(1, "Writing header ZM %lu %" PRId64, bytes, pts); if ( fprintf(outfile, "ZM %lu %" PRId64 "\n", bytes, pts) < 0 ) { - Error("Problem during writing: %s", strerror(errno)); + if (errno != EAGAIN) + Error("Problem during writing: %s", strerror(errno)); + else + Debug(1, "Problem during writing: %s", strerror(errno)); return false; } if (fwrite(data, bytes, 1, outfile) != 1) { From 3dc4bf265e6313d9c8afc71b671ce33fe1e73421 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 2 Mar 2021 10:15:21 -0500 Subject: [PATCH 0062/1277] Use braces --- src/zm_fifo.cpp | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/src/zm_fifo.cpp b/src/zm_fifo.cpp index 83876b740..a9ceae355 100644 --- a/src/zm_fifo.cpp +++ b/src/zm_fifo.cpp @@ -101,10 +101,11 @@ bool Fifo::writePacket(ZMPacket &packet) { Debug(1, "Writing header ZM %u %" PRId64, packet.packet.size, packet.pts); // Going to write a brief header if ( fprintf(outfile, "ZM %u %" PRId64 "\n", packet.packet.size, packet.pts) < 0 ) { - if (errno != EAGAIN) + if (errno != EAGAIN) { Error("Problem during writing: %s", strerror(errno)); - else + } else { Debug(1, "Problem during writing: %s", strerror(errno)); + } return false; } @@ -150,10 +151,11 @@ bool Fifo::write(uint8_t *data, size_t bytes, int64_t pts) { // Going to write a brief header Debug(1, "Writing header ZM %lu %" PRId64, bytes, pts); if ( fprintf(outfile, "ZM %lu %" PRId64 "\n", bytes, pts) < 0 ) { - if (errno != EAGAIN) + if (errno != EAGAIN) { Error("Problem during writing: %s", strerror(errno)); - else + } else { Debug(1, "Problem during writing: %s", strerror(errno)); + } return false; } if (fwrite(data, bytes, 1, outfile) != 1) { From 71d65685b98b20a301f8f33dba3d6c779d7258da Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 2 Mar 2021 11:12:57 -0500 Subject: [PATCH 0063/1277] Can't clear image data early. Might be needed for snapshot and pre alarm farmes --- src/zm_monitor.cpp | 5 ----- 1 file changed, 5 deletions(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 483cb35a9..44b2459be 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -2255,11 +2255,6 @@ bool Monitor::Analyse() { } #endif - if ((videowriter == PASSTHROUGH) and !savejpegs) { - // Don't need raw images anymore - delete snap->image; - snap->image = nullptr; - } // popPacket will have placed a second lock on snap, so release it here. snap->unlock(); From f8b7ec8cb9d81b6422c0c38218b4e7832044f516 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 2 Mar 2021 11:47:38 -0500 Subject: [PATCH 0064/1277] Save frames when in ALARM state so that pre-event frames get stored --- src/zm_event.cpp | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/src/zm_event.cpp b/src/zm_event.cpp index cd0ec76b4..daf1dfd39 100644 --- a/src/zm_event.cpp +++ b/src/zm_event.cpp @@ -563,9 +563,7 @@ void Event::AddFrame(Image *image, struct timeval timestamp, int score, Image *a Debug(1, "Have frame type %s from score(%d) state %d frames %d bulk frame interval %d and mod%d", frame_type_names[frame_type], score, monitor->GetState(), frames, config.bulk_frame_interval, (frames % config.bulk_frame_interval) ); - if ( score < 0 ) - score = 0; - + if (score < 0) score = 0; tot_score += score; if (image) { @@ -605,7 +603,14 @@ void Event::AddFrame(Image *image, struct timeval timestamp, int score, Image *a Debug(1, "No image"); } // end if has image - bool db_frame = ( frame_type == BULK ) or ( frame_type == ALARM ) or ( frames == 1 ) or ( score > (int)max_score ) or ( monitor_state == Monitor::ALERT ) or ( monitor_state == Monitor::PREALARM ); + bool db_frame = ( frame_type == BULK ) + or ( frame_type == ALARM ) + or ( frames == 1 ) + or ( score > (int)max_score ) + or ( monitor_state == Monitor::ALERT ) + or ( monitor_state == Monitor::ALARM ) + or ( monitor_state == Monitor::PREALARM ); + if (db_frame) { struct DeltaTimeval delta_time; From c03d9b298756b15cfe25e661e73e4b2df486d57a Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 2 Mar 2021 11:47:55 -0500 Subject: [PATCH 0065/1277] Set state before we write packets to event so that the event knows that it is alarmed --- src/zm_monitor.cpp | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 44b2459be..730d9ccd0 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -2079,6 +2079,12 @@ bool Monitor::Analyse() { ZMPacket *starting_packet = *(*start_it); event = new Event(this, *(starting_packet->timestamp), cause, noteSetMap); + shared_data->last_event_id = event->Id(); + //set up video store data + snprintf(video_store_data->event_file, sizeof(video_store_data->event_file), "%s", event->getEventFile()); + video_store_data->recording = event->StartTime(); + shared_data->state = state = ALARM; + // Write out starting packets, do not modify packetqueue it will garbage collect itself while ( *start_it != snap_it ) { event->AddPacket(starting_packet); @@ -2096,12 +2102,6 @@ bool Monitor::Analyse() { delete start_it; start_it = nullptr; - shared_data->last_event_id = event->Id(); - //set up video store data - snprintf(video_store_data->event_file, sizeof(video_store_data->event_file), "%s", event->getEventFile()); - video_store_data->recording = event->StartTime(); - shared_data->state = state = ALARM; - Info("%s: %03d - Opening new event %" PRIu64 ", alarm start", name, analysis_image_count, event->Id()); } // end if no event, so start it if ( alarm_frame_count ) { From f35a1c70c086d56e453ce9425a02c5541eccae0b Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 2 Mar 2021 12:07:51 -0500 Subject: [PATCH 0066/1277] fix build with old avcodec for xenial --- src/zm_monitor.cpp | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 730d9ccd0..821cffa74 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -2544,8 +2544,8 @@ int Monitor::Capture() { if ( packet->keyframe ) { // avcodec strips out important nals that describe the stream and // stick them in extradata. Need to send them along with keyframes - AVStream *stream = camera->get_VideoStream(); #if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) + AVStream *stream = camera->get_VideoStream(); video_fifo->write( static_cast(stream->codecpar->extradata), stream->codecpar->extradata_size, @@ -2987,14 +2987,24 @@ int Monitor::PrimeCapture() { snprintf(shared_data->video_fifo_path, sizeof(shared_data->video_fifo_path)-1, "%s/video_fifo_%d.%s", staticConfig.PATH_SOCKS.c_str(), id, - avcodec_get_name(videoStream->codecpar->codec_id)); +#if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) + avcodec_get_name(videoStream->codecpar->codec_id) +#else + avcodec_get_name(videoStream->codec->codec_id) +#endif + ); video_fifo = new Fifo(shared_data->video_fifo_path, true); } if (record_audio and (audio_stream_id >= 0)) { AVStream *audioStream = camera->get_AudioStream(); snprintf(shared_data->audio_fifo_path, sizeof(shared_data->audio_fifo_path)-1, "%s/video_fifo_%d.%s", staticConfig.PATH_SOCKS.c_str(), id, - avcodec_get_name(audioStream->codecpar->codec_id)); +#if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) + avcodec_get_name(audioStream->codecpar->codec_id) +#else + avcodec_get_name(audioStream->codec->codec_id) +#endif + ); audio_fifo = new Fifo(shared_data->audio_fifo_path, true); } } // end if rtsp_server From be99d097729f705ceeac5eac635f107cc2906f0c Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 26 Jan 2021 22:05:13 -0500 Subject: [PATCH 0067/1277] Accept liblivemedia64 for buster --- distros/ubuntu2004/control | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/distros/ubuntu2004/control b/distros/ubuntu2004/control index 22bd33cf2..67554b8ea 100644 --- a/distros/ubuntu2004/control +++ b/distros/ubuntu2004/control @@ -75,7 +75,7 @@ Depends: ${shlibs:Depends}, ${misc:Depends}, ${perl:Depends} ,libcrypt-eksblowfish-perl ,libdata-entropy-perl ,libvncclient1|libvncclient0 - ,liblivemedia62|liblivemedia77 + ,liblivemedia62|liblivemedia64|liblivemedia77 Recommends: ${misc:Recommends} ,libapache2-mod-php | php-fpm ,default-mysql-server | mariadb-server | virtual-mysql-server From e6358290b4167c8ab39339efb82dccd979efc0cf Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 2 Feb 2021 17:30:42 -0500 Subject: [PATCH 0068/1277] this=>self --- scripts/ZoneMinder/lib/ZoneMinder/Control/onvif.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ZoneMinder/lib/ZoneMinder/Control/onvif.pm b/scripts/ZoneMinder/lib/ZoneMinder/Control/onvif.pm index f69441aff..8c1f2114e 100644 --- a/scripts/ZoneMinder/lib/ZoneMinder/Control/onvif.pm +++ b/scripts/ZoneMinder/lib/ZoneMinder/Control/onvif.pm @@ -65,7 +65,7 @@ sub sendCmd { my $self = shift; my $cmd = shift; my $result = undef; - $this->printMsg($cmd, 'Tx'); + $self->printMsg($cmd, 'Tx'); my $req = HTTP::Request->new(GET=>'http://'.$self->{Monitor}->{ControlAddress}.'/'.$cmd); my $res = $self->{ua}->request($req); From 4cfe777e9399cd1f5056cad096ee0b91e4eff2e7 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 16 Feb 2021 11:16:06 -0500 Subject: [PATCH 0069/1277] Fix path to Monitor.php --- web/includes/actions/settings.php | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/web/includes/actions/settings.php b/web/includes/actions/settings.php index c33f3a54c..a4aa1b489 100644 --- a/web/includes/actions/settings.php +++ b/web/includes/actions/settings.php @@ -29,7 +29,7 @@ if ( ! canView('Control', $_REQUEST['mid']) ) { return; } -require_once('Monitor.php'); +require_once('includes/Monitor.php'); $mid = validInt($_REQUEST['mid']); if ( $action == 'settings' ) { $args = ' -m ' . escapeshellarg($mid); From 36f11158df69e991b64531f753457b061c8c2e9c Mon Sep 17 00:00:00 2001 From: Arek Kossendowski Date: Thu, 25 Feb 2021 22:37:34 +0000 Subject: [PATCH 0070/1277] Replaced the onvif control module with a modified version of Netcat. The original onvif module was NOT at all ONVIF protocol implementation. This one pretty much copies the Netcat module but with a bit of clarity to the XML that is sent to the camera but most importantly allows specification of full onvif URL including authentication and control URI through the Control Address field. Parsing of the url is done through a combination of sane defaults and the URI module. --- db/zm_create.sql.in | 2 +- .../lib/ZoneMinder/Control/onvif.pm | 629 +++++++++++++----- 2 files changed, 463 insertions(+), 168 deletions(-) diff --git a/db/zm_create.sql.in b/db/zm_create.sql.in index 7a87f2403..fe6bc3e98 100644 --- a/db/zm_create.sql.in +++ b/db/zm_create.sql.in @@ -893,7 +893,7 @@ INSERT INTO Controls VALUES (NULL,'Loftek Sentinel','Remote','LoftekSentinel',0, INSERT INTO Controls VALUES (NULL,'Toshiba IK-WB11A','Remote','Toshiba_IK_WB11A',0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,10,0,1,1,0,1,0,1,0,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0); INSERT INTO Controls VALUES (NULL,'WanscamPT','Remote','Wanscam',1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,16,0,0,0,0,0,1,16,1,1,1,1,0,0,0,1,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0); INSERT INTO Controls VALUES (NULL,'3S Domo N5071', 'Remote', '3S', 0, 0, 1, 0,1, 0, 1, 1, 0, 0, 9999, 0, 9999, 0, 0, 0, 1, 1, 1, 1, 0, 0, 9999, 20, 9999, 0, 0, 0, 1, 1, 1, 1, 0, 0, 9999, 1, 9999, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 64, 1, 0, 1, 1, 0, 0, 0, 0, 1, -180, 180, 40, 100, 1, 40, 100, 0, 0, 1, -180, 180, 40, 100, 1, 40, 100, 0, 0, 0, 0); -INSERT INTO Controls VALUES (NULL,'ONVIF Camera','Ffmpeg','onvif',0,0,1,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,255,16,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,6,1,1,0,0,0,1,10,0,1,1,1,0,0,0,1,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0); +INSERT INTO Controls VALUES (NULL,'ONVIF Camera','Ffmpeg','onvif',0,0,1,1,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,255,16,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,6,1,1,0,0,0,1,10,1,1,1,1,0,0,0,1,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0); INSERT INTO `Controls` VALUES (NULL,'Foscam 9831W','Ffmpeg','FI9831W',0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,16,1,1,1,1,0,0,0,1,1,0,360,0,360,1,0,4,0,0,1,0,90,0,90,0,0,0,0,0,0,0); INSERT INTO `Controls` VALUES (NULL,'Foscam FI8918W','Ffmpeg','FI8918W',0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,8,0,1,1,1,0,0,0,1,1,0,360,0,360,1,0,4,0,0,1,0,90,0,90,1,0,4,0,0,0,0); INSERT INTO `Controls` VALUES (NULL,'SunEyes SP-P1802SWPTZ','Libvlc','SPP1802SWPTZ',0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,8,0,1,1,0,0,0,0,1,1,0,0,0,0,1,0,64,0,0,1,0,0,0,0,1,0,64,0,0,0,0); diff --git a/scripts/ZoneMinder/lib/ZoneMinder/Control/onvif.pm b/scripts/ZoneMinder/lib/ZoneMinder/Control/onvif.pm index 8c1f2114e..4ae2ba1fc 100644 --- a/scripts/ZoneMinder/lib/ZoneMinder/Control/onvif.pm +++ b/scripts/ZoneMinder/lib/ZoneMinder/Control/onvif.pm @@ -1,7 +1,7 @@ # ========================================================================== # -# ZoneMinder ONVIF Control Protocol Module -# Copyright (C) Jan M. Hochstein +# ZoneMinder ONVIF Control Protocol Module, $Date: 2021-02-25 22:07:00 +0000 (Thu, 25 Feb 2021) $, $Revision: 0001 $ +# Based on the Netcat onvif script by Andrew Bauer (knnniggett@users.sourceforge.net) # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License @@ -19,28 +19,55 @@ # # ========================================================================== # -# This module contains the implementation of the ONVIF device control protocol +# This module contains the implementation of onvif protocol # package ZoneMinder::Control::onvif; use 5.006; use strict; use warnings; +use MIME::Base64; +use Digest::SHA; +use DateTime; +use URI; +use Data::Dumper; +require ZoneMinder::Base; require ZoneMinder::Control; our @ISA = qw(ZoneMinder::Control); our %CamParams = (); +our ($profileToken, $address, $port, %identity); +my ( $controlUri, $scheme ); + # ========================================================================== # -# ONVIF Control Protocol +# This script sends ONVIF compliant commands and may work with other cameras # -# On ControlAddress use the format : -# USERNAME:PASSWORD@ADDRESS:PORT -# eg : admin:@10.1.2.1:80 -# zoneminder:zonepass@10.0.100.1:40000 +# Configuration options (Source->Control tab) +# - Control Type: ONVIF +# - Control Device: prof0 - this is dependant on camera. It maybe required to sniff the traffic using Wireshark to find this out. If left empty value of "000" will be used. +# - Control Address: ://[:@][:port][control_uri] +# - Auto Stop Timeout: 1.00 - how long shold the camera move for when move command is issued. Value of 1.00 means 1s. +# - Track Motion: NOT IMPLEMENTED - this suppose to be a feature for automatic camera scrolling (moving). +# - Track Delay: NOT IMPLEMENTED +# - Return Location: Home|Preset 1 - NOT IMPLEMENTED +# +# Absolute minimum required supported "Control Address" would be: +# - 192.168.1.199 +# This will use the following defaults: +# - port: 80 +# - Control Device: 000 +# - Control URI: /onvif/device_control +# - No authentication +# - No Auto Stop Timeout (on movement command the camera will keep moving until it reaches it's edge) +# +# Example Control Address values: +# - http://user:password@192.168.1.199:888/onvif/device_control :Connect to camera at IP: 192.168.1.199 on port 888 with "username" and "password" credentials using /onvif/device_control URI +# - user:password@192.168.1.199 :Connect to camera at IP: 192.168.1.199 on default port 80 with "username" and "password" credentials using default /onvif/device_control URI +# - 192.168.1.199 :Connect to camera at IP: 192.168.1.199 without any authentication and use the default /onvif/device_control URI over HTTP. # # ========================================================================== @@ -54,6 +81,11 @@ sub open { $self->loadMonitor(); + $profileToken = $self->{Monitor}->{ControlDevice}; + if ($profileToken eq '') { $profileToken = '000'; } + + parseControlAddress($self->{Monitor}->{ControlAddress}); + use LWP::UserAgent; $self->{ua} = LWP::UserAgent->new; $self->{ua}->agent('ZoneMinder Control Agent/'.ZoneMinder::Base::ZM_VERSION); @@ -61,39 +93,140 @@ sub open { $self->{state} = 'open'; } +sub parseControlAddress { + + my $controlAddress = shift; + + #make sure url start with a scheme + if ( $controlAddress !~ m'^https?://') { + $scheme = "http"; + $controlAddress = $scheme."://".$controlAddress; + } + + my $url = URI->new($controlAddress); + + #set the scheme + $scheme = $url->scheme; + + #If we have authinfo + if ($url->userinfo){ + my ($username , $password) = split /:/, $url->userinfo; + %identity = (username => $username, password => $password); + } + + #If we have no explicitly defined port + if (!$url->port){ + $port = $url->default_port; + } else { + $port = $url->port; + } + + if (!$url->path){ + $controlUri = "/onvif/device_control"; + } else { + $controlUri = $url->path; + } + + $address = $url->host; +} + +sub digestBase64 { + my ($nonce, $date, $password) = @_; + my $shaGenerator = Digest::SHA->new(1); + $shaGenerator->add($nonce . $date . $password); + return encode_base64($shaGenerator->digest, ''); +} + +sub authentificationHeader { + my ($username, $password) = @_; + my @set = ('0' ..'9', 'A' .. 'Z', 'a' .. 'z'); + my $nonce = join '' => map $set[rand @set], 1 .. 20; + + my $nonceBase64 = encode_base64($nonce, ''); + my $currentDate = DateTime->now()->iso8601().'Z'; + + return ' + + + + ' . $username . ' + ' . digestBase64($nonce, $currentDate, $password) . ' + ' . $nonceBase64 . ' + ' . $currentDate . ' + + +'; +} + sub sendCmd { my $self = shift; my $cmd = shift; + my $msg_body = shift; + my $content_type = shift; my $result = undef; + + my $msg = ' + '. + ((%identity) ? authentificationHeader($identity{username}, $identity{password}) : '') . + $msg_body . ' + '; + $self->printMsg($cmd, 'Tx'); - my $req = HTTP::Request->new(GET=>'http://'.$self->{Monitor}->{ControlAddress}.'/'.$cmd); + my $server_endpoint = $scheme.'://'.$address.':'.$port.$controlUri; + my $req = HTTP::Request->new(POST => $server_endpoint); + $req->header('content-type' => $content_type); + $req->header('Host' => $address . ':' . $port); + $req->header('content-length' => length($msg)); + $req->header('accept-encoding' => 'gzip, deflate'); + $req->header('connection' => 'Close'); + $req->content($msg); + my $res = $self->{ua}->request($req); if ( $res->is_success ) { $result = !undef; } else { - Error("Error check failed:'".$res->status_line()."'" ); + Error("After sending PTZ command, camera returned the following error:'".$res->status_line()."'\nMSG:$msg\nResponse:".$res->content); } - return $result; } sub getCamParams { my $self = shift; + my $msg = ' + + + + 000 + + + '; + + my $server_endpoint = $scheme.'://'.$address.':'.$port.'/onvif/imaging'; + + my $req = HTTP::Request->new(POST => $server_endpoint); + $req->header('content-type' => 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/imaging/wsdl/GetImagingSettings"'); + $req->header('Host' => $address . ':' . $port); + $req->header('content-length' => length($msg)); + $req->header('accept-encoding' => 'gzip, deflate'); + $req->header('connection' => 'Close'); + $req->content($msg); - my $req = HTTP::Request->new(GET=>'http://'.$self->{Monitor}->{ControlAddress}.'/get_camera_params.cgi'); my $res = $self->{ua}->request($req); if ( $res->is_success ) { - # Parse results setting values in %FCParams + # We should really use an xml or soap library to parse the xml tags my $content = $res->decoded_content; - while ($content =~ s/var\s+([^=]+)=([^;]+);//ms) { + if ( $content =~ /.*(.+)<\/tt:Brightness>.*/ ) { + $CamParams{$1} = $2; + } + if ( $content =~ /.*(.+)<\/tt:Contrast>.*/ ) { $CamParams{$1} = $2; } } else { - Error("Error check failed:'".$res->status_line()."'"); + Error("Unable to retrieve camera image settings:'".$res->status_line()."'"); } } @@ -101,256 +234,418 @@ sub getCamParams { #This makes use of the ZoneMinder Auto Stop Timeout on the Control Tab sub autoStop { my $self = shift; - my $stop_command = shift; my $autostop = shift; - if ( $stop_command && $autostop ) { + + if ( $autostop ) { Debug('Auto Stop'); + my $cmd = $controlUri; + my $msg_body = ' + + + '.$profileToken.' + + true + + + false + + + '; + + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/ptz/wsdl/ContinuousMove"'; usleep($autostop); - my $cmd = 'decoder_control.cgi?command='.$stop_command; - $self->sendCmd($cmd); + $self->sendCmd($cmd, $msg_body, $content_type); } } -# Reset the Camera -sub reset { +# Reboot +sub reboot { + Debug('Camera reboot'); my $self = shift; + my $cmd = ''; + my $msg_body = << "END_MESSAGE"; + + + +END_MESSAGE + + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver10/device/wsdl/SystemReboot"'; + $self->sendCmd($cmd, $msg_body, $content_type); + +} + +# Reset(Reboot) the Camera +sub reset { Debug('Camera Reset'); - my $cmd = 'reboot.cgi?'; - $self->sendCmd($cmd); + my $self = shift; + my $cmd = ''; + my $msg_body = << "END_MESSAGE"; + + + +END_MESSAGE + + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver10/device/wsdl/SystemReboot"'; + $self->sendCmd($cmd, $msg_body, $content_type); +} + + +sub moveCamera { + + my $type = shift; + my $x = shift; + my $y = shift; + my $msg_move_body = ""; + + if ( $type == "move" ){ + $msg_move_body = ' + + + '.$profileToken.' + + + + + '; + + } elsif ( $type == "zoom" ) { + $msg_move_body = ' + + + '.$profileToken.' + + + + + '; + } + + return $msg_move_body; + } #Up Arrow -sub moveConUp -{ +sub moveConUp { + Debug('Move Up'); my $self = shift; - my $stop_command = "1"; - Debug( "Move Up" ); - my $cmd = "decoder_control.cgi?command=0"; - $self->sendCmd( $cmd ); - $self->autoStop( $stop_command, $self->{Monitor}->{AutoStopTimeout} ); + my $cmd = $controlUri; + my $msg_body = moveCamera("move", "0","0.5"); + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/ptz/wsdl/ContinuousMove"'; + $self->sendCmd($cmd, $msg_body, $content_type); + $self->autoStop($self->{Monitor}->{AutoStopTimeout}); } + #Down Arrow -sub moveConDown -{ +sub moveConDown { + Debug('Move Down'); my $self = shift; - my $stop_command = "3"; - Debug( "Move Down" ); - my $cmd = "decoder_control.cgi?command=2"; - $self->sendCmd( $cmd ); - $self->autoStop( $stop_command, $self->{Monitor}->{AutoStopTimeout} ); + my $cmd = $controlUri; + my $msg_body = moveCamera("move","0","-0.5"); + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/ptz/wsdl/ContinuousMove"'; + $self->sendCmd($cmd, $msg_body, $content_type); + $self->autoStop($self->{Monitor}->{AutoStopTimeout}); } #Left Arrow -sub moveConLeft -{ +sub moveConLeft { + Debug('Move Left'); my $self = shift; - my $stop_command = "5"; - Debug( "Move Left" ); - my $cmd = "decoder_control.cgi?command=4"; - $self->sendCmd( $cmd ); - $self->autoStop( $stop_command, $self->{Monitor}->{AutoStopTimeout} ); + my $cmd = $controlUri; + my $msg_body = moveCamera("move","-0.49","0"); + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/ptz/wsdl/ContinuousMove"'; + $self->sendCmd($cmd, $msg_body, $content_type); + $self->autoStop($self->{Monitor}->{AutoStopTimeout}); } #Right Arrow -sub moveConRight -{ +sub moveConRight { + Debug('Move Right'); my $self = shift; - my $stop_command = "7"; - Debug( "Move Right" ); - my $cmd = "decoder_control.cgi?command=6"; - $self->sendCmd( $cmd ); - $self->autoStop( $stop_command, $self->{Monitor}->{AutoStopTimeout} ); + my $cmd = $controlUri; + my $msg_body = moveCamera("move","0.49","0"); + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/ptz/wsdl/ContinuousMove"'; + $self->sendCmd($cmd, $msg_body, $content_type); + $self->autoStop($self->{Monitor}->{AutoStopTimeout}); } #Zoom In -sub zoomConTele -{ +sub zoomConTele { + Debug('Zoom Tele'); my $self = shift; - my $stop_command = "17"; - Debug( "Zoom Tele" ); - my $cmd = "decoder_control.cgi?command=18"; - $self->sendCmd( $cmd ); - $self->autoStop( $stop_command, $self->{Monitor}->{AutoStopTimeout} ); + my $cmd = $controlUri; + my $msg_body = moveCamera("zoom","0.49","0"); + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/ptz/wsdl/ContinuousMove"'; + $self->sendCmd($cmd, $msg_body, $content_type); + $self->autoStop($self->{Monitor}->{AutoStopTimeout}); } #Zoom Out -sub zoomConWide -{ +sub zoomConWide { + Debug('Zoom Wide'); my $self = shift; - my $stop_command = "19"; - Debug( "Zoom Wide" ); - my $cmd = "decoder_control.cgi?command=16"; - $self->sendCmd( $cmd ); - $self->autoStop( $stop_command, $self->{Monitor}->{AutoStopTimeout} ); + my $cmd = $controlUri; + my $msg_body = moveCamera("zoom","-0.49","0"); + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/ptz/wsdl/ContinuousMove"'; + $self->sendCmd($cmd, $msg_body, $content_type); + $self->autoStop($self->{Monitor}->{AutoStopTimeout}); } #Diagonally Up Right Arrow #This camera does not have builtin diagonal commands so we emulate them -sub moveConUpRight -{ +sub moveConUpRight { + Debug('Move Diagonally Up Right'); my $self = shift; - Debug( "Move Diagonally Up Right" ); - $self->moveConUp( ); - $self->moveConRight( ); + my $cmd = $controlUri; + my $msg_body = moveCamera("move","0.5","0.5"); + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/ptz/wsdl/ContinuousMove"'; + $self->sendCmd($cmd, $msg_body, $content_type); + $self->autoStop($self->{Monitor}->{AutoStopTimeout}); } #Diagonally Down Right Arrow #This camera does not have builtin diagonal commands so we emulate them -sub moveConDownRight -{ +sub moveConDownRight { + Debug('Move Diagonally Down Right'); my $self = shift; - Debug( "Move Diagonally Down Right" ); - $self->moveConDown( ); - $self->moveConRight( ); + my $cmd = $controlUri; + my $msg_body = moveCamera("move","0.5","-0.5"); + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/ptz/wsdl/ContinuousMove"'; + $self->sendCmd($cmd, $msg_body, $content_type); + $self->autoStop($self->{Monitor}->{AutoStopTimeout}); } #Diagonally Up Left Arrow #This camera does not have builtin diagonal commands so we emulate them -sub moveConUpLeft -{ +sub moveConUpLeft { + Debug('Move Diagonally Up Left'); my $self = shift; - Debug( "Move Diagonally Up Left" ); - $self->moveConUp( ); - $self->moveConLeft( ); + my $cmd = $controlUri; + my $msg_body = moveCamera("move","-0.5","0.5"); + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/ptz/wsdl/ContinuousMove"'; + $self->sendCmd($cmd, $msg_body, $content_type); + $self->autoStop($self->{Monitor}->{AutoStopTimeout}); } #Diagonally Down Left Arrow #This camera does not have builtin diagonal commands so we emulate them -sub moveConDownLeft -{ +sub moveConDownLeft { + Debug('Move Diagonally Down Left'); my $self = shift; - Debug( "Move Diagonally Down Left" ); - $self->moveConDown( ); - $self->moveConLeft( ); + my $cmd = $controlUri; + my $msg_body = moveCamera("move","-0.5","-0.5"); + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/ptz/wsdl/ContinuousMove"'; + $self->sendCmd($cmd, $msg_body, $content_type); + $self->autoStop($self->{Monitor}->{AutoStopTimeout}); } #Stop -sub moveStop -{ +sub moveStop { + Debug('Move Stop'); my $self = shift; - Debug( "Move Stop" ); - my $cmd = "decoder_control.cgi?command=1"; - $self->sendCmd( $cmd ); + my $cmd = $controlUri; + my $msg_body = ' + + + '.$profileToken.' + true + false + + '; + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/ptz/wsdl/ContinuousMove"'; + $self->sendCmd($cmd, $msg_body, $content_type); } #Set Camera Preset -#Presets must be translated into values internal to the camera -#Those values are: 30,32,34,36,38,40,42,44 for presets 1-8 respectively -sub presetSet -{ +sub presetSet { my $self = shift; my $params = shift; - my $preset = $self->getParam( $params, 'preset' ); - Debug( "Set Preset $preset" ); - - if (( $preset >= 1 ) && ( $preset <= 8 )) { - my $cmd = "decoder_control.cgi?command=".(($preset*2) + 28); - $self->sendCmd( $cmd ); - } + my $preset = $self->getParam($params, 'preset'); + Debug("Set Preset $preset"); + my $cmd = $controlUri; + my $msg_body =' + + + ' . $profileToken . ' + '.$preset.' + + '; + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/ptz/wsdl/SetPreset"'; + $self->sendCmd($cmd, $msg_body, $content_type); } #Recall Camera Preset -#Presets must be translated into values internal to the camera -#Those values are: 31,33,35,37,39,41,43,45 for presets 1-8 respectively -sub presetGoto -{ +sub presetGoto { my $self = shift; my $params = shift; - my $preset = $self->getParam( $params, 'preset' ); - Debug( "Goto Preset $preset" ); + my $preset = $self->getParam($params, 'preset'); - if (( $preset >= 1 ) && ( $preset <= 8 )) { - my $cmd = "decoder_control.cgi?command=".(($preset*2) + 29); - $self->sendCmd( $cmd ); - } - - if ( $preset == 9 ) { - $self->horizontalPatrol(); - } - - if ( $preset == 10 ) { - $self->horizontalPatrolStop(); - } + Debug("Goto Preset $preset"); + my $cmd = $controlUri; + my $msg_body =' + + + ' . $profileToken . ' + '.$preset.' + + '; + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/ptz/wsdl/GotoPreset"'; + $self->sendCmd( $cmd, $msg_body, $content_type ); } -#Horizontal Patrol - Vertical Patrols are not supported -sub horizontalPatrol -{ +#Recall Camera Preset +sub presetHome { my $self = shift; - Debug( "Horizontal Patrol" ); - my $cmd = "decoder_control.cgi?command=20"; - $self->sendCmd( $cmd ); + my $params = shift; + + Debug("Goto Home preset"); + my $cmd = $controlUri; + my $msg_body =' + + + ' . $profileToken . ' + + '; + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/ptz/wsdl/GotoPreset"'; + $self->sendCmd( $cmd, $msg_body, $content_type ); +} + +#Horizontal Patrol +#To be determined if this camera supports this feature +sub horizontalPatrol { + Debug('Horizontal Patrol'); + my $self = shift; + my $cmd = ''; + my $msg =''; + my $content_type = ''; + # $self->sendCmd( $cmd, $msg, $content_type ); + Error('PTZ Command not implemented in control script.'); } #Horizontal Patrol Stop -sub horizontalPatrolStop -{ +#To be determined if this camera supports this feature +sub horizontalPatrolStop { + Debug('Horizontal Patrol Stop'); my $self = shift; - Debug( "Horizontal Patrol Stop" ); - my $cmd = "decoder_control.cgi?command=21"; - $self->sendCmd( $cmd ); + my $cmd = ''; + my $msg =''; + my $content_type = ''; + # $self->sendCmd( $cmd, $msg, $content_type ); + Error('PTZ Command not implemented in control script.'); } # Increase Brightness -sub irisAbsOpen -{ +sub irisAbsOpen { + Debug("Iris $CamParams{Brightness}"); my $self = shift; my $params = shift; - $self->getCamParams() unless($CamParams{'brightness'}); - my $step = $self->getParam( $params, 'step' ); + $self->getCamParams() unless($CamParams{Brightness}); + my $step = $self->getParam($params, 'step'); + my $max = 100; - $CamParams{'brightness'} += $step; - $CamParams{'brightness'} = 255 if ($CamParams{'brightness'} > 255); - Debug( "Iris $CamParams{'brightness'}" ); - my $cmd = "camera_control.cgi?param=1&value=".$CamParams{'brightness'}; - $self->sendCmd( $cmd ); + $CamParams{Brightness} += $step; + $CamParams{Brightness} = $max if ($CamParams{Brightness} > $max); + + my $cmd = 'onvif/imaging'; + my $msg_body =' + + + 000 + + '.$CamParams{Brightness}.' + + true + + '; + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/imaging/wsdl/SetImagingSettings"'; + $self->sendCmd($cmd, $msg_body, $content_type); } # Decrease Brightness -sub irisAbsClose -{ +sub irisAbsClose { + Debug("Iris $CamParams{Brightness}"); my $self = shift; my $params = shift; - $self->getCamParams() unless($CamParams{'brightness'}); - my $step = $self->getParam( $params, 'step' ); + $self->getCamParams() unless($CamParams{brightness}); + my $step = $self->getParam($params, 'step'); + my $min = 0; - $CamParams{'brightness'} -= $step; - $CamParams{'brightness'} = 0 if ($CamParams{'brightness'} < 0); - Debug( "Iris $CamParams{'brightness'}" ); - my $cmd = "camera_control.cgi?param=1&value=".$CamParams{'brightness'}; - $self->sendCmd( $cmd ); + $CamParams{Brightness} -= $step; + $CamParams{Brightness} = $min if ($CamParams{Brightness} < $min); + + my $cmd = 'onvif/imaging'; + my $msg_body =' + + + 000 + + '.$CamParams{Brightness}.' + + true + + '; + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/imaging/wsdl/SetImagingSettings"'; + $self->sendCmd($cmd, $msg_body, $content_type); } # Increase Contrast -sub whiteAbsIn -{ +sub whiteAbsIn { + Debug("Iris $CamParams{Contrast}"); my $self = shift; my $params = shift; - $self->getCamParams() unless($CamParams{'contrast'}); - my $step = $self->getParam( $params, 'step' ); + $self->getCamParams() unless($CamParams{Contrast}); + my $step = $self->getParam($params, 'step'); + my $max = 100; - $CamParams{'contrast'} += $step; - $CamParams{'contrast'} = 6 if ($CamParams{'contrast'} > 6); - Debug( "Iris $CamParams{'contrast'}" ); - my $cmd = "camera_control.cgi?param=2&value=".$CamParams{'contrast'}; - $self->sendCmd( $cmd ); + $CamParams{Contrast} += $step; + $CamParams{Contrast} = $max if ($CamParams{Contrast} > $max); + + my $cmd = 'onvif/imaging'; + my $msg_body =' + + + 000 + + '.$CamParams{Contrast}.' + + true + + '; + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/imaging/wsdl/SetImagingSettings"'; + $self->sendCmd($cmd, $msg_body, $content_type); } # Decrease Contrast -sub whiteAbsOut -{ +sub whiteAbsOut { + Debug("Iris $CamParams{Contrast}"); my $self = shift; my $params = shift; - $self->getCamParams() unless($CamParams{'contrast'}); - my $step = $self->getParam( $params, 'step' ); + $self->getCamParams() unless($CamParams{Contrast}); + my $step = $self->getParam($params, 'step'); + my $min = 0; - $CamParams{'contrast'} -= $step; - $CamParams{'contrast'} = 0 if ($CamParams{'contrast'} < 0); - Debug( "Iris $CamParams{'contrast'}" ); - my $cmd = "camera_control.cgi?param=2&value=".$CamParams{'contrast'}; - $self->sendCmd( $cmd ); + $CamParams{Contrast} -= $step; + $CamParams{Contrast} = $min if ($CamParams{Contrast} < $min); + + my $cmd = 'onvif/imaging'; + my $msg_body =' + + + 000 + + '.$CamParams{Contrast}.' + + true + + '; + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/imaging/wsdl/SetImagingSettings"'; + $self->sendCmd($cmd, $msg_body, $content_type); } 1; - +__END__ From 0eab9987fce9924b152631b90e473d2a2593662e Mon Sep 17 00:00:00 2001 From: Arek Kossendowski Date: Fri, 26 Feb 2021 00:11:29 +0000 Subject: [PATCH 0071/1277] Just updated the script with moveMap and moveRel functions. --- db/zm_create.sql.in | 36 ++++----- .../lib/ZoneMinder/Control/onvif.pm | 77 +++++++++++++++---- 2 files changed, 79 insertions(+), 34 deletions(-) diff --git a/db/zm_create.sql.in b/db/zm_create.sql.in index fe6bc3e98..dfd32f310 100644 --- a/db/zm_create.sql.in +++ b/db/zm_create.sql.in @@ -876,24 +876,24 @@ VALUES ( -- -- Add in some sample control protocol definitions -- -INSERT INTO Controls VALUES (NULL,'Pelco-D','Local','PelcoD',1,1,0,0,1,1,0,0,1,NULL,NULL,NULL,NULL,1,0,3,1,1,0,0,1,NULL,NULL,NULL,NULL,0,NULL,NULL,1,1,0,1,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,1,0,1,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,20,1,1,1,1,0,0,0,1,1,NULL,NULL,NULL,NULL,1,0,63,1,254,1,NULL,NULL,NULL,NULL,1,0,63,1,254,0,0); -INSERT INTO Controls VALUES (NULL,'Pelco-P','Local','PelcoP',1,1,0,0,1,1,0,0,1,NULL,NULL,NULL,NULL,1,0,3,1,1,0,0,1,NULL,NULL,NULL,NULL,0,NULL,NULL,1,1,0,1,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,1,0,1,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,20,1,1,1,1,0,0,0,1,1,NULL,NULL,NULL,NULL,1,0,63,1,254,1,NULL,NULL,NULL,NULL,1,0,63,1,254,0,0); -INSERT INTO Controls VALUES (NULL,'Sony VISCA','Local','Visca',1,1,0,0,1,0,0,0,1,0,16384,10,4000,1,1,6,1,1,1,0,1,0,1536,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,3,1,1,1,1,0,1,1,0,1,-15578,15578,100,10000,1,1,50,1,254,1,-7789,7789,100,5000,1,1,50,1,254,0,0); -INSERT INTO Controls VALUES (NULL,'Axis API v2','Remote','AxisV2',0,0,0,0,1,0,0,1,0,0,9999,10,2500,0,NULL,NULL,1,1,0,1,0,0,9999,10,2500,0,NULL,NULL,1,1,0,1,0,0,9999,10,2500,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,12,1,1,1,1,1,0,1,0,1,-360,360,1,90,0,NULL,NULL,0,NULL,1,-360,360,1,90,0,NULL,NULL,0,NULL,0,0); -INSERT INTO Controls VALUES (NULL,'Panasonic IP','Remote','PanasonicIP',0,0,0,0,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,8,1,1,1,0,1,0,0,1,1,NULL,NULL,NULL,NULL,0,NULL,NULL,0,NULL,1,NULL,NULL,NULL,NULL,0,NULL,NULL,0,NULL,0,0); -INSERT INTO Controls VALUES (NULL,'Neu-Fusion NCS370','Remote','Ncs370',0,0,0,0,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,24,1,0,1,1,0,0,0,1,1,NULL,NULL,NULL,NULL,0,NULL,NULL,0,NULL,1,NULL,NULL,NULL,NULL,0,NULL,NULL,0,NULL,0,0); -INSERT INTO Controls VALUES (NULL,'AirLink SkyIPCam 7xx','Remote','SkyIPCam7xx',0,0,1,0,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,8,1,1,1,0,1,0,1,0,1,NULL,NULL,NULL,NULL,0,NULL,NULL,0,NULL,1,NULL,NULL,NULL,NULL,0,NULL,NULL,0,NULL,0,0); -INSERT INTO Controls VALUES (NULL,'Pelco-D','Ffmpeg','PelcoD',1,1,0,0,1,1,0,0,1,NULL,NULL,NULL,NULL,1,0,3,1,1,0,0,1,NULL,NULL,NULL,NULL,0,NULL,NULL,1,1,0,1,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,1,0,1,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,20,1,1,1,1,0,0,0,1,1,NULL,NULL,NULL,NULL,1,0,63,1,254,1,NULL,NULL,NULL,NULL,1,0,63,1,254,0,0); -INSERT INTO Controls VALUES (NULL,'Pelco-P','Ffmpeg','PelcoP',1,1,0,0,1,1,0,0,1,NULL,NULL,NULL,NULL,1,0,3,1,1,0,0,1,NULL,NULL,NULL,NULL,0,NULL,NULL,1,1,0,1,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,1,0,1,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,20,1,1,1,1,0,0,0,1,1,NULL,NULL,NULL,NULL,1,0,63,1,254,1,NULL,NULL,NULL,NULL,1,0,63,1,254,0,0); -INSERT INTO Controls VALUES (NULL,'Foscam FI8620','Ffmpeg','FI8620_Y2k',0,0,0,0,1,0,0,0,1,1,10,1,10,1,1,63,1,1,0,0,1,1,63,1,63,1,1,63,1,1,0,0,1,0,0,0,0,1,0,255,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,1,0,0,0,0,1,0,255,1,8,0,1,1,1,0,0,0,1,1,1,360,1,360,1,1,63,0,0,1,1,90,1,90,1,1,63,0,0,0,0); -INSERT INTO Controls VALUES (NULL,'Foscam FI8608W','Ffmpeg','FI8608W_Y2k',1,0,1,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,1,0,0,0,0,1,0,255,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,1,0,0,0,0,1,0,255,1,8,0,1,1,1,0,0,0,1,1,0,0,0,0,1,1,128,0,0,1,0,0,0,0,1,1,128,0,0,0,0); -INSERT INTO Controls VALUES (NULL,'Foscam FI8908W','Remote','FI8908W',0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,1,1,0,0,0,1,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0); -INSERT INTO Controls VALUES (NULL,'Foscam FI9821W','Ffmpeg','FI9821W_Y2k',1,0,1,0,1,0,0,0,1,0,0,0,0,0,0,0,1,1,0,0,1,0,0,0,0,1,0,100,1,1,0,0,1,0,100,0,100,1,0,100,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,1,0,100,0,100,1,0,100,1,16,0,1,1,1,0,0,0,1,1,0,360,0,360,1,0,4,0,0,1,0,90,0,90,1,0,4,0,0,0,0); -INSERT INTO Controls VALUES (NULL,'Loftek Sentinel','Remote','LoftekSentinel',0,0,1,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,255,16,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,6,1,1,0,0,0,1,10,0,1,1,1,0,0,0,1,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0); -INSERT INTO Controls VALUES (NULL,'Toshiba IK-WB11A','Remote','Toshiba_IK_WB11A',0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,10,0,1,1,0,1,0,1,0,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0); -INSERT INTO Controls VALUES (NULL,'WanscamPT','Remote','Wanscam',1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,16,0,0,0,0,0,1,16,1,1,1,1,0,0,0,1,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0); -INSERT INTO Controls VALUES (NULL,'3S Domo N5071', 'Remote', '3S', 0, 0, 1, 0,1, 0, 1, 1, 0, 0, 9999, 0, 9999, 0, 0, 0, 1, 1, 1, 1, 0, 0, 9999, 20, 9999, 0, 0, 0, 1, 1, 1, 1, 0, 0, 9999, 1, 9999, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 64, 1, 0, 1, 1, 0, 0, 0, 0, 1, -180, 180, 40, 100, 1, 40, 100, 0, 0, 1, -180, 180, 40, 100, 1, 40, 100, 0, 0, 0, 0); -INSERT INTO Controls VALUES (NULL,'ONVIF Camera','Ffmpeg','onvif',0,0,1,1,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,255,16,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,6,1,1,0,0,0,1,10,1,1,1,1,0,0,0,1,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0); +INSERT INTO `Controls` VALUES (NULL,'Pelco-D','Local','PelcoD',1,1,0,0,1,1,0,0,1,NULL,NULL,NULL,NULL,1,0,3,1,1,0,0,1,NULL,NULL,NULL,NULL,0,NULL,NULL,1,1,0,1,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,1,0,1,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,20,1,1,1,1,0,0,0,1,1,NULL,NULL,NULL,NULL,1,0,63,1,254,1,NULL,NULL,NULL,NULL,1,0,63,1,254,0,0); +INSERT INTO `Controls` VALUES (NULL,'Pelco-P','Local','PelcoP',1,1,0,0,1,1,0,0,1,NULL,NULL,NULL,NULL,1,0,3,1,1,0,0,1,NULL,NULL,NULL,NULL,0,NULL,NULL,1,1,0,1,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,1,0,1,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,20,1,1,1,1,0,0,0,1,1,NULL,NULL,NULL,NULL,1,0,63,1,254,1,NULL,NULL,NULL,NULL,1,0,63,1,254,0,0); +INSERT INTO `Controls` VALUES (NULL,'Sony VISCA','Local','Visca',1,1,0,0,1,0,0,0,1,0,16384,10,4000,1,1,6,1,1,1,0,1,0,1536,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,3,1,1,1,1,0,1,1,0,1,-15578,15578,100,10000,1,1,50,1,254,1,-7789,7789,100,5000,1,1,50,1,254,0,0); +INSERT INTO `Controls` VALUES (NULL,'Axis API v2','Remote','AxisV2',0,0,0,0,1,0,0,1,0,0,9999,10,2500,0,NULL,NULL,1,1,0,1,0,0,9999,10,2500,0,NULL,NULL,1,1,0,1,0,0,9999,10,2500,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,12,1,1,1,1,1,0,1,0,1,-360,360,1,90,0,NULL,NULL,0,NULL,1,-360,360,1,90,0,NULL,NULL,0,NULL,0,0); +INSERT INTO `Controls` VALUES (NULL,'Panasonic IP','Remote','PanasonicIP',0,0,0,0,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,8,1,1,1,0,1,0,0,1,1,NULL,NULL,NULL,NULL,0,NULL,NULL,0,NULL,1,NULL,NULL,NULL,NULL,0,NULL,NULL,0,NULL,0,0); +INSERT INTO `Controls` VALUES (NULL,'Neu-Fusion NCS370','Remote','Ncs370',0,0,0,0,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,24,1,0,1,1,0,0,0,1,1,NULL,NULL,NULL,NULL,0,NULL,NULL,0,NULL,1,NULL,NULL,NULL,NULL,0,NULL,NULL,0,NULL,0,0); +INSERT INTO `Controls` VALUES (NULL,'AirLink SkyIPCam 7xx','Remote','SkyIPCam7xx',0,0,1,0,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,8,1,1,1,0,1,0,1,0,1,NULL,NULL,NULL,NULL,0,NULL,NULL,0,NULL,1,NULL,NULL,NULL,NULL,0,NULL,NULL,0,NULL,0,0); +INSERT INTO `Controls` VALUES (NULL,'Pelco-D','Ffmpeg','PelcoD',1,1,0,0,1,1,0,0,1,NULL,NULL,NULL,NULL,1,0,3,1,1,0,0,1,NULL,NULL,NULL,NULL,0,NULL,NULL,1,1,0,1,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,1,0,1,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,20,1,1,1,1,0,0,0,1,1,NULL,NULL,NULL,NULL,1,0,63,1,254,1,NULL,NULL,NULL,NULL,1,0,63,1,254,0,0); +INSERT INTO `Controls` VALUES (NULL,'Pelco-P','Ffmpeg','PelcoP',1,1,0,0,1,1,0,0,1,NULL,NULL,NULL,NULL,1,0,3,1,1,0,0,1,NULL,NULL,NULL,NULL,0,NULL,NULL,1,1,0,1,0,NULL,NULL,NULL,NULL,0,NULL,NULL,0,0,0,0,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,1,0,1,0,NULL,NULL,NULL,NULL,0,NULL,NULL,1,20,1,1,1,1,0,0,0,1,1,NULL,NULL,NULL,NULL,1,0,63,1,254,1,NULL,NULL,NULL,NULL,1,0,63,1,254,0,0); +INSERT INTO `Controls` VALUES (NULL,'Foscam FI8620','Ffmpeg','FI8620_Y2k',0,0,0,0,1,0,0,0,1,1,10,1,10,1,1,63,1,1,0,0,1,1,63,1,63,1,1,63,1,1,0,0,1,0,0,0,0,1,0,255,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,1,0,0,0,0,1,0,255,1,8,0,1,1,1,0,0,0,1,1,1,360,1,360,1,1,63,0,0,1,1,90,1,90,1,1,63,0,0,0,0); +INSERT INTO `Controls` VALUES (NULL,'Foscam FI8608W','Ffmpeg','FI8608W_Y2k',1,0,1,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,1,0,0,0,0,1,0,255,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,1,0,0,0,0,1,0,255,1,8,0,1,1,1,0,0,0,1,1,0,0,0,0,1,1,128,0,0,1,0,0,0,0,1,1,128,0,0,0,0); +INSERT INTO `Controls` VALUES (NULL,'Foscam FI8908W','Remote','FI8908W',0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,1,1,0,0,0,1,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0); +INSERT INTO `Controls` VALUES (NULL,'Foscam FI9821W','Ffmpeg','FI9821W_Y2k',1,0,1,0,1,0,0,0,1,0,0,0,0,0,0,0,1,1,0,0,1,0,0,0,0,1,0,100,1,1,0,0,1,0,100,0,100,1,0,100,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,1,0,100,0,100,1,0,100,1,16,0,1,1,1,0,0,0,1,1,0,360,0,360,1,0,4,0,0,1,0,90,0,90,1,0,4,0,0,0,0); +INSERT INTO `Controls` VALUES (NULL,'Loftek Sentinel','Remote','LoftekSentinel',0,0,1,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,255,16,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,6,1,1,0,0,0,1,10,0,1,1,1,0,0,0,1,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0); +INSERT INTO `Controls` VALUES (NULL,'Toshiba IK-WB11A','Remote','Toshiba_IK_WB11A',0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,10,0,1,1,0,1,0,1,0,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0); +INSERT INTO `Controls` VALUES (NULL,'WanscamPT','Remote','Wanscam',1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,16,0,0,0,0,0,1,16,1,1,1,1,0,0,0,1,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0); +INSERT INTO `Controls` VALUES (NULL,'3S Domo N5071', 'Remote', '3S', 0, 0, 1, 0,1, 0, 1, 1, 0, 0, 9999, 0, 9999, 0, 0, 0, 1, 1, 1, 1, 0, 0, 9999, 20, 9999, 0, 0, 0, 1, 1, 1, 1, 0, 0, 9999, 1, 9999, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 64, 1, 0, 1, 1, 0, 0, 0, 0, 1, -180, 180, 40, 100, 1, 40, 100, 0, 0, 1, -180, 180, 40, 100, 1, 40, 100, 0, 0, 0, 0); +INSERT INTO `Controls` VALUES (NULL,'ONVIF Camera','Ffmpeg','onvif',0,0,1,1,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,255,16,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,6,1,1,0,0,0,1,10,1,1,1,1,1,0,1,1,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0); INSERT INTO `Controls` VALUES (NULL,'Foscam 9831W','Ffmpeg','FI9831W',0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,16,1,1,1,1,0,0,0,1,1,0,360,0,360,1,0,4,0,0,1,0,90,0,90,0,0,0,0,0,0,0); INSERT INTO `Controls` VALUES (NULL,'Foscam FI8918W','Ffmpeg','FI8918W',0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,8,0,1,1,1,0,0,0,1,1,0,360,0,360,1,0,4,0,0,1,0,90,0,90,1,0,4,0,0,0,0); INSERT INTO `Controls` VALUES (NULL,'SunEyes SP-P1802SWPTZ','Libvlc','SPP1802SWPTZ',0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,8,0,1,1,0,0,0,0,1,1,0,0,0,0,1,0,64,0,0,1,0,0,0,0,1,0,64,0,0,0,0); diff --git a/scripts/ZoneMinder/lib/ZoneMinder/Control/onvif.pm b/scripts/ZoneMinder/lib/ZoneMinder/Control/onvif.pm index 4ae2ba1fc..12a78144c 100644 --- a/scripts/ZoneMinder/lib/ZoneMinder/Control/onvif.pm +++ b/scripts/ZoneMinder/lib/ZoneMinder/Control/onvif.pm @@ -60,14 +60,14 @@ my ( $controlUri, $scheme ); # This will use the following defaults: # - port: 80 # - Control Device: 000 -# - Control URI: /onvif/device_control +# - Control URI: /onvif/PTZ # - No authentication # - No Auto Stop Timeout (on movement command the camera will keep moving until it reaches it's edge) # # Example Control Address values: # - http://user:password@192.168.1.199:888/onvif/device_control :Connect to camera at IP: 192.168.1.199 on port 888 with "username" and "password" credentials using /onvif/device_control URI -# - user:password@192.168.1.199 :Connect to camera at IP: 192.168.1.199 on default port 80 with "username" and "password" credentials using default /onvif/device_control URI -# - 192.168.1.199 :Connect to camera at IP: 192.168.1.199 without any authentication and use the default /onvif/device_control URI over HTTP. +# - user:password@192.168.1.199 :Connect to camera at IP: 192.168.1.199 on default port 80 with "username" and "password" credentials using default /onvif/PTZ URI +# - 192.168.1.199 :Connect to camera at IP: 192.168.1.199 without any authentication and use the default /onvif/PTZ URI over HTTP. # # ========================================================================== @@ -122,7 +122,7 @@ sub parseControlAddress { } if (!$url->path){ - $controlUri = "/onvif/device_control"; + $controlUri = "/onvif/PTZ"; } else { $controlUri = $url->path; } @@ -289,9 +289,54 @@ END_MESSAGE $self->sendCmd($cmd, $msg_body, $content_type); } +sub moveMap { + my $self = shift; + my $params = shift; + my $x = $self->getParam($params,'xcoord'); + my $y = $self->getParam($params,'ycoord'); + Debug("Move map to $x x $y"); + + my $cmd = $controlUri; + my $msg_body =' + + + ' . $profileToken . ' + + + + + + + + '; + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/ptz/wsdl/ContinuousMove"'; + $self->sendCmd($cmd, $msg_body, $content_type); +} + +sub moveRel { + my $self = shift; + my $params = shift; + my $x = $self->getParam($params,'xcoord'); + my $speed = $self->getParam($params,'speed'); + my $y = $self->getParam($params,'ycoord'); + Debug("Move rel to $x x $y"); + + my $cmd = $controlUri; + my $msg_body =' + + + ' . $profileToken . ' + + + + + + '; + my $content_type = 'application/soap+xml; charset=utf-8; action="http://www.onvif.org/ver20/ptz/wsdl/ContinuousMove"'; + $self->sendCmd($cmd, $msg_body, $content_type); +} sub moveCamera { - my $type = shift; my $x = shift; my $y = shift; @@ -299,17 +344,17 @@ sub moveCamera { if ( $type == "move" ){ $msg_move_body = ' - - - '.$profileToken.' - - + + '.$profileToken.' + + - - - '; + + + '; } elsif ( $type == "zoom" ) { $msg_move_body = ' @@ -320,9 +365,9 @@ sub moveCamera { - - - '; + + + '; } return $msg_move_body; From 1c40145e89578ea293da5bf3cdca90453828f758 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 2 Mar 2021 12:41:26 -0500 Subject: [PATCH 0072/1277] not finding space for pts is debug now and show the contents of header --- src/zm_rtsp_server_fifo_source.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/zm_rtsp_server_fifo_source.cpp b/src/zm_rtsp_server_fifo_source.cpp index 3aa5f85a6..ee23d8167 100644 --- a/src/zm_rtsp_server_fifo_source.cpp +++ b/src/zm_rtsp_server_fifo_source.cpp @@ -184,7 +184,7 @@ int ZoneMinderFifoSource::getNextFrame() { char *pts_ptr = strchr(content_length_ptr, ' '); if (!pts_ptr) { m_buffer.consume(header_start-m_buffer.head() + 2); - Warning("Didn't find space delineating pts"); + Debug(1, "Didn't find space delineating pts in %s", header); delete header; return -1; } From be1b439dcc6ca02fe4c3fbb9c55dc3a76ddf6150 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 2 Mar 2021 14:42:54 -0500 Subject: [PATCH 0073/1277] Add State_Strings and code cleanup. Fix not going into alarm state when already recording. Fixes #3184 --- src/zm_monitor.cpp | 71 ++++++++++++++++++++++++++++------------------ 1 file changed, 43 insertions(+), 28 deletions(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 821cffa74..731a243c6 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -102,6 +102,14 @@ std::string CameraType_Strings[] = { "VNC", }; +std::string State_Strings[] = { + "IDLE", + "PREALARM", + "ALARM", + "ALERT", + "TAPE" +}; + Monitor::MonitorLink::MonitorLink(unsigned int p_id, const char *p_name) : id(p_id), shared_data(nullptr), @@ -1881,8 +1889,8 @@ bool Monitor::Analyse() { ref_image.Assign(*(snap->image)); }// else - if ( signal ) { - if ( snap->image or (snap->packet.stream_index == video_stream_id) ) { + if (signal) { + if (snap->image or (snap->packet.stream_index == video_stream_id)) { struct timeval *timestamp = snap->timestamp; if ( Active() and (function == MODECT or function == MOCORD) and snap->image ) { @@ -1914,17 +1922,16 @@ bool Monitor::Analyse() { } else { Debug(1, "Skipped motion detection"); } - if ( motion_score ) { + if (motion_score) { score += motion_score; - if ( cause.length() ) - cause += ", "; + if (cause.length()) cause += ", "; cause += MOTION_CAUSE; noteSetMap[MOTION_CAUSE] = zoneSet; } // end if motion_score } // end if active and doing motion detection // Check to see if linked monitors are triggering. - if ( n_linked_monitors > 0 ) { + if (n_linked_monitors > 0) { Debug(4, "Checking linked monitors"); // FIXME improve logic here bool first_link = true; @@ -1960,12 +1967,12 @@ bool Monitor::Analyse() { noteSetMap[LINKED_CAUSE] = noteSet; } // end if linked_monitors - if ( function == RECORD || function == MOCORD ) { + if (function == RECORD or function == MOCORD) { // If doing record, check to see if we need to close the event or not. - if ( event ) { + if (event) { Debug(2, "Have event %" PRIu64 " in mocord", event->Id()); - if ( section_length + if (section_length && ( ( timestamp->tv_sec - video_store_data->recording.tv_sec ) >= section_length ) && ( (function == MOCORD && (event_close_mode != CLOSE_TIME)) || ! ( timestamp->tv_sec % section_length ) ) ) { @@ -1980,9 +1987,9 @@ bool Monitor::Analyse() { } // end if section_length } // end if event - if ( !event ) { + if (!event) { Debug(2, "Creating continuous event"); - if ( !snap->keyframe and (videowriter == PASSTHROUGH) ) { + if (!snap->keyframe and (videowriter == PASSTHROUGH)) { // Must start on a keyframe so rewind. Only for passthrough though I guess. // FIXME this iterator is not protected from invalidation packetqueue_iterator *start_it = packetqueue.get_event_start_packet_it( @@ -2032,16 +2039,16 @@ bool Monitor::Analyse() { Info("%s: %03d - Opened new event %" PRIu64 ", section start", name, analysis_image_count, event->Id()); /* To prevent cancelling out an existing alert\prealarm\alarm state */ - if ( state == IDLE ) { + if (state == IDLE) { shared_data->state = state = TAPE; } } // end if ! event } // end if RECORDING - if ( score ) { - if ( (state == IDLE) || (state == TAPE) || (state == PREALARM) ) { + if (score) { + if ((state == IDLE) || (state == TAPE) || (state == PREALARM)) { // If we should end then previous continuous event and start a new non-continuous event - if ( event && event->Frames() + if (event && event->Frames() && (!event->AlarmFrames()) && (event_close_mode == CLOSE_ALARM) && ( ( timestamp->tv_sec - video_store_data->recording.tv_sec ) >= min_section_length ) @@ -2050,28 +2057,28 @@ bool Monitor::Analyse() { Info("%s: %03d - Closing event %" PRIu64 ", continuous end, alarm begins", name, image_count, event->Id()); closeEvent(); - } else if ( event ) { + } else if (event) { // This is so if we need more than 1 alarm frame before going into alarm, so it is basically if we have enough alarm frames Debug(3, "pre-alarm-count in event %d, event frames %d, alarm frames %d event length %d >=? %d min", Event::PreAlarmCount(), event->Frames(), event->AlarmFrames(), ( timestamp->tv_sec - video_store_data->recording.tv_sec ), min_section_length ); } - if ( (!pre_event_count) || (Event::PreAlarmCount() >= alarm_frame_count-1) ) { + if ((!pre_event_count) || (Event::PreAlarmCount() >= alarm_frame_count-1)) { // lets construct alarm cause. It will contain cause + names of zones alarmed std::string alarm_cause = ""; - for ( int i=0; i < n_zones; i++ ) { - if ( zones[i]->Alarmed() ) { + for (int i=0; i < n_zones; i++) { + if (zones[i]->Alarmed()) { alarm_cause = alarm_cause + "," + std::string(zones[i]->Label()); } } - if ( !alarm_cause.empty() ) alarm_cause[0] = ' '; + if (!alarm_cause.empty()) alarm_cause[0] = ' '; alarm_cause = cause + alarm_cause; strncpy(shared_data->alarm_cause, alarm_cause.c_str(), sizeof(shared_data->alarm_cause)-1); Info("%s: %03d - Gone into alarm state PreAlarmCount: %u > AlarmFrameCount:%u Cause:%s", name, image_count, Event::PreAlarmCount(), alarm_frame_count, shared_data->alarm_cause); - if ( !event ) { + if (!event) { packetqueue_iterator *start_it = packetqueue.get_event_start_packet_it( snap_it, (pre_event_count > alarm_frame_count ? pre_event_count : alarm_frame_count) @@ -2080,13 +2087,12 @@ bool Monitor::Analyse() { event = new Event(this, *(starting_packet->timestamp), cause, noteSetMap); shared_data->last_event_id = event->Id(); - //set up video store data snprintf(video_store_data->event_file, sizeof(video_store_data->event_file), "%s", event->getEventFile()); video_store_data->recording = event->StartTime(); shared_data->state = state = ALARM; // Write out starting packets, do not modify packetqueue it will garbage collect itself - while ( *start_it != snap_it ) { + while (*start_it != snap_it) { event->AddPacket(starting_packet); packetqueue.increment_it(start_it); @@ -2103,16 +2109,18 @@ bool Monitor::Analyse() { start_it = nullptr; Info("%s: %03d - Opening new event %" PRIu64 ", alarm start", name, analysis_image_count, event->Id()); + } else { + shared_data->state = state = ALARM; } // end if no event, so start it if ( alarm_frame_count ) { Debug(1, "alarm frame count so SavePreAlarmFrames"); event->SavePreAlarmFrames(); } - } else if ( state != PREALARM ) { + } else if (state != PREALARM) { Info("%s: %03d - Gone into prealarm state", name, analysis_image_count); shared_data->state = state = PREALARM; } - } else if ( state == ALERT ) { + } else if (state == ALERT) { alert_to_alarm_frame_count--; Info("%s: %03d - Alarmed frame while in alert state. Consecutive alarmed frames left to return to alarm state: %03d", name, analysis_image_count, alert_to_alarm_frame_count); @@ -2120,11 +2128,18 @@ bool Monitor::Analyse() { Info("%s: %03d - Gone back into alarm state", name, analysis_image_count); shared_data->state = state = ALARM; } + } else if (state == TAPE) { + // Already recording, but IDLE so switch to ALARM + shared_data->state = state = ALARM; + Debug(1, "Was in TAPE, going into ALARM"); + } else { + Debug(1, "Staying in %s", State_Strings[state].c_str()); + } last_alarm_count = analysis_image_count; } else { // no score? alert_to_alarm_frame_count = alarm_frame_count; // load same value configured for alarm_frame_count - if ( state == ALARM ) { + if (state == ALARM) { Info("%s: %03d - Gone into alert state", name, analysis_image_count); shared_data->state = state = ALERT; } else if ( state == ALERT ) { @@ -2149,8 +2164,8 @@ bool Monitor::Analyse() { // Back to IDLE shared_data->state = state = ((function != MOCORD) ? IDLE : TAPE); } else { - Debug(1, "State %d because image_count(%d)-last_alarm_count(%d) > post_event_count(%d) and timestamp.tv_sec(%d) - recording.tv_src(%d) >= min_section_length(%d)", - state, analysis_image_count, last_alarm_count, post_event_count, + Debug(1, "State %s because image_count(%d)-last_alarm_count(%d) > post_event_count(%d) and timestamp.tv_sec(%d) - recording.tv_src(%d) >= min_section_length(%d)", + State_Strings[state].c_str(), analysis_image_count, last_alarm_count, post_event_count, timestamp->tv_sec, video_store_data->recording.tv_sec, min_section_length); } if ( Event::PreAlarmCount() ) From 4ff016c991458088d9ec03f6e9fdf7e5d626316c Mon Sep 17 00:00:00 2001 From: Admin Date: Tue, 2 Mar 2021 20:51:16 +0100 Subject: [PATCH 0074/1277] Make capturing/analysing log lines follow same pattern --- src/zm_monitor.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 731a243c6..ddde36fe8 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -1725,7 +1725,7 @@ void Monitor::UpdateCaptureFPS() { now_double, last_analysis_fps_time, elapsed, new_capture_fps ); - Info("%s: images:%d - Capturing at %.2lf fps, capturing bandwidth %ubytes/sec", + Info("%s: %d - Capturing at %.2lf fps, capturing bandwidth %ubytes/sec", name, image_count, new_capture_fps, new_capture_bandwidth); shared_data->capture_fps = new_capture_fps; last_fps_time = now_double; From 7d02c6893884fc60606b92490a567098b0cc4e56 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 2 Mar 2021 15:10:02 -0500 Subject: [PATCH 0075/1277] silence warning when cookies not used --- web/api/app/Controller/AppController.php | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/web/api/app/Controller/AppController.php b/web/api/app/Controller/AppController.php index 38aa525e6..60c7492c3 100644 --- a/web/api/app/Controller/AppController.php +++ b/web/api/app/Controller/AppController.php @@ -100,7 +100,7 @@ class AppController extends Controller { } generateAuthHash(ZM_AUTH_HASH_IPS); session_write_close(); - } else if ( $_COOKIE['ZMSESSID'] and !$user ) { + } else if ( isset($_COOKIE['ZMSESSID']) and !$user ) { # Have a cookie set, try to load user by session if ( ! is_session_started() ) zm_session_start(); From 849d71efff05453582605ea66d074fd947c1c6ad Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 2 Mar 2021 15:10:15 -0500 Subject: [PATCH 0076/1277] Add filtering to groups --- web/api/app/Controller/GroupsController.php | 27 +++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-) diff --git a/web/api/app/Controller/GroupsController.php b/web/api/app/Controller/GroupsController.php index f7ed57637..fe83bd3b3 100644 --- a/web/api/app/Controller/GroupsController.php +++ b/web/api/app/Controller/GroupsController.php @@ -31,8 +31,31 @@ class GroupsController extends AppController { * @return void */ public function index() { - $this->Group->recursive = -1; - $groups = $this->Group->find('all'); + $this->Group->recursive = 0; + + if ( $this->request->params['named'] ) { + $this->FilterComponent = $this->Components->load('Filter'); + $conditions = $this->FilterComponent->buildFilter($this->request->params['named']); + } else { + $conditions = array(); + } + + $find_array = array( + 'conditions' => &$conditions, + 'contain' => array('Monitor'), + 'joins' => array( + array( + 'table' => 'Groups_Monitors', + 'type' => 'left', + 'conditions' => array( + 'Groups_Monitors.GroupId = Group.Id', + ), + ), + ), + 'group' => '`Group`.`Id`', + ); + + $groups = $this->Group->find('all', $find_array); $this->set(array( 'groups' => $groups, '_serialize' => array('groups') From 53425257cbcef15c75ccca8035e0628f80ac2b60 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 2 Mar 2021 16:20:47 -0500 Subject: [PATCH 0077/1277] Handle old avcodec --- src/zm_rtsp_server_thread.cpp | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/src/zm_rtsp_server_thread.cpp b/src/zm_rtsp_server_thread.cpp index 30e1bd0d9..c57de8b07 100644 --- a/src/zm_rtsp_server_thread.cpp +++ b/src/zm_rtsp_server_thread.cpp @@ -141,7 +141,13 @@ void RTSPServerThread::addStream(std::string &streamname, AVStream *video_stream if ( video_stream ) { StreamReplicator* videoReplicator = nullptr; FramedSource *source = nullptr; - std::string rtpFormat = getRtpFormat(video_stream->codecpar->codec_id, false); + std::string rtpFormat = getRtpFormat( +#if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) + video_stream->codecpar->codec_id +#else + video_stream->codec->codec_id +#endif + , false); if ( rtpFormat.empty() ) { Error("No streaming format"); return; @@ -169,7 +175,13 @@ void RTSPServerThread::addStream(std::string &streamname, AVStream *video_stream if ( audio_stream ) { StreamReplicator* replicator = nullptr; FramedSource *source = nullptr; - std::string rtpFormat = getRtpFormat(audio_stream->codecpar->codec_id, false); + std::string rtpFormat = getRtpFormat( +#if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) + audio_stream->codecpar->codec_id +#else + audio_stream->codec->codec_id +#endif + , false); if ( rtpFormat == "audio/AAC" ) { source = ADTS_ZoneMinderDeviceSource::createNew(*env, monitor_, audio_stream, queueSize); Debug(1, "ADTS source %p", source); From 78a3d154574b5965201ebcd3e77792aa7d4bdb01 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 2 Mar 2021 16:30:40 -0500 Subject: [PATCH 0078/1277] Fix compile on old avcodec --- src/zm_rtsp_server_adts_source.cpp | 8 +++++++- src/zm_rtsp_server_adts_source.h | 15 +++++++-------- src/zm_rtsp_server_h264_device_source.cpp | 16 ++++++++++++++-- 3 files changed, 28 insertions(+), 11 deletions(-) diff --git a/src/zm_rtsp_server_adts_source.cpp b/src/zm_rtsp_server_adts_source.cpp index 1c0c1a5a4..f6d1d81c0 100644 --- a/src/zm_rtsp_server_adts_source.cpp +++ b/src/zm_rtsp_server_adts_source.cpp @@ -34,7 +34,13 @@ ADTS_ZoneMinderDeviceSource::ADTS_ZoneMinderDeviceSource( : ZoneMinderDeviceSource(env, std::move(monitor), stream, queueSize), samplingFrequencyIndex(0), - channels(stream->codecpar->channels) + channels( +#if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) + stream->codecpar->channels +#else + stream->codec->channels +#endif + ) { std::ostringstream os; os << diff --git a/src/zm_rtsp_server_adts_source.h b/src/zm_rtsp_server_adts_source.h index d8516bc30..b6f906a12 100644 --- a/src/zm_rtsp_server_adts_source.h +++ b/src/zm_rtsp_server_adts_source.h @@ -28,8 +28,6 @@ class ADTS_ZoneMinderDeviceSource : public ZoneMinderDeviceSource { AVStream * stream, unsigned int queueSize ) { - Debug(1, "m_stream %p codecpar %p channels %d", - stream, stream->codecpar, stream->codecpar->channels); return new ADTS_ZoneMinderDeviceSource(env, std::move(monitor), stream, queueSize); }; protected: @@ -47,15 +45,16 @@ class ADTS_ZoneMinderDeviceSource : public ZoneMinderDeviceSource { virtual unsigned char* findMarker(unsigned char *frame, size_t size, size_t &length); */ public: - int samplingFrequency() { return m_stream->codecpar->sample_rate; }; + int samplingFrequency() { return +#if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) + m_stream->codecpar->sample_rate; +#else + m_stream->codec->sample_rate; +#endif + }; const char *configStr() { return config.c_str(); }; int numChannels() { - Debug(1, "this %p m_stream %p channels %d", - this, m_stream, channels); - Debug(1, "m_stream %p codecpar %p channels %d => %d", - m_stream, m_stream->codecpar, m_stream->codecpar->channels, channels); return channels; - return m_stream->codecpar->channels; } protected: diff --git a/src/zm_rtsp_server_h264_device_source.cpp b/src/zm_rtsp_server_h264_device_source.cpp index c10852196..71b641021 100644 --- a/src/zm_rtsp_server_h264_device_source.cpp +++ b/src/zm_rtsp_server_h264_device_source.cpp @@ -32,7 +32,13 @@ H264_ZoneMinderDeviceSource::H264_ZoneMinderDeviceSource( : H26X_ZoneMinderDeviceSource(env, std::move(monitor), stream, queueSize, repeatConfig, keepMarker) { // extradata appears to simply be the SPS and PPS NAL's - this->splitFrames(m_stream->codecpar->extradata, m_stream->codecpar->extradata_size); + this->splitFrames( +#if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) + m_stream->codecpar->extradata, m_stream->codecpar->extradata_size +#else + m_stream->codec->extradata, m_stream->codec->extradata_size +#endif + ); } // split packet into frames @@ -97,7 +103,13 @@ H265_ZoneMinderDeviceSource::H265_ZoneMinderDeviceSource( : H26X_ZoneMinderDeviceSource(env, std::move(monitor), stream, queueSize, repeatConfig, keepMarker) { // extradata appears to simply be the SPS and PPS NAL's - this->splitFrames(m_stream->codecpar->extradata, m_stream->codecpar->extradata_size); + this->splitFrames( +#if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) + m_stream->codecpar->extradata, m_stream->codecpar->extradata_size +#else + m_stream->codec->extradata, m_stream->codec->extradata_size +#endif + ); } // split packet in frames From 9f5f215ef445ab4e246c370a0158c7e20c228cf5 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 3 Mar 2021 09:51:43 -0500 Subject: [PATCH 0079/1277] Check codec type instead of stream_index to determine video/audio --- src/zm_videostore.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/zm_videostore.cpp b/src/zm_videostore.cpp index 0a3cc6002..a2df34e35 100644 --- a/src/zm_videostore.cpp +++ b/src/zm_videostore.cpp @@ -945,9 +945,9 @@ bool VideoStore::setup_resampler() { } // end bool VideoStore::setup_resampler() int VideoStore::writePacket(ZMPacket *ipkt) { - if ( ipkt->packet.stream_index == video_in_stream_index ) { + if ( ipkt->codec_type == AVMEDIA_TYPE_VIDEO ) { return writeVideoFramePacket(ipkt); - } else if ( ipkt->packet.stream_index == audio_in_stream_index ) { + } else if ( ipkt->codec_type == AVMEDIA_TYPE_AUDIO ) { return writeAudioFramePacket(ipkt); } Error("Unknown stream type in packet (%d) input video stream is (%d) and audio is (%d)", From cf27482ebe4dca94cf61909fa8dcdd0d54b91d79 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 3 Mar 2021 09:52:13 -0500 Subject: [PATCH 0080/1277] Change api of packetqueue. stream_index will not be incoming stream_index it will be a packetqueue specific stream_index. --- src/zm_packetqueue.cpp | 21 ++++++++++++--------- src/zm_packetqueue.h | 2 +- 2 files changed, 13 insertions(+), 10 deletions(-) diff --git a/src/zm_packetqueue.cpp b/src/zm_packetqueue.cpp index c89347692..588355287 100644 --- a/src/zm_packetqueue.cpp +++ b/src/zm_packetqueue.cpp @@ -38,17 +38,20 @@ PacketQueue::PacketQueue(): /* Assumes queue is empty when adding streams * Assumes first stream added will be the video stream */ -void PacketQueue::addStreamId(int p_stream_id) { +int PacketQueue::addStream() { deleting = false; - if ( video_stream_id == -1 ) - video_stream_id = p_stream_id; - if ( max_stream_id < p_stream_id ) { - if ( packet_counts ) delete[] packet_counts; - max_stream_id = p_stream_id; - packet_counts = new int[max_stream_id+1]; - for ( int i=0; i <= max_stream_id; ++i ) - packet_counts[i] = 0; + if ( max_stream_id == -1 ) { + video_stream_id = 0; + max_stream_id = 0; + } else { + max_stream_id ++; } + + if ( packet_counts ) delete[] packet_counts; + packet_counts = new int[max_stream_id+1]; + for ( int i=0; i <= max_stream_id; ++i ) + packet_counts[i] = 0; + return max_stream_id; } PacketQueue::~PacketQueue() { diff --git a/src/zm_packetqueue.h b/src/zm_packetqueue.h index cd88648b9..c0ba045b8 100644 --- a/src/zm_packetqueue.h +++ b/src/zm_packetqueue.h @@ -48,7 +48,7 @@ class PacketQueue { std::list::const_iterator end() const { return pktQueue.end(); } std::list::const_iterator begin() const { return pktQueue.begin(); } - void addStreamId(int p_stream_id); + int addStream(); void setMaxVideoPackets(int p); bool queuePacket(ZMPacket* packet); From 46ec4e75d41b7d7bef67dc34e949fc2d1ef6833e Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 3 Mar 2021 09:52:27 -0500 Subject: [PATCH 0081/1277] Check codec type instead of stream_index to determine video/audio --- src/zm_monitor.cpp | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index ddde36fe8..9e0e55475 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -1890,7 +1890,7 @@ bool Monitor::Analyse() { }// else if (signal) { - if (snap->image or (snap->packet.stream_index == video_stream_id)) { + if (snap->image or (snap->codec_type == AVMEDIA_TYPE_VIDEO)) { struct timeval *timestamp = snap->timestamp; if ( Active() and (function == MODECT or function == MOCORD) and snap->image ) { @@ -2554,7 +2554,7 @@ int Monitor::Capture() { Debug(2, "Have packet stream_index:%d ?= videostream_id:(%d) q.vpktcount(%d) event?(%d) ", packet->packet.stream_index, video_stream_id, packetqueue.packet_count(video_stream_id), ( event ? 1 : 0 ) ); - if (packet->packet.stream_index == video_stream_id) { + if (packet->codec_type == AVMEDIA_TYPE_VIDEO) { if (video_fifo) { if ( packet->keyframe ) { // avcodec strips out important nals that describe the stream and @@ -2569,12 +2569,13 @@ int Monitor::Capture() { } video_fifo->writePacket(*packet); } - } else if (packet->packet.stream_index == audio_stream_id) { + } else if (packet->codec_type == AVMEDIA_TYPE_VIDEO) { if (audio_fifo) audio_fifo->writePacket(*packet); } - if ( (packet->packet.stream_index != video_stream_id) and ! packet->image ) { + //if ( (packet->packet.stream_index != video_stream_id) and ! packet->image ) { + if ( (packet->codec_type == AVMEDIA_TYPE_AUDIO) and ! packet->image ) { // Only queue if we have some video packets in there. Should push this logic into packetqueue if ( record_audio and (packetqueue.packet_count(video_stream_id) or event) ) { packet->image_index=-1; @@ -2982,13 +2983,13 @@ unsigned int Monitor::SubpixelOrder() const { return camera->SubpixelOrder(); } int Monitor::PrimeCapture() { int ret = camera->PrimeCapture(); if ( ret > 0 ) { - video_stream_id = camera->get_VideoStreamId(); + if ( -1 != camera->get_VideoStreamId() ) { + video_stream_id = packetqueue.addStream(); + } - packetqueue.addStreamId(video_stream_id); - - audio_stream_id = camera->get_AudioStreamId(); - if ( audio_stream_id >= 0 ) { - packetqueue.addStreamId(audio_stream_id); + if ( -1 != camera->get_AudioStreamId() ) { + audio_stream_id = packetqueue.addStream(); + packetqueue.addStream(); shared_data->audio_frequency = camera->getFrequency(); shared_data->audio_channels = camera->getChannels(); } @@ -3057,7 +3058,7 @@ void Monitor::get_ref_image() { ( !( snap = packetqueue.get_packet(analysis_it)) or - ( snap->packet.stream_index != video_stream_id ) + ( snap->codec_type != AVMEDIA_TYPE_VIDEO ) or ! snap->image ) From dcd1804b4d8c886b002eacb0b0efd3da2ea03bb5 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 3 Mar 2021 09:52:44 -0500 Subject: [PATCH 0082/1277] add get_format_context() --- src/zm_ffmpeg_input.h | 1 + 1 file changed, 1 insertion(+) diff --git a/src/zm_ffmpeg_input.h b/src/zm_ffmpeg_input.h index bd368bf62..6a339f081 100644 --- a/src/zm_ffmpeg_input.h +++ b/src/zm_ffmpeg_input.h @@ -36,6 +36,7 @@ class FFmpeg_Input { int get_audio_stream_id() const { return audio_stream_id; } + AVFormatContext * get_format_context() { return input_format_context; }; private: typedef struct { From 0550e692246c78f8481eeae2fe865ac53807c591 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 2 Mar 2021 16:30:40 -0500 Subject: [PATCH 0083/1277] Fix compile on old avcodec --- src/zm_rtsp_server_adts_source.cpp | 8 +++++++- src/zm_rtsp_server_adts_source.h | 15 +++++++-------- src/zm_rtsp_server_h264_device_source.cpp | 16 ++++++++++++++-- 3 files changed, 28 insertions(+), 11 deletions(-) diff --git a/src/zm_rtsp_server_adts_source.cpp b/src/zm_rtsp_server_adts_source.cpp index 1c0c1a5a4..f6d1d81c0 100644 --- a/src/zm_rtsp_server_adts_source.cpp +++ b/src/zm_rtsp_server_adts_source.cpp @@ -34,7 +34,13 @@ ADTS_ZoneMinderDeviceSource::ADTS_ZoneMinderDeviceSource( : ZoneMinderDeviceSource(env, std::move(monitor), stream, queueSize), samplingFrequencyIndex(0), - channels(stream->codecpar->channels) + channels( +#if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) + stream->codecpar->channels +#else + stream->codec->channels +#endif + ) { std::ostringstream os; os << diff --git a/src/zm_rtsp_server_adts_source.h b/src/zm_rtsp_server_adts_source.h index d8516bc30..b6f906a12 100644 --- a/src/zm_rtsp_server_adts_source.h +++ b/src/zm_rtsp_server_adts_source.h @@ -28,8 +28,6 @@ class ADTS_ZoneMinderDeviceSource : public ZoneMinderDeviceSource { AVStream * stream, unsigned int queueSize ) { - Debug(1, "m_stream %p codecpar %p channels %d", - stream, stream->codecpar, stream->codecpar->channels); return new ADTS_ZoneMinderDeviceSource(env, std::move(monitor), stream, queueSize); }; protected: @@ -47,15 +45,16 @@ class ADTS_ZoneMinderDeviceSource : public ZoneMinderDeviceSource { virtual unsigned char* findMarker(unsigned char *frame, size_t size, size_t &length); */ public: - int samplingFrequency() { return m_stream->codecpar->sample_rate; }; + int samplingFrequency() { return +#if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) + m_stream->codecpar->sample_rate; +#else + m_stream->codec->sample_rate; +#endif + }; const char *configStr() { return config.c_str(); }; int numChannels() { - Debug(1, "this %p m_stream %p channels %d", - this, m_stream, channels); - Debug(1, "m_stream %p codecpar %p channels %d => %d", - m_stream, m_stream->codecpar, m_stream->codecpar->channels, channels); return channels; - return m_stream->codecpar->channels; } protected: diff --git a/src/zm_rtsp_server_h264_device_source.cpp b/src/zm_rtsp_server_h264_device_source.cpp index c10852196..71b641021 100644 --- a/src/zm_rtsp_server_h264_device_source.cpp +++ b/src/zm_rtsp_server_h264_device_source.cpp @@ -32,7 +32,13 @@ H264_ZoneMinderDeviceSource::H264_ZoneMinderDeviceSource( : H26X_ZoneMinderDeviceSource(env, std::move(monitor), stream, queueSize, repeatConfig, keepMarker) { // extradata appears to simply be the SPS and PPS NAL's - this->splitFrames(m_stream->codecpar->extradata, m_stream->codecpar->extradata_size); + this->splitFrames( +#if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) + m_stream->codecpar->extradata, m_stream->codecpar->extradata_size +#else + m_stream->codec->extradata, m_stream->codec->extradata_size +#endif + ); } // split packet into frames @@ -97,7 +103,13 @@ H265_ZoneMinderDeviceSource::H265_ZoneMinderDeviceSource( : H26X_ZoneMinderDeviceSource(env, std::move(monitor), stream, queueSize, repeatConfig, keepMarker) { // extradata appears to simply be the SPS and PPS NAL's - this->splitFrames(m_stream->codecpar->extradata, m_stream->codecpar->extradata_size); + this->splitFrames( +#if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) + m_stream->codecpar->extradata, m_stream->codecpar->extradata_size +#else + m_stream->codec->extradata, m_stream->codec->extradata_size +#endif + ); } // split packet in frames From 95fe689d58eceaf164009a1578b3ead6bd65b75b Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 14 Aug 2019 09:44:28 -0400 Subject: [PATCH 0084/1277] WIP, rabbit hole too deep --- db/zm_create.sql.in | 1 + db/zm_update-1.35.20.sql | 15 +++++++++++++++ src/zm_ffmpeg_camera.cpp | 2 ++ src/zm_ffmpeg_camera.h | 14 +++++++++++++- web/skins/classic/views/monitor.php | 9 ++++++++- 5 files changed, 39 insertions(+), 2 deletions(-) create mode 100644 db/zm_update-1.35.20.sql diff --git a/db/zm_create.sql.in b/db/zm_create.sql.in index f70ccc55d..ae8d500ab 100644 --- a/db/zm_create.sql.in +++ b/db/zm_create.sql.in @@ -468,6 +468,7 @@ CREATE TABLE `Monitors` ( `Port` varchar(8) NOT NULL default '', `SubPath` varchar(64) NOT NULL default '', `Path` varchar(255), + `SecondPath` varchar(255), `Options` varchar(255), `User` varchar(64), `Pass` varchar(64), diff --git a/db/zm_update-1.35.20.sql b/db/zm_update-1.35.20.sql new file mode 100644 index 000000000..275515dbf --- /dev/null +++ b/db/zm_update-1.35.20.sql @@ -0,0 +1,15 @@ +-- +-- Add SecondPath to Monitors +-- + +SET @s = (SELECT IF( + (SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS WHERE table_schema = DATABASE() + AND table_name = 'Monitors' + AND column_name = 'SecondPath' + ) > 0, +"SELECT 'Column SecondPath already exists in Monitors'", +"ALTER TABLE `Monitors` ADD `SecondPath` VARCHAR(255) AFTER `Path`" +)); + +PREPARE stmt FROM @s; +EXECUTE stmt; diff --git a/src/zm_ffmpeg_camera.cpp b/src/zm_ffmpeg_camera.cpp index afc4f79fa..9059ed477 100644 --- a/src/zm_ffmpeg_camera.cpp +++ b/src/zm_ffmpeg_camera.cpp @@ -93,6 +93,7 @@ static enum AVPixelFormat find_fmt_by_hw_type(const enum AVHWDeviceType type) { FfmpegCamera::FfmpegCamera( const Monitor *monitor, const std::string &p_path, + const std::string &p_second_path, const std::string &p_method, const std::string &p_options, int p_width, @@ -121,6 +122,7 @@ FfmpegCamera::FfmpegCamera( p_record_audio ), mPath(p_path), + mSecondPath(p_second_path), mMethod(p_method), mOptions(p_options), hwaccel_name(p_hwaccel_name), diff --git a/src/zm_ffmpeg_camera.h b/src/zm_ffmpeg_camera.h index bac1c42d3..6c268628a 100644 --- a/src/zm_ffmpeg_camera.h +++ b/src/zm_ffmpeg_camera.h @@ -34,6 +34,7 @@ typedef struct DecodeContext { class FfmpegCamera : public Camera { protected: std::string mPath; + std::string mSecondPath; std::string mMethod; std::string mOptions; @@ -42,7 +43,17 @@ class FfmpegCamera : public Camera { std::string hwaccel_device; int frameCount; - + +#if HAVE_LIBAVFORMAT + AVFormatContext *mFormatContext[2]; + int mFormats; + int mFormatIndex; + int mVideoStreamId; + int mAudioStreamId; + AVCodecContext *mVideoCodecContext; + AVCodecContext *mAudioCodecContext; + AVCodec *mVideoCodec; + AVCodec *mAudioCodec; _AVPIXELFORMAT imagePixFormat; bool use_hwaccel; //will default to on if hwaccel specified, will get turned off if there is a failure @@ -69,6 +80,7 @@ class FfmpegCamera : public Camera { FfmpegCamera( const Monitor *monitor, const std::string &path, + const std::string &second_path, const std::string &p_method, const std::string &p_options, int p_width, diff --git a/web/skins/classic/views/monitor.php b/web/skins/classic/views/monitor.php index d63a57b21..5c6f17a10 100644 --- a/web/skins/classic/views/monitor.php +++ b/web/skins/classic/views/monitor.php @@ -676,7 +676,10 @@ switch ( $name ) { { if ( ZM_HAS_V4L && $monitor->Type() == 'Local' ) { ?> - + + + + Method(), array('onchange'=>'submitTab', 'data-tab-name'=>$tab) ); ?> @@ -819,6 +822,10 @@ include('_monitor_source_nvsocket.php'); } if ( $monitor->Type() == 'Ffmpeg' ) { ?> + + + + From 86541779d9aebd11ee4ed890ba5c859fc65e50f6 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 3 Mar 2021 09:53:11 -0500 Subject: [PATCH 0085/1277] Add SecondFormatContext --- src/zm_camera.cpp | 9 +++++++-- src/zm_camera.h | 3 ++- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/src/zm_camera.cpp b/src/zm_camera.cpp index a4ebf485e..f130f827a 100644 --- a/src/zm_camera.cpp +++ b/src/zm_camera.cpp @@ -54,6 +54,7 @@ Camera::Camera( mVideoStream(nullptr), mAudioStream(nullptr), mFormatContext(nullptr), + mSecondFormatContext(nullptr), bytes(0) { linesize = width * colours; @@ -68,9 +69,13 @@ Camera::~Camera() { if ( mFormatContext ) { // Should also free streams avformat_free_context(mFormatContext); - mVideoStream = nullptr; - mAudioStream = nullptr; } + if ( mSecondFormatContext ) { + // Should also free streams + avformat_free_context(mSecondFormatContext); + } + mVideoStream = nullptr; + mAudioStream = nullptr; } AVStream *Camera::get_VideoStream() { diff --git a/src/zm_camera.h b/src/zm_camera.h index 10797c987..f4172eeb1 100644 --- a/src/zm_camera.h +++ b/src/zm_camera.h @@ -56,7 +56,8 @@ protected: AVCodecContext *mAudioCodecContext; AVStream *mVideoStream; AVStream *mAudioStream; - AVFormatContext *mFormatContext; + AVFormatContext *mFormatContext; // One for video, one for audio + AVFormatContext *mSecondFormatContext; // One for video, one for audio unsigned int bytes; public: From 4cc1da8b89c82ca8ccc7f815ecd46450ca1b8e6b Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 3 Mar 2021 09:53:58 -0500 Subject: [PATCH 0086/1277] Add SecondFormatContext and open it using an FFmpeg_Input --- src/zm_ffmpeg_camera.cpp | 58 +++++++++++++++++++++++++++------------- src/zm_ffmpeg_camera.h | 11 +------- 2 files changed, 41 insertions(+), 28 deletions(-) diff --git a/src/zm_ffmpeg_camera.cpp b/src/zm_ffmpeg_camera.cpp index 9059ed477..dc9bdd94a 100644 --- a/src/zm_ffmpeg_camera.cpp +++ b/src/zm_ffmpeg_camera.cpp @@ -19,6 +19,8 @@ #include "zm_ffmpeg_camera.h" +#include "zm_ffmpeg_input.h" +#include "zm_monitor.h" #include "zm_packet.h" #include "zm_signal.h" #include "zm_utils.h" @@ -186,15 +188,25 @@ int FfmpegCamera::PreCapture() { } int FfmpegCamera::Capture(ZMPacket &zm_packet) { - if ( !mCanCapture ) - return -1; + if (!mCanCapture) return -1; int ret; - if ( (ret = av_read_frame(mFormatContext, &packet)) < 0 ) { + AVStream *stream; + if ((mFormatContextPtr != mFormatContext) or !mSecondFormatContext) { + mFormatContextPtr = mFormatContext; + stream = mVideoStream; + Debug(1, "Using video input"); + } else { + mFormatContextPtr = mSecondFormatContext; + stream = mAudioStream; + Debug(1, "Using audio input"); + } + + if ( (ret = av_read_frame(mFormatContextPtr, &packet)) < 0 ) { if ( // Check if EOF. - (ret == AVERROR_EOF || (mFormatContext->pb && mFormatContext->pb->eof_reached)) || + (ret == AVERROR_EOF || (mFormatContextPtr->pb && mFormatContextPtr->pb->eof_reached)) || // Check for Connection failure. (ret == -110) ) { @@ -206,16 +218,16 @@ int FfmpegCamera::Capture(ZMPacket &zm_packet) { } return -1; } - ZM_DUMP_STREAM_PACKET(mFormatContext->streams[packet.stream_index], packet, "ffmpeg_camera in"); + ZM_DUMP_STREAM_PACKET(stream, packet, "ffmpeg_camera in"); #if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) - zm_packet.codec_type = mFormatContext->streams[packet.stream_index]->codecpar->codec_type; + zm_packet.codec_type = stream->codecpar->codec_type; #else - zm_packet.codec_type = mFormatContext->streams[packet.stream_index]->codec->codec_type; + zm_packet.codec_type = stream->codec->codec_type; #endif bytes += packet.size; zm_packet.set_packet(&packet); - zm_packet.pts = av_rescale_q(packet.pts, mFormatContext->streams[packet.stream_index]->time_base, AV_TIME_BASE_Q); + zm_packet.pts = av_rescale_q(packet.pts, stream->time_base, AV_TIME_BASE_Q); zm_av_packet_unref(&packet); return 1; } // FfmpegCamera::Capture @@ -232,7 +244,7 @@ int FfmpegCamera::OpenFfmpeg() { // Open the input, not necessarily a file #if !LIBAVFORMAT_VERSION_CHECK(53, 2, 0, 4, 0) - if ( av_open_input_file(&mFormatContext, mPath.c_str(), nullptr, 0, nullptr) != 0 ) + if (av_open_input_file(&mFormatContext, mPath.c_str(), nullptr, 0, nullptr) != 0) #else // Handle options AVDictionary *opts = nullptr; @@ -261,7 +273,6 @@ int FfmpegCamera::OpenFfmpeg() { Warning("Could not set rtsp_transport method '%s'", method.c_str()); } } // end if RTSP - // #av_dict_set(&opts, "timeout", "10000000", 0); // in microseconds. Debug(1, "Calling avformat_open_input for %s", mPath.c_str()); @@ -478,13 +489,25 @@ int FfmpegCamera::OpenFfmpeg() { } zm_dump_codec(mVideoCodecContext); + if (mAudioStreamId == -1 and !monitor->GetSecondPath().empty()) { + Debug(1, "Trying secondary stream at %s", monitor->GetSecondPath().c_str()); + FFmpeg_Input *second_input = new FFmpeg_Input(); + if (second_input->Open(monitor->GetSecondPath().c_str()) > 0) { + mSecondFormatContext = second_input->get_format_context(); + mAudioStreamId = second_input->get_audio_stream_id(); + mAudioStream = second_input->get_audio_stream(); + } else { + Warning("Failed to open secondary input"); + } + } // end if have audio stream + if ( mAudioStreamId >= 0 ) { AVCodec *mAudioCodec = nullptr; if ( (mAudioCodec = avcodec_find_decoder( #if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) - mFormatContext->streams[mAudioStreamId]->codecpar->codec_id + mAudioStream->codecpar->codec_id #else - mFormatContext->streams[mAudioStreamId]->codec->codec_id + mAudioStream->codec->codec_id #endif )) == nullptr ) { Debug(1, "Can't find codec for audio stream from %s", mPath.c_str()); @@ -493,11 +516,10 @@ int FfmpegCamera::OpenFfmpeg() { mAudioCodecContext = avcodec_alloc_context3(mAudioCodec); avcodec_parameters_to_context( mAudioCodecContext, - mFormatContext->streams[mAudioStreamId]->codecpar + mAudioStream->codecpar ); #else - mAudioCodecContext = mFormatContext->streams[mAudioStreamId]->codec; - // = avcodec_alloc_context3(mAudioCodec); + mAudioCodecContext = mAudioStream->codec; #endif zm_dump_stream_format(mFormatContext, mAudioStreamId, 0, 0); @@ -510,9 +532,9 @@ int FfmpegCamera::OpenFfmpeg() { { Error("Unable to open codec for audio stream from %s", mPath.c_str()); return -1; - } // end if opened - } // end if found decoder - } // end if have audio stream + } // end if opened + } // end if found decoder + } // end if mAudioStreamId if ( ((unsigned int)mVideoCodecContext->width != width) diff --git a/src/zm_ffmpeg_camera.h b/src/zm_ffmpeg_camera.h index 6c268628a..2701fb0d6 100644 --- a/src/zm_ffmpeg_camera.h +++ b/src/zm_ffmpeg_camera.h @@ -44,17 +44,8 @@ class FfmpegCamera : public Camera { int frameCount; -#if HAVE_LIBAVFORMAT - AVFormatContext *mFormatContext[2]; - int mFormats; - int mFormatIndex; - int mVideoStreamId; - int mAudioStreamId; - AVCodecContext *mVideoCodecContext; - AVCodecContext *mAudioCodecContext; - AVCodec *mVideoCodec; - AVCodec *mAudioCodec; _AVPIXELFORMAT imagePixFormat; + AVFormatContext *mFormatContextPtr; bool use_hwaccel; //will default to on if hwaccel specified, will get turned off if there is a failure #if HAVE_LIBAVUTIL_HWCONTEXT_H From e31c87193de537d4f6a25a9a300336fe8ed64c08 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 3 Mar 2021 09:54:24 -0500 Subject: [PATCH 0087/1277] Make Open() return 1 instead of 0 --- src/zm_ffmpeg_input.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/zm_ffmpeg_input.cpp b/src/zm_ffmpeg_input.cpp index 2b3bcfd74..d717a4c3a 100644 --- a/src/zm_ffmpeg_input.cpp +++ b/src/zm_ffmpeg_input.cpp @@ -118,11 +118,11 @@ int FFmpeg_Input::Open(const char *filepath) { } // end foreach stream if ( video_stream_id == -1 ) - Error("Unable to locate video stream in %s", filepath); + Warning("Unable to locate video stream in %s", filepath); if ( audio_stream_id == -1 ) Debug(3, "Unable to locate audio stream in %s", filepath); - return 0; + return 1; } // end int FFmpeg_Input::Open( const char * filepath ) int FFmpeg_Input::Close( ) { From 9310825d9301396c6c22b1c8ad704968e7fd6132 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 3 Mar 2021 09:54:38 -0500 Subject: [PATCH 0088/1277] Add get_video_stream and get_audio_stream. --- src/zm_ffmpeg_input.h | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/src/zm_ffmpeg_input.h b/src/zm_ffmpeg_input.h index 6a339f081..5ca5f07dc 100644 --- a/src/zm_ffmpeg_input.h +++ b/src/zm_ffmpeg_input.h @@ -36,7 +36,13 @@ class FFmpeg_Input { int get_audio_stream_id() const { return audio_stream_id; } - AVFormatContext * get_format_context() { return input_format_context; }; + AVStream *get_video_stream() { + return ( video_stream_id >= 0 ) ? input_format_context->streams[video_stream_id] : nullptr; + } + AVStream *get_audio_stream() { + return ( audio_stream_id >= 0 ) ? input_format_context->streams[audio_stream_id] : nullptr; + } + AVFormatContext *get_format_context() { return input_format_context; }; private: typedef struct { From a01a24ec365880032b2677f3781bed3a138b2508 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 3 Mar 2021 09:54:58 -0500 Subject: [PATCH 0089/1277] Add loading SecondPath in monitor --- src/zm_monitor.cpp | 6 ++++-- src/zm_monitor.h | 2 ++ 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 9e0e55475..329ccb06c 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -78,7 +78,7 @@ std::string load_monitor_sql = "SELECT `Id`, `Name`, `ServerId`, `StorageId`, `Type`, `Function`+0, `Enabled`, `DecodingEnabled`, " "`LinkedMonitors`, `AnalysisFPSLimit`, `AnalysisUpdateDelay`, `MaxFPS`, `AlarmMaxFPS`," "`Device`, `Channel`, `Format`, `V4LMultiBuffer`, `V4LCapturesPerFrame`, " // V4L Settings -"`Protocol`, `Method`, `Options`, `User`, `Pass`, `Host`, `Port`, `Path`, `Width`, `Height`, `Colours`, `Palette`, `Orientation`+0, `Deinterlacing`, " +"`Protocol`, `Method`, `Options`, `User`, `Pass`, `Host`, `Port`, `Path`, `SecondPath`, `Width`, `Height`, `Colours`, `Palette`, `Orientation`+0, `Deinterlacing`, " "`DecoderHWAccelName`, `DecoderHWAccelDevice`, `RTSPDescribe`, " "`SaveJPEGs`, `VideoWriter`, `EncoderParameters`, " "`OutputCodec`, `Encoder`, `OutputContainer`, " @@ -420,7 +420,7 @@ Monitor::Monitor() "SELECT Id, Name, ServerId, StorageId, Type, Function+0, Enabled, DecodingEnabled, LinkedMonitors, " "AnalysisFPSLimit, AnalysisUpdateDelay, MaxFPS, AlarmMaxFPS," "Device, Channel, Format, V4LMultiBuffer, V4LCapturesPerFrame, " // V4L Settings - "Protocol, Method, Options, User, Pass, Host, Port, Path, Width, Height, Colours, Palette, Orientation+0, Deinterlacing, RTSPDescribe, " + "Protocol, Method, Options, User, Pass, Host, Port, Path, SecondPath, Width, Height, Colours, Palette, Orientation+0, Deinterlacing, RTSPDescribe, " "SaveJPEGs, VideoWriter, EncoderParameters, "OutputCodec, Encoder, OutputContainer," "RecordAudio, " @@ -512,6 +512,7 @@ void Monitor::Load(MYSQL_ROW dbrow, bool load_zones=true, Purpose p = QUERY) { host = dbrow[col] ? dbrow[col] : ""; col++; port = dbrow[col] ? dbrow[col] : ""; col++; path = dbrow[col] ? dbrow[col] : ""; col++; + second_path = dbrow[col] ? dbrow[col] : ""; col++; camera_width = atoi(dbrow[col]); col++; camera_height = atoi(dbrow[col]); col++; @@ -771,6 +772,7 @@ void Monitor::LoadCamera() { case FFMPEG: { camera = ZM::make_unique(this, path, + second_path, method, options, camera_width, diff --git a/src/zm_monitor.h b/src/zm_monitor.h index b4cc577a0..f302a439b 100644 --- a/src/zm_monitor.h +++ b/src/zm_monitor.h @@ -248,6 +248,7 @@ protected: std::string user; std::string pass; std::string path; + std::string second_path; char device[64]; int palette; @@ -490,6 +491,7 @@ public: AVStream *GetVideoStream() const { return camera ? camera->get_VideoStream() : nullptr; }; AVCodecContext *GetVideoCodecContext() const { return camera ? camera->get_VideoCodecContext() : nullptr; }; + const std::string GetSecondPath() const { return second_path; }; const std::string GetVideoFifoPath() const { return shared_data ? shared_data->video_fifo_path : ""; }; const std::string GetAudioFifoPath() const { return shared_data ? shared_data->audio_fifo_path : ""; }; const std::string GetRTSPStreamName() const { return rtsp_streamname; }; From ce4e133c94438b52e14e91dddc711b8a000dcbba Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 3 Mar 2021 09:55:23 -0500 Subject: [PATCH 0090/1277] Add loading SecondPath in monitor --- web/includes/Monitor.php | 1 + web/skins/classic/css/base/views/monitor.css | 2 ++ web/skins/classic/views/monitor.php | 2 +- 3 files changed, 4 insertions(+), 1 deletion(-) diff --git a/web/includes/Monitor.php b/web/includes/Monitor.php index ad118b73f..189adb08f 100644 --- a/web/includes/Monitor.php +++ b/web/includes/Monitor.php @@ -53,6 +53,7 @@ class Monitor extends ZM_Object { 'Port' => '', 'SubPath' => '', 'Path' => null, + 'SecondPath' => null, 'Options' => null, 'User' => null, 'Pass' => null, diff --git a/web/skins/classic/css/base/views/monitor.css b/web/skins/classic/css/base/views/monitor.css index 29af3612a..1592d5565 100644 --- a/web/skins/classic/css/base/views/monitor.css +++ b/web/skins/classic/css/base/views/monitor.css @@ -11,6 +11,8 @@ textarea, input[name="newMonitor[Name]"], +input[name="newMonitor[Path]"], +input[name="newMonitor[SecondPath]"], input[name="newMonitor[LabelFormat]"], input[name="newMonitor[ControlDevice]"], input[name="newMonitor[ControlAddress]"] { diff --git a/web/skins/classic/views/monitor.php b/web/skins/classic/views/monitor.php index 5c6f17a10..0b629e0bd 100644 --- a/web/skins/classic/views/monitor.php +++ b/web/skins/classic/views/monitor.php @@ -823,7 +823,7 @@ include('_monitor_source_nvsocket.php'); if ( $monitor->Type() == 'Ffmpeg' ) { ?> - + From a90ec5b15dbbfadb6bc0dd477354f028de60dee6 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 3 Mar 2021 12:03:36 -0500 Subject: [PATCH 0091/1277] Deprecate video_in_stream_index and audio_in_stream_index as they are not useful --- src/zm_videostore.cpp | 8 +------- src/zm_videostore.h | 2 -- 2 files changed, 1 insertion(+), 9 deletions(-) diff --git a/src/zm_videostore.cpp b/src/zm_videostore.cpp index a2df34e35..07b03df91 100644 --- a/src/zm_videostore.cpp +++ b/src/zm_videostore.cpp @@ -49,8 +49,6 @@ VideoStore::VideoStore( oc(nullptr), video_out_stream(nullptr), audio_out_stream(nullptr), - video_in_stream_index(-1), - audio_in_stream_index(-1), video_out_codec(nullptr), video_in_ctx(p_video_in_ctx), video_out_ctx(nullptr), @@ -123,8 +121,6 @@ bool VideoStore::open() { #if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) zm_dump_codecpar(video_in_stream->codecpar); #endif - video_in_stream_index = video_in_stream->index; - if ( monitor->GetOptVideoWriter() == Monitor::PASSTHROUGH ) { // Don't care what codec, just copy parameters video_out_ctx = avcodec_alloc_context3(nullptr); @@ -950,9 +946,7 @@ int VideoStore::writePacket(ZMPacket *ipkt) { } else if ( ipkt->codec_type == AVMEDIA_TYPE_AUDIO ) { return writeAudioFramePacket(ipkt); } - Error("Unknown stream type in packet (%d) input video stream is (%d) and audio is (%d)", - ipkt->packet.stream_index, video_in_stream_index, ( audio_in_stream ? audio_in_stream_index : -1 ) - ); + Error("Unknown stream type in packet (%d)", ipkt->codec_type); return 0; } diff --git a/src/zm_videostore.h b/src/zm_videostore.h index d350db65b..6430d92c3 100644 --- a/src/zm_videostore.h +++ b/src/zm_videostore.h @@ -40,8 +40,6 @@ class VideoStore { AVFormatContext *oc; AVStream *video_out_stream; AVStream *audio_out_stream; - int video_in_stream_index; - int audio_in_stream_index; AVCodec *video_out_codec; AVCodecContext *video_in_ctx; From b87d859f729821a010d18d1183cd3949973d907f Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 3 Mar 2021 12:06:34 -0500 Subject: [PATCH 0092/1277] Set the packet's stream_index to the packetqueue stream. Rename get_ functions to get --- src/zm_camera.cpp | 2 +- src/zm_camera.h | 12 ++++++------ src/zm_ffmpeg_camera.cpp | 1 + src/zm_libvlc_camera.cpp | 2 ++ src/zm_libvnc_camera.cpp | 3 ++- src/zm_local_camera.cpp | 3 ++- src/zm_monitor.cpp | 25 +++++++++++++------------ src/zm_monitor.h | 8 ++++---- src/zm_packet.cpp | 3 +++ src/zm_packet.h | 1 + src/zm_remote_camera_http.cpp | 3 ++- src/zm_remote_camera_rtsp.cpp | 1 + 12 files changed, 38 insertions(+), 26 deletions(-) diff --git a/src/zm_camera.cpp b/src/zm_camera.cpp index f130f827a..cc3d34d40 100644 --- a/src/zm_camera.cpp +++ b/src/zm_camera.cpp @@ -78,7 +78,7 @@ Camera::~Camera() { mAudioStream = nullptr; } -AVStream *Camera::get_VideoStream() { +AVStream *Camera::getVideoStream() { if ( !mVideoStream ) { if ( !mFormatContext ) mFormatContext = avformat_alloc_context(); diff --git a/src/zm_camera.h b/src/zm_camera.h index f4172eeb1..def377c4d 100644 --- a/src/zm_camera.h +++ b/src/zm_camera.h @@ -120,12 +120,12 @@ public: //return (type == FFMPEG_SRC )||(type == REMOTE_SRC); } - virtual AVStream *get_VideoStream(); - virtual AVStream *get_AudioStream() { return mAudioStream; }; - virtual AVCodecContext *get_VideoCodecContext() { return mVideoCodecContext; }; - virtual AVCodecContext *get_AudioCodecContext() { return mAudioCodecContext; }; - int get_VideoStreamId() { return mVideoStreamId; }; - int get_AudioStreamId() { return mAudioStreamId; }; + virtual AVStream *getVideoStream(); + virtual AVStream *getAudioStream() { return mAudioStream; }; + virtual AVCodecContext *getVideoCodecContext() { return mVideoCodecContext; }; + virtual AVCodecContext *getAudioCodecContext() { return mAudioCodecContext; }; + int getVideoStreamId() { return mVideoStreamId; }; + int getAudioStreamId() { return mAudioStreamId; }; virtual int PrimeCapture() { return 0; } virtual int PreCapture() = 0; diff --git a/src/zm_ffmpeg_camera.cpp b/src/zm_ffmpeg_camera.cpp index dc9bdd94a..69ce43a0e 100644 --- a/src/zm_ffmpeg_camera.cpp +++ b/src/zm_ffmpeg_camera.cpp @@ -227,6 +227,7 @@ int FfmpegCamera::Capture(ZMPacket &zm_packet) { #endif bytes += packet.size; zm_packet.set_packet(&packet); + zm_packet.stream = stream; zm_packet.pts = av_rescale_q(packet.pts, stream->time_base, AV_TIME_BASE_Q); zm_av_packet_unref(&packet); return 1; diff --git a/src/zm_libvlc_camera.cpp b/src/zm_libvlc_camera.cpp index fd4d077b8..90d43da61 100644 --- a/src/zm_libvlc_camera.cpp +++ b/src/zm_libvlc_camera.cpp @@ -279,6 +279,8 @@ int LibvlcCamera::Capture( ZMPacket &zm_packet ) { mLibvlcData.mutex.lock(); zm_packet.image->Assign(width, height, colours, subpixelorder, mLibvlcData.buffer, width * height * mBpp); + zm_packet.packet.stream_index = mVideoStreamId; + zm_packet.stream = mVideoStream; mLibvlcData.newImage.setValueImmediate(false); mLibvlcData.mutex.unlock(); diff --git a/src/zm_libvnc_camera.cpp b/src/zm_libvnc_camera.cpp index c00a37e30..7e1d260ab 100644 --- a/src/zm_libvnc_camera.cpp +++ b/src/zm_libvnc_camera.cpp @@ -178,7 +178,7 @@ int VncCamera::PrimeCapture() { Warning("Specified dimensions do not match screen size monitor: (%dx%d) != vnc: (%dx%d)", width, height, mRfb->width, mRfb->height); } - get_VideoStream(); + getVideoStream(); return 1; } @@ -207,6 +207,7 @@ int VncCamera::Capture(ZMPacket &zm_packet) { zm_packet.keyframe = 1; zm_packet.codec_type = AVMEDIA_TYPE_VIDEO; zm_packet.packet.stream_index = mVideoStreamId; + zm_packet.stream = mVideoStream; uint8_t *directbuffer = zm_packet.image->WriteBuffer(width, height, colours, subpixelorder); Debug(1, "scale src %p, %d, dest %p %d %d %dx%d %dx%d", mVncData.buffer, diff --git a/src/zm_local_camera.cpp b/src/zm_local_camera.cpp index f515539a1..b5c0ab665 100644 --- a/src/zm_local_camera.cpp +++ b/src/zm_local_camera.cpp @@ -1975,7 +1975,7 @@ int LocalCamera::Contrast( int p_contrast ) { } int LocalCamera::PrimeCapture() { - get_VideoStream(); + getVideoStream(); if ( !device_prime ) return 1; @@ -2240,6 +2240,7 @@ int LocalCamera::Capture(ZMPacket &zm_packet) { } // end if doing conversion or not zm_packet.packet.stream_index = mVideoStreamId; + zm_packet.stream = mVideoStream; zm_packet.codec_type = AVMEDIA_TYPE_VIDEO; zm_packet.keyframe = 1; return 1; diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 329ccb06c..19ae3ac87 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -2259,7 +2259,7 @@ bool Monitor::Analyse() { if ( snap->keyframe ) { // avcodec strips out important nals that describe the stream and // stick them in extradata. Need to send them along with keyframes - AVStream *stream = camera->get_VideoStream(); + AVStream *stream = camera->getVideoStream(); video_fifo->write( static_cast(stream->codecpar->extradata), stream->codecpar->extradata_size); @@ -2557,12 +2557,13 @@ int Monitor::Capture() { packet->packet.stream_index, video_stream_id, packetqueue.packet_count(video_stream_id), ( event ? 1 : 0 ) ); if (packet->codec_type == AVMEDIA_TYPE_VIDEO) { + packet->packet.stream_index = video_stream_id; // Convert to packetQueue's index if (video_fifo) { if ( packet->keyframe ) { // avcodec strips out important nals that describe the stream and // stick them in extradata. Need to send them along with keyframes #if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) - AVStream *stream = camera->get_VideoStream(); + AVStream *stream = camera->getVideoStream(); video_fifo->write( static_cast(stream->codecpar->extradata), stream->codecpar->extradata_size, @@ -2571,17 +2572,15 @@ int Monitor::Capture() { } video_fifo->writePacket(*packet); } - } else if (packet->codec_type == AVMEDIA_TYPE_VIDEO) { + } else if (packet->codec_type == AVMEDIA_TYPE_AUDIO) { if (audio_fifo) audio_fifo->writePacket(*packet); - } - //if ( (packet->packet.stream_index != video_stream_id) and ! packet->image ) { - if ( (packet->codec_type == AVMEDIA_TYPE_AUDIO) and ! packet->image ) { // Only queue if we have some video packets in there. Should push this logic into packetqueue - if ( record_audio and (packetqueue.packet_count(video_stream_id) or event) ) { + if (record_audio and (packetqueue.packet_count(video_stream_id) or event)) { packet->image_index=-1; Debug(2, "Queueing audio packet"); + packet->packet.stream_index = audio_stream_id; // Convert to packetQueue's index packetqueue.queuePacket(packet); } else { Debug(4, "Not Queueing audio packet"); @@ -2590,6 +2589,8 @@ int Monitor::Capture() { // Don't update last_write_index because that is used for live streaming //shared_data->last_write_time = image_buffer[index].timestamp->tv_sec; return 1; + } else { + Debug(1, "Unknown codec type %d", packet->codec_type); } // end if audio if ( !packet->image ) { @@ -2602,7 +2603,7 @@ int Monitor::Capture() { // We don't actually care about camera colours, pixel order etc. We care about the desired settings // //capture_image = packet->image = new Image(width, height, camera->Colours(), camera->SubpixelOrder()); - int ret = packet->decode(camera->get_VideoCodecContext()); + int ret = packet->decode(camera->getVideoCodecContext()); if ( ret < 0 ) { Error("decode failed"); } else if ( ret == 0 ) { @@ -2985,11 +2986,11 @@ unsigned int Monitor::SubpixelOrder() const { return camera->SubpixelOrder(); } int Monitor::PrimeCapture() { int ret = camera->PrimeCapture(); if ( ret > 0 ) { - if ( -1 != camera->get_VideoStreamId() ) { + if ( -1 != camera->getVideoStreamId() ) { video_stream_id = packetqueue.addStream(); } - if ( -1 != camera->get_AudioStreamId() ) { + if ( -1 != camera->getAudioStreamId() ) { audio_stream_id = packetqueue.addStream(); packetqueue.addStream(); shared_data->audio_frequency = camera->getFrequency(); @@ -3001,7 +3002,7 @@ int Monitor::PrimeCapture() { if (rtsp_server) { if (video_stream_id >= 0) { - AVStream *videoStream = camera->get_VideoStream(); + AVStream *videoStream = camera->getVideoStream(); snprintf(shared_data->video_fifo_path, sizeof(shared_data->video_fifo_path)-1, "%s/video_fifo_%d.%s", staticConfig.PATH_SOCKS.c_str(), id, @@ -3014,7 +3015,7 @@ int Monitor::PrimeCapture() { video_fifo = new Fifo(shared_data->video_fifo_path, true); } if (record_audio and (audio_stream_id >= 0)) { - AVStream *audioStream = camera->get_AudioStream(); + AVStream *audioStream = camera->getAudioStream(); snprintf(shared_data->audio_fifo_path, sizeof(shared_data->audio_fifo_path)-1, "%s/video_fifo_%d.%s", staticConfig.PATH_SOCKS.c_str(), id, #if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) diff --git a/src/zm_monitor.h b/src/zm_monitor.h index f302a439b..f26e60e03 100644 --- a/src/zm_monitor.h +++ b/src/zm_monitor.h @@ -486,10 +486,10 @@ public: int32_t GetImageBufferCount() const { return image_buffer_count; }; State GetState() const { return (State)shared_data->state; } - AVStream *GetAudioStream() const { return camera ? camera->get_AudioStream() : nullptr; }; - AVCodecContext *GetAudioCodecContext() const { return camera ? camera->get_AudioCodecContext() : nullptr; }; - AVStream *GetVideoStream() const { return camera ? camera->get_VideoStream() : nullptr; }; - AVCodecContext *GetVideoCodecContext() const { return camera ? camera->get_VideoCodecContext() : nullptr; }; + AVStream *GetAudioStream() const { return camera ? camera->getAudioStream() : nullptr; }; + AVCodecContext *GetAudioCodecContext() const { return camera ? camera->getAudioCodecContext() : nullptr; }; + AVStream *GetVideoStream() const { return camera ? camera->getVideoStream() : nullptr; }; + AVCodecContext *GetVideoCodecContext() const { return camera ? camera->getVideoCodecContext() : nullptr; }; const std::string GetSecondPath() const { return second_path; }; const std::string GetVideoFifoPath() const { return shared_data ? shared_data->video_fifo_path : ""; }; diff --git a/src/zm_packet.cpp b/src/zm_packet.cpp index 133747ea2..d6174cee8 100644 --- a/src/zm_packet.cpp +++ b/src/zm_packet.cpp @@ -149,6 +149,9 @@ int ZMPacket::decode(AVCodecContext *ctx) { in_frame = zm_av_frame_alloc(); } + // packets are always stored in AV_TIME_BASE_Q so need to convert to codec time base + //av_packet_rescale_ts(&packet, AV_TIME_BASE_Q, ctx->time_base); + int ret = zm_send_packet_receive_frame(ctx, in_frame, packet); if ( ret < 0 ) { if ( AVERROR(EAGAIN) != ret ) { diff --git a/src/zm_packet.h b/src/zm_packet.h index 6b567bc4e..abd0f3486 100644 --- a/src/zm_packet.h +++ b/src/zm_packet.h @@ -38,6 +38,7 @@ class ZMPacket { std::recursive_mutex mutex; int keyframe; + AVStream *stream; // Input stream AVPacket packet; // Input packet, undecoded AVFrame *in_frame; // Input image, decoded Theoretically only filled if needed. AVFrame *out_frame; // output image, Only filled if needed. diff --git a/src/zm_remote_camera_http.cpp b/src/zm_remote_camera_http.cpp index 98625aab8..8342c2766 100644 --- a/src/zm_remote_camera_http.cpp +++ b/src/zm_remote_camera_http.cpp @@ -1047,7 +1047,7 @@ int RemoteCameraHttp::PrimeCapture() { mode = SINGLE_IMAGE; buffer.clear(); } - get_VideoStream(); + getVideoStream(); return 1; } @@ -1088,6 +1088,7 @@ int RemoteCameraHttp::Capture(ZMPacket &packet) { packet.keyframe = 1; packet.codec_type = AVMEDIA_TYPE_VIDEO; packet.packet.stream_index = mVideoStreamId; + packet.stream = mVideoStream; switch ( format ) { case JPEG : diff --git a/src/zm_remote_camera_rtsp.cpp b/src/zm_remote_camera_rtsp.cpp index dbd489e18..44d5046af 100644 --- a/src/zm_remote_camera_rtsp.cpp +++ b/src/zm_remote_camera_rtsp.cpp @@ -309,6 +309,7 @@ int RemoteCameraRtsp::Capture(ZMPacket &zm_packet) { #endif } zm_packet.codec_type = mVideoCodecContext->codec_type; + zm_packet.stream = mVideoStream; frameComplete = true; Debug(2, "Frame: %d - %d/%d", frameCount, bytes_consumed, buffer.size()); packet->data = nullptr; From fda0656f844be3f1910bf705da103d2c704d1d64 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 3 Mar 2021 12:07:33 -0500 Subject: [PATCH 0093/1277] Bump version to 1.35.20 to pick up SecondPath db change --- distros/redhat/zoneminder.spec | 2 +- version | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/distros/redhat/zoneminder.spec b/distros/redhat/zoneminder.spec index d1da6e852..9211793e9 100644 --- a/distros/redhat/zoneminder.spec +++ b/distros/redhat/zoneminder.spec @@ -28,7 +28,7 @@ %global _hardened_build 1 Name: zoneminder -Version: 1.35.19 +Version: 1.35.20 Release: 1%{?dist} Summary: A camera monitoring and analysis tool Group: System Environment/Daemons diff --git a/version b/version index 71fcfce81..a3dc39092 100644 --- a/version +++ b/version @@ -1 +1 @@ -1.35.19 +1.35.20 From 3149ba276f3ad5f7ea61299fa58e82012b7b937b Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 3 Mar 2021 12:44:45 -0500 Subject: [PATCH 0094/1277] audio_in_stream_id is dprecated --- src/zm_videostore.cpp | 1 - 1 file changed, 1 deletion(-) diff --git a/src/zm_videostore.cpp b/src/zm_videostore.cpp index 07b03df91..1ffc69e90 100644 --- a/src/zm_videostore.cpp +++ b/src/zm_videostore.cpp @@ -300,7 +300,6 @@ bool VideoStore::open() { if ( audio_in_stream and audio_in_ctx ) { Debug(2, "Have audio_in_stream %p", audio_in_stream); - audio_in_stream_index = audio_in_stream->index; if ( #if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) From eaaf04420ad3fbfde023f2ebc0fa12192f548f9f Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 3 Mar 2021 12:45:05 -0500 Subject: [PATCH 0095/1277] Keep track of stream last_pts. So we can at least try to sync streams --- src/zm_camera.cpp | 2 ++ src/zm_camera.h | 2 ++ src/zm_ffmpeg_camera.cpp | 28 +++++++++++++++++++++++----- 3 files changed, 27 insertions(+), 5 deletions(-) diff --git a/src/zm_camera.cpp b/src/zm_camera.cpp index cc3d34d40..aa90a6be4 100644 --- a/src/zm_camera.cpp +++ b/src/zm_camera.cpp @@ -55,6 +55,8 @@ Camera::Camera( mAudioStream(nullptr), mFormatContext(nullptr), mSecondFormatContext(nullptr), + mLastVideoPTS(0), + mLastAudioPTS(0), bytes(0) { linesize = width * colours; diff --git a/src/zm_camera.h b/src/zm_camera.h index def377c4d..5fd2a4bc9 100644 --- a/src/zm_camera.h +++ b/src/zm_camera.h @@ -58,6 +58,8 @@ protected: AVStream *mAudioStream; AVFormatContext *mFormatContext; // One for video, one for audio AVFormatContext *mSecondFormatContext; // One for video, one for audio + int64_t mLastVideoPTS; + int64_t mLastAudioPTS; unsigned int bytes; public: diff --git a/src/zm_ffmpeg_camera.cpp b/src/zm_ffmpeg_camera.cpp index 69ce43a0e..73200fe57 100644 --- a/src/zm_ffmpeg_camera.cpp +++ b/src/zm_ffmpeg_camera.cpp @@ -193,14 +193,24 @@ int FfmpegCamera::Capture(ZMPacket &zm_packet) { int ret; AVStream *stream; - if ((mFormatContextPtr != mFormatContext) or !mSecondFormatContext) { - mFormatContextPtr = mFormatContext; - stream = mVideoStream; - Debug(1, "Using video input"); - } else { + if ( mSecondFormatContext and + ( + av_rescale_q(mLastAudioPTS, mAudioStream->time_base, AV_TIME_BASE_Q) + < + av_rescale_q(mLastVideoPTS, mVideoStream->time_base, AV_TIME_BASE_Q) + ) ) { + // if audio stream is behind video stream, then read from audio, otherwise video mFormatContextPtr = mSecondFormatContext; stream = mAudioStream; Debug(1, "Using audio input"); + } else { + + mFormatContextPtr = mFormatContext; + stream = mVideoStream; + Debug(1, "Using video input beacuse %" PRId64 " >= %" PRId64, + av_rescale_q(mLastAudioPTS, mAudioStream->time_base, AV_TIME_BASE_Q), + av_rescale_q(mLastVideoPTS, mVideoStream->time_base, AV_TIME_BASE_Q) + ); } if ( (ret = av_read_frame(mFormatContextPtr, &packet)) < 0 ) { @@ -229,7 +239,15 @@ int FfmpegCamera::Capture(ZMPacket &zm_packet) { zm_packet.set_packet(&packet); zm_packet.stream = stream; zm_packet.pts = av_rescale_q(packet.pts, stream->time_base, AV_TIME_BASE_Q); + if ( packet.pts != AV_NOPTS_VALUE ) { + if ( stream == mVideoStream ) { + mLastVideoPTS = packet.pts; + } else { + mLastAudioPTS = packet.pts; + } + } zm_av_packet_unref(&packet); + return 1; } // FfmpegCamera::Capture From 8aebcf7337a4dbf9f9e63e42accad29f0b1d5732 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 3 Mar 2021 12:51:27 -0500 Subject: [PATCH 0096/1277] Fix ['mid'] not being defined. Use instead --- web/skins/classic/views/monitor.php | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/web/skins/classic/views/monitor.php b/web/skins/classic/views/monitor.php index 0b629e0bd..1806a1745 100644 --- a/web/skins/classic/views/monitor.php +++ b/web/skins/classic/views/monitor.php @@ -36,6 +36,7 @@ if ( !$Server ) { $monitor = null; if ( !empty($_REQUEST['mid']) ) { + $nextId = validInt($_REQUEST['mid']); $monitor = new ZM\Monitor($_REQUEST['mid']); if ( $monitor and ZM_OPT_X10 ) $x10Monitor = dbFetchOne('SELECT * FROM TriggersX10 WHERE MonitorId = ?', NULL, array($_REQUEST['mid'])); @@ -466,8 +467,8 @@ if ( canEdit('Monitors') ) {
- - + +
$value ) { From de0bbf7e046879dfcbc8b98a6e2ba35b73e6caff Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 3 Mar 2021 14:23:35 -0500 Subject: [PATCH 0097/1277] Instead of fatal error triggering, turn off file logging and log the error elsewhere --- web/includes/logger.php | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/web/includes/logger.php b/web/includes/logger.php index c6a629327..02da47a15 100644 --- a/web/includes/logger.php +++ b/web/includes/logger.php @@ -397,7 +397,8 @@ class Logger { if ( !error_log($message."\n", 3, $this->logFile) ) { if ( strnatcmp(phpversion(), '5.2.0') >= 0 ) { $error = error_get_last(); - trigger_error("Can't write to log file '".$this->logFile."': ".$error['message'].' @ '.$error['file'].'/'.$error['line'], E_USER_ERROR); + $this->fileLevel = self::NOLOG; + Error("Can't write to log file '".$this->logFile."': ".$error['message'].' @ '.$error['file'].'/'.$error['line'], E_USER_ERROR); } } } else if ( $this->logFd ) { From 8a33fb666509033f3d5edef755cdcc8096d7b3a0 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 3 Mar 2021 15:23:58 -0500 Subject: [PATCH 0098/1277] Include codec_type in stream and packet dumps --- src/zm_ffmpeg.cpp | 8 ++++++-- src/zm_ffmpeg.h | 3 ++- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/src/zm_ffmpeg.cpp b/src/zm_ffmpeg.cpp index 6ff053c34..352e45b85 100644 --- a/src/zm_ffmpeg.cpp +++ b/src/zm_ffmpeg.cpp @@ -257,9 +257,10 @@ void zm_dump_codecpar(const AVCodecParameters *par) { #endif void zm_dump_codec(const AVCodecContext *codec) { - Debug(1, "Dumping codec_context codec_type(%d) codec_id(%d %s) width(%d) height(%d) timebase(%d/%d) format(%s) " + Debug(1, "Dumping codec_context codec_type(%d %s) codec_id(%d %s) width(%d) height(%d) timebase(%d/%d) format(%s) " "gop_size %d max_b_frames %d me_cmp %d me_range %d qmin %d qmax %d", codec->codec_type, + av_get_media_type_string(codec->codec_type), codec->codec_id, avcodec_get_name(codec->codec_id), codec->width, @@ -306,7 +307,10 @@ void zm_dump_stream_format(AVFormatContext *ic, int i, int index, int is_output) ); #if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) - Debug(1, "codec: %s", avcodec_get_name(st->codecpar->codec_id)); + Debug(1, "codec: %s %s", + avcodec_get_name(st->codecpar->codec_id), + av_get_media_type_string(st->codecpar->codec_type) + ); #else char buf[256]; avcodec_string(buf, sizeof(buf), st->codec, is_output); diff --git a/src/zm_ffmpeg.h b/src/zm_ffmpeg.h index ec22bfc33..0f28a1176 100644 --- a/src/zm_ffmpeg.h +++ b/src/zm_ffmpeg.h @@ -370,7 +370,7 @@ void zm_dump_codecpar(const AVCodecParameters *par); double pts_time = static_cast(av_rescale_q(pkt.pts, stream->time_base, AV_TIME_BASE_Q)) / AV_TIME_BASE; \ \ Debug(2, "%s: pts: %" PRId64 " * %u/%u=%f, dts: %" PRId64 \ - ", size: %d, stream_index: %d, flags: %04x, keyframe(%d) pos: %" PRId64", duration: %" AV_PACKET_DURATION_FMT, \ + ", size: %d, stream_index: %d, %s flags: %04x, keyframe(%d) pos: %" PRId64", duration: %" AV_PACKET_DURATION_FMT, \ text, \ pkt.pts, \ stream->time_base.num, \ @@ -379,6 +379,7 @@ void zm_dump_codecpar(const AVCodecParameters *par); pkt.dts, \ pkt.size, \ pkt.stream_index, \ + av_get_media_type_string(stream->codecpar->codec_type), \ pkt.flags, \ pkt.flags & AV_PKT_FLAG_KEY, \ pkt.pos, \ From 253ed928ec43217a64f5000c83572c61a0672906 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 3 Mar 2021 15:24:11 -0500 Subject: [PATCH 0099/1277] Fix using wrong stream to set codec_type --- src/zm_ffmpeg_camera.cpp | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/src/zm_ffmpeg_camera.cpp b/src/zm_ffmpeg_camera.cpp index 73200fe57..621bb578f 100644 --- a/src/zm_ffmpeg_camera.cpp +++ b/src/zm_ffmpeg_camera.cpp @@ -192,7 +192,6 @@ int FfmpegCamera::Capture(ZMPacket &zm_packet) { int ret; - AVStream *stream; if ( mSecondFormatContext and ( av_rescale_q(mLastAudioPTS, mAudioStream->time_base, AV_TIME_BASE_Q) @@ -201,17 +200,14 @@ int FfmpegCamera::Capture(ZMPacket &zm_packet) { ) ) { // if audio stream is behind video stream, then read from audio, otherwise video mFormatContextPtr = mSecondFormatContext; - stream = mAudioStream; Debug(1, "Using audio input"); } else { - mFormatContextPtr = mFormatContext; - stream = mVideoStream; Debug(1, "Using video input beacuse %" PRId64 " >= %" PRId64, av_rescale_q(mLastAudioPTS, mAudioStream->time_base, AV_TIME_BASE_Q), av_rescale_q(mLastVideoPTS, mVideoStream->time_base, AV_TIME_BASE_Q) ); - } + } if ( (ret = av_read_frame(mFormatContextPtr, &packet)) < 0 ) { if ( @@ -228,6 +224,9 @@ int FfmpegCamera::Capture(ZMPacket &zm_packet) { } return -1; } + + + AVStream *stream = mFormatContextPtr->streams[packet.stream_index]; ZM_DUMP_STREAM_PACKET(stream, packet, "ffmpeg_camera in"); #if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) From d34d2e2398b466b8f09fd36756b1c94aa74e0195 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 3 Mar 2021 15:43:59 -0500 Subject: [PATCH 0100/1277] Use a define to deal with the codec/codecpar version problem --- src/zm_ffmpeg.h | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/src/zm_ffmpeg.h b/src/zm_ffmpeg.h index 0f28a1176..6d6a19418 100644 --- a/src/zm_ffmpeg.h +++ b/src/zm_ffmpeg.h @@ -351,6 +351,13 @@ void zm_dump_codecpar(const AVCodecParameters *par); # define AV_PACKET_DURATION_FMT "d" #endif +#if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) +#define CODEC_TYPE(stream) stream->codecpar->codec_type +#else +#define CODEC_TYPE(stream) stream->codec->codec_type +#endif + + #ifndef DBG_OFF # define ZM_DUMP_PACKET(pkt, text) \ Debug(2, "%s: pts: %" PRId64 ", dts: %" PRId64 \ @@ -379,7 +386,7 @@ void zm_dump_codecpar(const AVCodecParameters *par); pkt.dts, \ pkt.size, \ pkt.stream_index, \ - av_get_media_type_string(stream->codecpar->codec_type), \ + av_get_media_type_string(CODEC_TYPE(stream)), \ pkt.flags, \ pkt.flags & AV_PKT_FLAG_KEY, \ pkt.pos, \ From 20fe6153d521c5fe526bd7f0bb91377874837674 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 3 Mar 2021 15:52:31 -0500 Subject: [PATCH 0101/1277] Don't treat data packets as video --- src/zm_monitor.cpp | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 19ae3ac87..d250f4ca4 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -2591,6 +2591,8 @@ int Monitor::Capture() { return 1; } else { Debug(1, "Unknown codec type %d", packet->codec_type); + delete packet; + return 1; } // end if audio if ( !packet->image ) { From e67626b4e26512bd9ebf33cbda1d61517e8c4858 Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Tue, 2 Mar 2021 08:59:07 +0100 Subject: [PATCH 0102/1277] logger: Move log_mutex to std::recursive_mutex --- src/zm_logger.h | 6 +++--- src/zm_monitor.cpp | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/src/zm_logger.h b/src/zm_logger.h index ee358b206..b2a9bbeb0 100644 --- a/src/zm_logger.h +++ b/src/zm_logger.h @@ -23,9 +23,9 @@ #include "zm_db.h" #include "zm_config.h" #include "zm_define.h" -#include "zm_thread.h" -#include #include +#include +#include #ifdef HAVE_SYS_SYSCALL_H #include @@ -89,7 +89,7 @@ private: static bool smInitialised; static Logger *smInstance; - RecursiveMutex log_mutex; + std::recursive_mutex log_mutex; static StringMap smCodes; static IntMap smSyslogPriorities; diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index d250f4ca4..7d00cf4d5 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -1033,7 +1033,7 @@ bool Monitor::connect() { trigger_data->trigger_cause[0] = 0; trigger_data->trigger_text[0] = 0; trigger_data->trigger_showtext[0] = 0; - video_store_data->recording = (struct timeval){0,0}; + video_store_data->recording = {}; // Uh, why nothing? Why not nullptr? snprintf(video_store_data->event_file, sizeof(video_store_data->event_file), "nothing"); video_store_data->size = sizeof(VideoStoreData); @@ -2560,7 +2560,7 @@ int Monitor::Capture() { packet->packet.stream_index = video_stream_id; // Convert to packetQueue's index if (video_fifo) { if ( packet->keyframe ) { - // avcodec strips out important nals that describe the stream and + // avcodec strips out important nals that describe the stream and // stick them in extradata. Need to send them along with keyframes #if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) AVStream *stream = camera->getVideoStream(); From 8f941c75cdc7cb10d625dae13044a2621b9231db Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Tue, 2 Mar 2021 19:56:35 +0100 Subject: [PATCH 0103/1277] RtspThread: Convert to std::thread --- src/zm_remote_camera_rtsp.cpp | 17 ++++----- src/zm_remote_camera_rtsp.h | 2 +- src/zm_rtp_ctrl.cpp | 2 +- src/zm_rtp_data.cpp | 2 +- src/zm_rtsp.cpp | 66 +++++++++++++++++++---------------- src/zm_rtsp.h | 23 ++++++------ 6 files changed, 56 insertions(+), 56 deletions(-) diff --git a/src/zm_remote_camera_rtsp.cpp b/src/zm_remote_camera_rtsp.cpp index 44d5046af..7cf59b64d 100644 --- a/src/zm_remote_camera_rtsp.cpp +++ b/src/zm_remote_camera_rtsp.cpp @@ -48,7 +48,6 @@ RemoteCameraRtsp::RemoteCameraRtsp( p_brightness, p_contrast, p_hue, p_colour, p_capture, p_record_audio), rtsp_describe(p_rtsp_describe), - rtspThread(0), frameCount(0) { if ( p_method == "rtpUni" ) @@ -114,19 +113,15 @@ void RemoteCameraRtsp::Terminate() { } int RemoteCameraRtsp::Connect() { - rtspThread = new RtspThread(monitor->Id(), method, protocol, host, port, path, auth, rtsp_describe); - - rtspThread->start(); + rtspThread = ZM::make_unique(monitor->Id(), method, protocol, host, port, path, auth, rtsp_describe); return 0; } int RemoteCameraRtsp::Disconnect() { - if ( rtspThread ) { - rtspThread->stop(); - rtspThread->join(); - delete rtspThread; - rtspThread = nullptr; + if (rtspThread) { + rtspThread->Stop(); + rtspThread.reset(); } return 0; } @@ -214,7 +209,7 @@ int RemoteCameraRtsp::PrimeCapture() { } // end PrimeCapture int RemoteCameraRtsp::PreCapture() { - if ( !rtspThread->isRunning() ) + if (!rtspThread || rtspThread->IsStopped()) return -1; if ( !rtspThread->hasSources() ) { Error("Cannot precapture, no RTP sources"); @@ -234,7 +229,7 @@ int RemoteCameraRtsp::Capture(ZMPacket &zm_packet) { while ( !frameComplete ) { buffer.clear(); - if ( !rtspThread->isRunning() ) + if (!rtspThread || rtspThread->IsStopped()) return -1; if ( rtspThread->getFrame(buffer) ) { diff --git a/src/zm_remote_camera_rtsp.h b/src/zm_remote_camera_rtsp.h index bd8b30a7c..e9014e66b 100644 --- a/src/zm_remote_camera_rtsp.h +++ b/src/zm_remote_camera_rtsp.h @@ -44,7 +44,7 @@ protected: RtspThread::RtspMethod method; - RtspThread *rtspThread; + std::unique_ptr rtspThread; int frameCount; diff --git a/src/zm_rtp_ctrl.cpp b/src/zm_rtp_ctrl.cpp index a978d2f2d..8c0875c19 100644 --- a/src/zm_rtp_ctrl.cpp +++ b/src/zm_rtp_ctrl.cpp @@ -327,7 +327,7 @@ int RtpCtrlThread::run() { } // end foeach comms iterator } rtpCtrlServer.close(); - mRtspThread.stop(); + mRtspThread.Stop(); return 0; } diff --git a/src/zm_rtp_data.cpp b/src/zm_rtp_data.cpp index 1d8443a37..4409d39ff 100644 --- a/src/zm_rtp_data.cpp +++ b/src/zm_rtp_data.cpp @@ -98,7 +98,7 @@ int RtpDataThread::run() { } // end foreach commsList } rtpDataSocket.close(); - mRtspThread.stop(); + mRtspThread.Stop(); return 0; } diff --git a/src/zm_rtsp.cpp b/src/zm_rtsp.cpp index 67373813e..92e14fccc 100644 --- a/src/zm_rtsp.cpp +++ b/src/zm_rtsp.cpp @@ -161,7 +161,7 @@ RtspThread::RtspThread( mSsrc(0), mDist(UNDEFINED), mRtpTime(0), - mStop(false) + mTerminate(false) { mUrl = mProtocol+"://"+mHost+":"+mPort; if ( !mPath.empty() ) { @@ -185,9 +185,15 @@ RtspThread::RtspThread( mAuthenticator = new zm::Authenticator(parts[0], parts[1]); else mAuthenticator = new zm::Authenticator(parts[0], ""); + + mThread = std::thread(&RtspThread::Run, this); } RtspThread::~RtspThread() { + Stop(); + if (mThread.joinable()) + mThread.join(); + if ( mFormatContext ) { #if LIBAVFORMAT_VERSION_CHECK(52, 96, 0, 96, 0) avformat_free_context(mFormatContext); @@ -204,7 +210,7 @@ RtspThread::~RtspThread() { mAuthenticator = nullptr; } -int RtspThread::run() { +void RtspThread::Run() { std::string message; std::string response; @@ -246,11 +252,11 @@ int RtspThread::run() { Debug(2, "Sending HTTP message: %s", message.c_str()); if ( mRtspSocket.send(message.c_str(), message.size()) != (int)message.length() ) { Error("Unable to send message '%s': %s", message.c_str(), strerror(errno)); - return -1; + return; } if ( mRtspSocket.recv(response) < 0 ) { Error("Recv failed; %s", strerror(errno)); - return -1; + return; } Debug(2, "Received HTTP response: %s (%zd bytes)", response.c_str(), response.size()); @@ -264,7 +270,7 @@ int RtspThread::run() { if ( response.size() ) Hexdump(Logger::ERROR, response.data(), std::min(int(response.size()), 16)); } - return -1; + return; } // If Server requests authentication, check WWW-Authenticate header and fill required fields // for requested authentication method @@ -282,7 +288,7 @@ int RtspThread::run() { if ( respCode != 200 ) { Error("Unexpected response code %d, text is '%s'", respCode, respText); - return -1; + return; } message = "POST "+mPath+" HTTP/1.0\r\n"; @@ -295,7 +301,7 @@ int RtspThread::run() { Debug(2, "Sending HTTP message: %s", message.c_str()); if ( mRtspSocket2.send(message.c_str(), message.size()) != (int)message.length() ) { Error("Unable to send message '%s': %s", message.c_str(), strerror(errno)); - return -1; + return; } } // end if ( mMethod == RTP_RTSP_HTTP ) @@ -305,18 +311,18 @@ int RtspThread::run() { // Request supported RTSP commands by the server message = "OPTIONS "+mUrl+" RTSP/1.0\r\n"; if ( !sendCommand(message) ) - return -1; + return; // A negative return here may indicate auth failure, but we will have setup the auth mechanisms so we need to retry. if ( !recvResponse(response) ) { if ( mNeedAuth ) { Debug(2, "Resending OPTIONS due to possible auth requirement"); if ( !sendCommand(message) ) - return -1; + return; if ( !recvResponse(response) ) - return -1; + return; } else { - return -1; + return; } } // end if failed response maybe due to auth @@ -347,7 +353,7 @@ int RtspThread::run() { const std::string endOfHeaders = "\r\n\r\n"; size_t sdpStart = response.find(endOfHeaders); if ( sdpStart == std::string::npos ) - return -1; + return; if ( mRtspDescribe ) { std::string DescHeader = response.substr(0, sdpStart); @@ -374,7 +380,7 @@ int RtspThread::run() { mFormatContext = mSessDesc->generateFormatContext(); } catch ( const Exception &e ) { Error(e.getMessage().c_str()); - return -1; + return; } #if 0 @@ -450,9 +456,9 @@ int RtspThread::run() { } if ( !sendCommand(message) ) - return -1; + return; if ( !recvResponse(response) ) - return -1; + return; lines = split(response, "\r\n"); std::string session; @@ -525,9 +531,9 @@ int RtspThread::run() { message = "PLAY "+mUrl+" RTSP/1.0\r\nSession: "+session+"\r\nRange: npt=0.000-\r\n"; if ( !sendCommand(message) ) - return -1; + return; if ( !recvResponse(response) ) - return -1; + return; lines = split(response, "\r\n"); std::string rtpInfo; @@ -590,14 +596,14 @@ int RtspThread::run() { rtpDataThread.start(); rtpCtrlThread.start(); - while( !mStop ) { + while (!mTerminate) { now = time(nullptr); // Send a keepalive message if the server supports this feature and we are close to the timeout expiration Debug(5, "sendkeepalive %d, timeout %d, now: %d last: %d since: %d", sendKeepalive, timeout, now, lastKeepalive, (now-lastKeepalive) ); if ( sendKeepalive && (timeout > 0) && ((now-lastKeepalive) > (timeout-5)) ) { if ( !sendCommand( message ) ) - return( -1 ); + return; lastKeepalive = now; } usleep( 100000 ); @@ -612,9 +618,9 @@ int RtspThread::run() { message = "TEARDOWN "+mUrl+" RTSP/1.0\r\nSession: "+session+"\r\n"; if ( !sendCommand( message ) ) - return( -1 ); + return; if ( !recvResponse( response ) ) - return( -1 ); + return; rtpDataThread.stop(); rtpCtrlThread.stop(); @@ -624,7 +630,7 @@ int RtspThread::run() { rtpDataThread.join(); rtpCtrlThread.join(); - + delete mSources[ssrc]; mSources.clear(); @@ -647,7 +653,7 @@ int RtspThread::run() { Buffer buffer( ZM_NETWORK_BUFSIZ ); std::string keepaliveMessage = "OPTIONS "+mUrl+" RTSP/1.0\r\n"; std::string keepaliveResponse = "RTSP/1.0 200 OK\r\n"; - while ( !mStop && select.wait() >= 0 ) { + while (!mTerminate && select.wait() >= 0) { ZM::Select::CommsList readable = select.getReadable(); if ( readable.size() == 0 ) { Error( "RTSP timed out" ); @@ -720,7 +726,7 @@ Debug(5, "sendkeepalive %d, timeout %d, now: %d last: %d since: %d", sendKeepali if ( sendKeepalive && (timeout > 0) && ((now-lastKeepalive) > (timeout-5)) ) { if ( !sendCommand( message ) ) - return( -1 ); + return; lastKeepalive = now; } buffer.tidy( 1 ); @@ -735,7 +741,7 @@ Debug(5, "sendkeepalive %d, timeout %d, now: %d last: %d since: %d", sendKeepali // Send a teardown message but don't expect a response as this may not be implemented on the server when using TCP message = "TEARDOWN "+mUrl+" RTSP/1.0\r\nSession: "+session+"\r\n"; if ( !sendCommand( message ) ) - return( -1 ); + return; delete mSources[ssrc]; mSources.clear(); @@ -752,11 +758,11 @@ Debug(5, "sendkeepalive %d, timeout %d, now: %d last: %d since: %d", sendKeepali rtpDataThread.start(); rtpCtrlThread.start(); - while ( !mStop ) { + while (!mTerminate) { // Send a keepalive message if the server supports this feature and we are close to the timeout expiration if ( sendKeepalive && (timeout > 0) && ((time(nullptr)-lastKeepalive) > (timeout-5)) ) { if ( !sendCommand( message ) ) - return -1; + return; lastKeepalive = time(nullptr); } usleep(100000); @@ -770,9 +776,9 @@ Debug(5, "sendkeepalive %d, timeout %d, now: %d last: %d since: %d", sendKeepali #endif message = "TEARDOWN "+mUrl+" RTSP/1.0\r\nSession: "+session+"\r\n"; if ( !sendCommand(message) ) - return -1; + return; if ( !recvResponse(response) ) - return -1; + return; rtpDataThread.stop(); rtpCtrlThread.stop(); @@ -791,7 +797,7 @@ Debug(5, "sendkeepalive %d, timeout %d, now: %d last: %d since: %d", sendKeepali break; } - return 0; + return; } #endif // HAVE_LIBAVFORMAT diff --git a/src/zm_rtsp.h b/src/zm_rtsp.h index 26189cd20..6cb88d091 100644 --- a/src/zm_rtsp.h +++ b/src/zm_rtsp.h @@ -24,10 +24,12 @@ #include "zm_rtp_source.h" #include "zm_rtsp_auth.h" #include "zm_sdp.h" +#include #include #include +#include -class RtspThread : public Thread { +class RtspThread { public: typedef enum { RTP_UNICAST, RTP_MULTICAST, RTP_RTSP, RTP_RTSP_HTTP } RtspMethod; typedef enum { UNDEFINED, UNICAST, MULTICAST } RtspDist; @@ -84,12 +86,14 @@ private: unsigned long mRtpTime; - bool mStop; + std::thread mThread; + std::atomic mTerminate; private: bool sendCommand( std::string message ); bool recvResponse( std::string &response ); - void checkAuthResponse(std::string &response); + void checkAuthResponse(std::string &response); + void Run(); public: RtspThread( int id, RtspMethod method, const std::string &protocol, const std::string &host, const std::string &port, const std::string &path, const std::string &auth, bool rtsp_describe ); @@ -124,15 +128,10 @@ public: return( false ); return( iter->second->getFrame( frame ) ); } - int run(); - void stop() - { - mStop = true; - } - bool stopped() const - { - return( mStop ); - } + + void Stop() { mTerminate = true; } + bool IsStopped() const { return mTerminate; } + int getAddressFamily () { return mRtspSocket.getDomain(); From ff2bfb58daf1a6b27743b6608c2c676075a5ec82 Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Wed, 3 Mar 2021 00:35:07 +0100 Subject: [PATCH 0104/1277] RtspThread: Fix an unguarded SQL query causing race conditions --- src/zm_rtsp.cpp | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/src/zm_rtsp.cpp b/src/zm_rtsp.cpp index 92e14fccc..e4e519630 100644 --- a/src/zm_rtsp.cpp +++ b/src/zm_rtsp.cpp @@ -91,16 +91,9 @@ int RtspThread::requestPorts() { char sql[ZM_SQL_SML_BUFSIZ]; //FIXME Why not load specifically by Id? This will get ineffeicient with a lot of monitors strncpy(sql, "SELECT `Id` FROM `Monitors` WHERE `Function` != 'None' AND `Type` = 'Remote' AND `Protocol` = 'rtsp' AND `Method` = 'rtpUni' ORDER BY `Id` ASC", sizeof(sql)); - if ( mysql_query(&dbconn, sql) ) { - Error("Can't run query: %s", mysql_error(&dbconn)); - exit(mysql_errno(&dbconn)); - } - MYSQL_RES *result = mysql_store_result(&dbconn); - if ( !result ) { - Error("Can't use query result: %s", mysql_error(&dbconn)); - exit(mysql_errno(&dbconn)); - } + MYSQL_RES *result = zmDbFetch(sql); + int nMonitors = mysql_num_rows(result); int position = 0; if ( nMonitors ) { From 71edb9d830696a618e9f8a5911c748bbf4356f92 Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Wed, 3 Mar 2021 00:35:34 +0100 Subject: [PATCH 0105/1277] RtpCtrlThread: Convert to std::thread --- src/zm_rtp_ctrl.cpp | 21 +++++++++++++-------- src/zm_rtp_ctrl.h | 15 +++++++++------ src/zm_rtsp.cpp | 7 ++----- 3 files changed, 24 insertions(+), 19 deletions(-) diff --git a/src/zm_rtp_ctrl.cpp b/src/zm_rtp_ctrl.cpp index 8c0875c19..7185ef6aa 100644 --- a/src/zm_rtp_ctrl.cpp +++ b/src/zm_rtp_ctrl.cpp @@ -25,9 +25,16 @@ #if HAVE_LIBAVFORMAT -RtpCtrlThread::RtpCtrlThread( RtspThread &rtspThread, RtpSource &rtpSource ) - : mRtspThread( rtspThread ), mRtpSource( rtpSource ), mStop( false ) +RtpCtrlThread::RtpCtrlThread(RtspThread &rtspThread, RtpSource &rtpSource) + : mRtspThread(rtspThread), mRtpSource(rtpSource), mTerminate(false) { + mThread = std::thread(&RtpCtrlThread::Run, this); +} + +RtpCtrlThread::~RtpCtrlThread() { + Stop(); + if (mThread.joinable()) + mThread.join(); } int RtpCtrlThread::recvPacket( const unsigned char *packet, ssize_t packetLen ) { @@ -121,7 +128,7 @@ int RtpCtrlThread::recvPacket( const unsigned char *packet, ssize_t packetLen ) } case RTCP_BYE : Debug(5, "RTCP Got BYE"); - mStop = true; + Stop(); break; case RTCP_APP : // Ignoring as per RFC 3550 @@ -241,7 +248,7 @@ int RtpCtrlThread::recvPackets( unsigned char *buffer, ssize_t nBytes ) { return nBytes; } -int RtpCtrlThread::run() { +void RtpCtrlThread::Run() { Debug( 2, "Starting control thread %x on port %d", mRtpSource.getSsrc(), mRtpSource.getLocalCtrlPort() ); ZM::SockAddrInet localAddr, remoteAddr; @@ -272,8 +279,7 @@ int RtpCtrlThread::run() { time_t last_receive = time(nullptr); bool timeout = false; // used as a flag that we had a timeout, and then sent an RR to see if we wake back up. Real timeout will happen when this is true. - while ( !mStop && select.wait() >= 0 ) { - + while (!mTerminate && select.wait() >= 0) { time_t now = time(nullptr); ZM::Select::CommsList readable = select.getReadable(); if ( readable.size() == 0 ) { @@ -318,7 +324,7 @@ int RtpCtrlThread::run() { } } else { // Here is another case of not receiving some data causing us to terminate... why? Sometimes there are pauses in the interwebs. - mStop = true; + Stop(); break; } } else { @@ -328,7 +334,6 @@ int RtpCtrlThread::run() { } rtpCtrlServer.close(); mRtspThread.Stop(); - return 0; } #endif // HAVE_LIBAVFORMAT diff --git a/src/zm_rtp_ctrl.h b/src/zm_rtp_ctrl.h index 5fd428ead..3c3706ef6 100644 --- a/src/zm_rtp_ctrl.h +++ b/src/zm_rtp_ctrl.h @@ -21,6 +21,8 @@ #define ZM_RTP_CTRL_H #include "zm_thread.h" +#include +#include // Defined in ffmpeg rtp.h //#define RTP_MAX_SDES 255 // maximum text length for SDES @@ -32,7 +34,7 @@ class RtspThread; class RtpSource; -class RtpCtrlThread : public Thread { +class RtpCtrlThread { friend class RtspThread; private: @@ -121,7 +123,9 @@ private: RtspThread &mRtspThread; RtpSource &mRtpSource; int mPort; - bool mStop; + + std::atomic mTerminate; + std::thread mThread; private: int recvPacket( const unsigned char *packet, ssize_t packetLen ); @@ -129,14 +133,13 @@ private: int generateSdes( const unsigned char *packet, ssize_t packetLen ); int generateBye( const unsigned char *packet, ssize_t packetLen ); int recvPackets( unsigned char *buffer, ssize_t nBytes ); - int run(); + void Run(); public: RtpCtrlThread( RtspThread &rtspThread, RtpSource &rtpSource ); + ~RtpCtrlThread(); - void stop() { - mStop = true; - } + void Stop() { mTerminate = true; } }; #endif // ZM_RTP_CTRL_H diff --git a/src/zm_rtsp.cpp b/src/zm_rtsp.cpp index e4e519630..9b17679e2 100644 --- a/src/zm_rtsp.cpp +++ b/src/zm_rtsp.cpp @@ -587,7 +587,6 @@ void RtspThread::Run() { RtpCtrlThread rtpCtrlThread( *this, *source ); rtpDataThread.start(); - rtpCtrlThread.start(); while (!mTerminate) { now = time(nullptr); @@ -616,7 +615,7 @@ void RtspThread::Run() { return; rtpDataThread.stop(); - rtpCtrlThread.stop(); + rtpCtrlThread.Stop(); //rtpDataThread.kill( SIGTERM ); //rtpCtrlThread.kill( SIGTERM ); @@ -749,7 +748,6 @@ Debug(5, "sendkeepalive %d, timeout %d, now: %d last: %d since: %d", sendKeepali RtpCtrlThread rtpCtrlThread( *this, *source ); rtpDataThread.start(); - rtpCtrlThread.start(); while (!mTerminate) { // Send a keepalive message if the server supports this feature and we are close to the timeout expiration @@ -774,10 +772,9 @@ Debug(5, "sendkeepalive %d, timeout %d, now: %d last: %d since: %d", sendKeepali return; rtpDataThread.stop(); - rtpCtrlThread.stop(); + rtpCtrlThread.Stop(); rtpDataThread.join(); - rtpCtrlThread.join(); delete mSources[ssrc]; mSources.clear(); From 8f0431d85b3cf72ae83bff45ae06ce77a5a9b386 Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Wed, 3 Mar 2021 00:40:57 +0100 Subject: [PATCH 0106/1277] RtpDataThread: Convert to std::thread --- src/zm_rtp_data.cpp | 18 ++++++++++++------ src/zm_rtp_data.h | 17 +++++++++-------- src/zm_rtsp.cpp | 12 ++---------- 3 files changed, 23 insertions(+), 24 deletions(-) diff --git a/src/zm_rtp_data.cpp b/src/zm_rtp_data.cpp index 4409d39ff..aed02aea6 100644 --- a/src/zm_rtp_data.cpp +++ b/src/zm_rtp_data.cpp @@ -26,8 +26,15 @@ #if HAVE_LIBAVFORMAT RtpDataThread::RtpDataThread(RtspThread &rtspThread, RtpSource &rtpSource) : - mRtspThread(rtspThread), mRtpSource(rtpSource), mStop(false) + mRtspThread(rtspThread), mRtpSource(rtpSource), mTerminate(false) { + mThread = std::thread(&RtpDataThread::Run, this); +} + +RtpDataThread::~RtpDataThread() { + Stop(); + if (mThread.joinable()) + mThread.join(); } bool RtpDataThread::recvPacket(const unsigned char *packet, size_t packetLen) { @@ -54,7 +61,7 @@ bool RtpDataThread::recvPacket(const unsigned char *packet, size_t packetLen) { return mRtpSource.handlePacket(packet, packetLen); } -int RtpDataThread::run() { +void RtpDataThread::Run() { Debug(2, "Starting data thread %d on port %d", mRtpSource.getSsrc(), mRtpSource.getLocalDataPort()); @@ -75,11 +82,11 @@ int RtpDataThread::run() { select.addReader(&rtpDataSocket); unsigned char buffer[ZM_NETWORK_BUFSIZ]; - while ( !zm_terminate && !mStop && (select.wait() >= 0) ) { + while ( !zm_terminate && !mTerminate && (select.wait() >= 0) ) { ZM::Select::CommsList readable = select.getReadable(); if ( readable.size() == 0 ) { Error("RTP timed out"); - mStop = true; + Stop(); break; } for ( ZM::Select::CommsList::iterator iter = readable.begin(); iter != readable.end(); ++iter ) { @@ -89,7 +96,7 @@ int RtpDataThread::run() { if ( nBytes ) { recvPacket(buffer, nBytes); } else { - mStop = true; + Stop(); break; } } else { @@ -99,7 +106,6 @@ int RtpDataThread::run() { } rtpDataSocket.close(); mRtspThread.Stop(); - return 0; } #endif // HAVE_LIBAVFORMAT diff --git a/src/zm_rtp_data.h b/src/zm_rtp_data.h index 9ecef8305..e20878212 100644 --- a/src/zm_rtp_data.h +++ b/src/zm_rtp_data.h @@ -21,7 +21,8 @@ #define ZM_RTP_DATA_H #include "zm_define.h" -#include "zm_thread.h" +#include +#include class RtspThread; class RtpSource; @@ -40,26 +41,26 @@ struct RtpDataHeader uint32_t csrc[]; // optional CSRC list }; -class RtpDataThread : public Thread +class RtpDataThread { friend class RtspThread; private: RtspThread &mRtspThread; RtpSource &mRtpSource; - bool mStop; + + std::atomic mTerminate; + std::thread mThread; private: bool recvPacket( const unsigned char *packet, size_t packetLen ); - int run(); + void Run(); public: RtpDataThread( RtspThread &rtspThread, RtpSource &rtpSource ); + ~RtpDataThread(); - void stop() - { - mStop = true; - } + void Stop() { mTerminate = true; } }; #endif // ZM_RTP_DATA_H diff --git a/src/zm_rtsp.cpp b/src/zm_rtsp.cpp index 9b17679e2..36ae56e8c 100644 --- a/src/zm_rtsp.cpp +++ b/src/zm_rtsp.cpp @@ -586,8 +586,6 @@ void RtspThread::Run() { RtpDataThread rtpDataThread( *this, *source ); RtpCtrlThread rtpCtrlThread( *this, *source ); - rtpDataThread.start(); - while (!mTerminate) { now = time(nullptr); // Send a keepalive message if the server supports this feature and we are close to the timeout expiration @@ -614,15 +612,12 @@ void RtspThread::Run() { if ( !recvResponse( response ) ) return; - rtpDataThread.stop(); + rtpDataThread.Stop(); rtpCtrlThread.Stop(); //rtpDataThread.kill( SIGTERM ); //rtpCtrlThread.kill( SIGTERM ); - rtpDataThread.join(); - rtpCtrlThread.join(); - delete mSources[ssrc]; mSources.clear(); @@ -747,7 +742,6 @@ Debug(5, "sendkeepalive %d, timeout %d, now: %d last: %d since: %d", sendKeepali RtpDataThread rtpDataThread( *this, *source ); RtpCtrlThread rtpCtrlThread( *this, *source ); - rtpDataThread.start(); while (!mTerminate) { // Send a keepalive message if the server supports this feature and we are close to the timeout expiration @@ -771,10 +765,8 @@ Debug(5, "sendkeepalive %d, timeout %d, now: %d last: %d since: %d", sendKeepali if ( !recvResponse(response) ) return; - rtpDataThread.stop(); + rtpDataThread.Stop(); rtpCtrlThread.Stop(); - - rtpDataThread.join(); delete mSources[ssrc]; mSources.clear(); From c78e174e788295af56c10b19010c40dc1ee332a0 Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Wed, 3 Mar 2021 01:11:39 +0100 Subject: [PATCH 0107/1277] RTSPServerThread: Convert to std::thread --- src/zm_rtsp_server.cpp | 11 ++--------- src/zm_rtsp_server_thread.cpp | 37 +++++++++++++++++++++-------------- src/zm_rtsp_server_thread.h | 15 +++++++++----- 3 files changed, 34 insertions(+), 29 deletions(-) diff --git a/src/zm_rtsp_server.cpp b/src/zm_rtsp_server.cpp index 563a219e7..376a35a01 100644 --- a/src/zm_rtsp_server.cpp +++ b/src/zm_rtsp_server.cpp @@ -157,9 +157,9 @@ int main(int argc, char *argv[]) { sigaddset(&block_set, SIGUSR1); sigaddset(&block_set, SIGUSR2); - RTSPServerThread * rtsp_server_thread = nullptr; + std::unique_ptr rtsp_server_thread; if (config.min_rtsp_port) { - rtsp_server_thread = new RTSPServerThread(config.min_rtsp_port); + rtsp_server_thread = ZM::make_unique(config.min_rtsp_port); Debug(1, "Starting RTSP server because min_rtsp_port is set"); } else { Debug(1, "Not starting RTSP server because min_rtsp_port not set"); @@ -168,8 +168,6 @@ int main(int argc, char *argv[]) { ServerMediaSession **sessions = new ServerMediaSession *[monitors.size()]; for (size_t i = 0; i < monitors.size(); i++) sessions[i] = nullptr; - rtsp_server_thread->start(); - while (!zm_terminate) { for (size_t i = 0; i < monitors.size(); i++) { @@ -226,11 +224,6 @@ int main(int argc, char *argv[]) { } // end if zm_reload } // end while ! zm_terminate - rtsp_server_thread->stop(); - rtsp_server_thread->join(); - delete rtsp_server_thread; - rtsp_server_thread = nullptr; - delete[] sessions; sessions = nullptr; diff --git a/src/zm_rtsp_server_thread.cpp b/src/zm_rtsp_server_thread.cpp index c57de8b07..c337d669f 100644 --- a/src/zm_rtsp_server_thread.cpp +++ b/src/zm_rtsp_server_thread.cpp @@ -12,7 +12,7 @@ #include RTSPServerThread::RTSPServerThread(int port) : - terminate(0) + terminate_(false), scheduler_watch_var_(0) { //unsigned short rtsp_over_http_port = 0; //const char *realm = "ZoneMinder"; @@ -34,9 +34,15 @@ RTSPServerThread::RTSPServerThread(int port) : } const char *prefix = rtspServer->rtspURLPrefix(); delete[] prefix; -} // end RTSPServerThread::RTSPServerThread + + thread_ = std::thread(&RTSPServerThread::Run, this); +} RTSPServerThread::~RTSPServerThread() { + Stop(); + if (thread_.joinable()) + thread_.join(); + if (rtspServer) { Medium::close(rtspServer); } // end if rtsp_server @@ -49,25 +55,26 @@ RTSPServerThread::~RTSPServerThread() { delete scheduler; } -int RTSPServerThread::run() { - Debug(1, "RTSPServerThread::run()"); - if ( rtspServer ) - env->taskScheduler().doEventLoop(&terminate); // does not return +void RTSPServerThread::Run() { + Debug(1, "RTSPServerThread::Run()"); + if (rtspServer) + env->taskScheduler().doEventLoop(&scheduler_watch_var_); // does not return Debug(1, "RTSPServerThread::done()"); - return 0; -} // end in RTSPServerThread::run() +} -void RTSPServerThread::stop() { +void RTSPServerThread::Stop() { Debug(1, "RTSPServerThread::stop()"); - terminate = 1; + terminate_ = true; + + { + std::lock_guard lck(scheduler_watch_var_mutex_); + scheduler_watch_var_ = 1; + } + for ( std::list::iterator it = sources.begin(); it != sources.end(); ++it ) { (*it)->stopGettingFrames(); } -} // end RTSPServerThread::stop() - -bool RTSPServerThread::stopped() const { - return terminate ? true : false; -} // end RTSPServerThread::stopped() +} ServerMediaSession *RTSPServerThread::addSession(std::string &streamname) { ServerMediaSession *sms = ServerMediaSession::createNew(*env, streamname.c_str()); diff --git a/src/zm_rtsp_server_thread.h b/src/zm_rtsp_server_thread.h index 717ca1bdc..5921ebe0d 100644 --- a/src/zm_rtsp_server_thread.h +++ b/src/zm_rtsp_server_thread.h @@ -6,6 +6,7 @@ #include "zm_thread.h" #include "zm_rtsp_server_server_media_subsession.h" #include "zm_rtsp_server_fifo_source.h" +#include #include #include @@ -15,10 +16,14 @@ class Monitor; -class RTSPServerThread : public Thread { +class RTSPServerThread { private: std::shared_ptr monitor_; - char terminate; + + std::thread thread_; + std::atomic terminate_; + std::mutex scheduler_watch_var_mutex_; + char scheduler_watch_var_; TaskScheduler* scheduler; UsageEnvironment* env; @@ -34,9 +39,9 @@ class RTSPServerThread : public Thread { void removeSession(ServerMediaSession *sms); void addStream(std::string &streamname, AVStream *, AVStream *); FramedSource *addFifo(ServerMediaSession *sms, std::string fifo); - int run(); - void stop(); - bool stopped() const; + void Run(); + void Stop(); + bool IsStopped() const { return terminate_; }; private: const std::string getRtpFormat(AVCodecID codec, bool muxTS); int addSession( From c7f9cc3368ba3a196bf56c1332febec1e490f6f0 Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Wed, 3 Mar 2021 09:20:06 +0100 Subject: [PATCH 0108/1277] LibvlcCamera: Convert ThreadData with std::condition_variable --- src/zm_libvlc_camera.cpp | 25 +++++++++++++++++-------- src/zm_libvlc_camera.h | 10 +++++++--- 2 files changed, 24 insertions(+), 11 deletions(-) diff --git a/src/zm_libvlc_camera.cpp b/src/zm_libvlc_camera.cpp index 90d43da61..c2690ee94 100644 --- a/src/zm_libvlc_camera.cpp +++ b/src/zm_libvlc_camera.cpp @@ -94,7 +94,11 @@ void LibvlcUnlockBuffer(void* opaque, void* picture, void *const *planes) { // Return frames slightly faster than 1fps (if time() supports greater than one second resolution) if ( newFrame || difftime(now, data->prevTime) >= 0.8 ) { data->prevTime = now; - data->newImage.updateValueSignal(true); + { + std::lock_guard lck(data->newImageMutex); + data->newImage = true; + } + data->newImageCv.notify_all(); } } @@ -165,6 +169,9 @@ LibvlcCamera::~LibvlcCamera() { if ( capture ) { Terminate(); } + + mLibvlcData.newImageCv.notify_all(); // to unblock on termination (zm_terminate) + if ( mLibvlcMediaPlayer != nullptr ) { (*libvlc_media_player_release_f)(mLibvlcMediaPlayer); mLibvlcMediaPlayer = nullptr; @@ -255,8 +262,8 @@ int LibvlcCamera::PrimeCapture() { // Libvlc wants 32 byte alignment for images (should in theory do this for all image lines) mLibvlcData.buffer = (uint8_t*)zm_mallocaligned(64, mLibvlcData.bufferSize); mLibvlcData.prevBuffer = (uint8_t*)zm_mallocaligned(64, mLibvlcData.bufferSize); - - mLibvlcData.newImage.setValueImmediate(false); + + mLibvlcData.newImage = false; (*libvlc_media_player_play_f)(mLibvlcMediaPlayer); @@ -271,17 +278,19 @@ int LibvlcCamera::PreCapture() { // Should not return -1 as cancels capture. Always wait for image if available. int LibvlcCamera::Capture( ZMPacket &zm_packet ) { // newImage is a mutex/condition based flag to tell us when there is an image available - while( !mLibvlcData.newImage.getValueImmediate() ) { - if (zm_terminate) - return 0; - mLibvlcData.newImage.getUpdatedValue(1); + { + std::unique_lock lck(mLibvlcData.newImageMutex); + mLibvlcData.newImageCv.wait(lck, [&]{ return mLibvlcData.newImage || zm_terminate; }); + mLibvlcData.newImage = false; } + if (zm_terminate) + return 0; + mLibvlcData.mutex.lock(); zm_packet.image->Assign(width, height, colours, subpixelorder, mLibvlcData.buffer, width * height * mBpp); zm_packet.packet.stream_index = mVideoStreamId; zm_packet.stream = mVideoStream; - mLibvlcData.newImage.setValueImmediate(false); mLibvlcData.mutex.unlock(); return 1; diff --git a/src/zm_libvlc_camera.h b/src/zm_libvlc_camera.h index 074989d17..bcf48ce13 100644 --- a/src/zm_libvlc_camera.h +++ b/src/zm_libvlc_camera.h @@ -21,7 +21,8 @@ #define ZM_LIBVLC_CAMERA_H #include "zm_camera.h" -#include "zm_thread.h" +#include +#include #if HAVE_LIBVLC @@ -35,8 +36,11 @@ struct LibvlcPrivateData { uint8_t* prevBuffer; time_t prevTime; uint32_t bufferSize; - Mutex mutex; - ThreadData newImage; + std::mutex mutex; + + bool newImage; + std::mutex newImageMutex; + std::condition_variable newImageCv; }; class LibvlcCamera : public Camera { From 1dfa41923b5bce271b19617f0c467f46ce8bb18f Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Wed, 3 Mar 2021 23:29:28 +0100 Subject: [PATCH 0109/1277] Fifo: F_{G,S}ETPIPE_SZ is Linux specific --- src/zm_fifo.cpp | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/zm_fifo.cpp b/src/zm_fifo.cpp index a9ceae355..09a808bcd 100644 --- a/src/zm_fifo.cpp +++ b/src/zm_fifo.cpp @@ -78,6 +78,7 @@ bool Fifo::open() { return false; } } +#ifdef __linux__ int ret = fcntl(raw_fd, F_SETPIPE_SZ, 1024 * 1024); if (ret < 0) { Error("set pipe size failed."); @@ -87,6 +88,7 @@ bool Fifo::open() { perror("get pipe size failed."); } Debug(1, "default pipe size: %ld\n", pipe_size); +#endif return true; } From 69185e2204fe3f0b791c4415353e32a198ff229c Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Wed, 3 Mar 2021 21:36:46 +0100 Subject: [PATCH 0110/1277] RtpSource: Convert ThreadData with std::condition_variable --- src/zm_rtp_source.cpp | 55 +++++++++++++++++++++++++++---------------- src/zm_rtp_source.h | 15 ++++++++++-- 2 files changed, 48 insertions(+), 22 deletions(-) diff --git a/src/zm_rtp_source.cpp b/src/zm_rtp_source.cpp index e9edfb821..1515c8eb4 100644 --- a/src/zm_rtp_source.cpp +++ b/src/zm_rtp_source.cpp @@ -79,6 +79,12 @@ RtpSource::RtpSource( Warning("The device is using a codec (%d) that may not be supported. Do not be surprised if things don't work.", mCodecId); } +RtpSource::~RtpSource() { + mTerminate = true; + mFrameReadyCv.notify_all(); + mFrameProcessedCv.notify_all(); +} + void RtpSource::init(uint16_t seq) { Debug(3, "Initialising sequence"); mBaseSeq = seq; @@ -292,7 +298,7 @@ bool RtpSource::handlePacket(const unsigned char *packet, size_t packetLen) { extraHeader = 2; break; - default: + default: Debug(3, "Unhandled nalType %d", nalType); } @@ -301,7 +307,7 @@ bool RtpSource::handlePacket(const unsigned char *packet, size_t packetLen) { mFrame.append("\x0\x0\x1", 3); } // end if H264 mFrame.append(packet+rtpHeaderSize+extraHeader, - packetLen-rtpHeaderSize-extraHeader); + packetLen-rtpHeaderSize-extraHeader); } else { Debug(3, "NOT H264 frame: type is %d", mCodecId); } @@ -312,16 +318,21 @@ bool RtpSource::handlePacket(const unsigned char *packet, size_t packetLen) { if ( mFrameGood ) { Debug(3, "Got new frame %d, %d bytes", mFrameCount, mFrame.size()); - mFrameProcessed.setValueImmediate(false); - mFrameReady.updateValueSignal(true); - if ( !mFrameProcessed.getValueImmediate() ) { - // What is the point of this for loop? Is it just me, or will it call getUpdatedValue once or twice? Could it not be better written as - // if ( ! mFrameProcessed.getUpdatedValue( 1 ) && mFrameProcessed.getUpdatedValue( 1 ) ) return false; - - for ( int count = 0; !mFrameProcessed.getUpdatedValue(1); count++ ) - if ( count > 1 ) - return false; + { + std::lock_guard lck(mFrameReadyMutex); + mFrameReady = true; } + mFrameReadyCv.notify_all(); + + { + std::unique_lock lck(mFrameProcessedMutex); + mFrameProcessedCv.wait(lck, [&]{ return mFrameProcessed || mTerminate; }); + mFrameProcessed = false; + } + + if (mTerminate) + return false; + mFrameCount++; } else { Warning("Discarding incomplete frame %d, %d bytes", mFrameCount, mFrame.size()); @@ -349,17 +360,21 @@ bool RtpSource::handlePacket(const unsigned char *packet, size_t packetLen) { } bool RtpSource::getFrame(Buffer &buffer) { - if ( !mFrameReady.getValueImmediate() ) { - Debug(3, "Getting frame but not ready"); - // Allow for a couple of spurious returns - for ( int count = 0; !mFrameReady.getUpdatedValue(1); count++ ) { - if ( count > 1 ) - return false; - } + { + std::unique_lock lck(mFrameReadyMutex); + mFrameReadyCv.wait(lck, [&]{ return mFrameReady || mTerminate; }); + mFrameReady = false; } + + if (mTerminate) + return false; + buffer = mFrame; - mFrameReady.setValueImmediate(false); - mFrameProcessed.updateValueSignal(true); + { + std::lock_guard lck(mFrameProcessedMutex); + mFrameProcessed = true; + } + mFrameProcessedCv.notify_all(); Debug(4, "Copied %d bytes", buffer.size()); return true; } diff --git a/src/zm_rtp_source.h b/src/zm_rtp_source.h index 522ba4324..7869d016d 100644 --- a/src/zm_rtp_source.h +++ b/src/zm_rtp_source.h @@ -25,6 +25,8 @@ #include "zm_define.h" #include "zm_ffmpeg.h" #include "zm_thread.h" +#include +#include #include #include @@ -90,14 +92,23 @@ private: int mFrameCount; bool mFrameGood; bool prevM; - ThreadData mFrameReady; - ThreadData mFrameProcessed; + + bool mTerminate; + + bool mFrameReady; + std::condition_variable mFrameReadyCv; + std::mutex mFrameReadyMutex; + + bool mFrameProcessed; + std::condition_variable mFrameProcessedCv; + std::mutex mFrameProcessedMutex; private: void init(uint16_t seq); public: RtpSource( int id, const std::string &localHost, int localPortBase, const std::string &remoteHost, int remotePortBase, uint32_t ssrc, uint16_t seq, uint32_t rtpClock, uint32_t rtpTime, _AVCODECID codecId ); + ~RtpSource(); bool updateSeq( uint16_t seq ); void updateJitter( const RtpDataHeader *header ); From a78236d05c543be302d7d4a48126ff20cf89b4e3 Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Wed, 3 Mar 2021 23:31:22 +0100 Subject: [PATCH 0111/1277] Fifo: Fix some missing c_str()s for formatting logs --- src/zm_fifo.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/zm_fifo.cpp b/src/zm_fifo.cpp index 09a808bcd..7f7632ba1 100644 --- a/src/zm_fifo.cpp +++ b/src/zm_fifo.cpp @@ -124,7 +124,7 @@ bool Fifo::writePacket(std::string filename, ZMPacket &packet) { if ( !on_blocking_abort ) { if ( (outfile = fopen(filename.c_str(), "wb")) == nullptr ) { - Error("Can't open %s for writing: %s", filename, strerror(errno)); + Error("Can't open %s for writing: %s", filename.c_str(), strerror(errno)); return false; } } else { @@ -174,7 +174,7 @@ bool Fifo::write(std::string filename, uint8_t *data, size_t bytes) { if ( !on_blocking_abort ) { if ( (outfile = fopen(filename.c_str(), "wb")) == nullptr ) { - Error("Can't open %s for writing: %s", filename, strerror(errno)); + Error("Can't open %s for writing: %s", filename.c_str(), strerror(errno)); return false; } } else { From 15bb9969da8af772e1bcee7dc6b85cad26fc5cf4 Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Wed, 3 Mar 2021 23:38:15 +0100 Subject: [PATCH 0112/1277] Timer: Unused, let's remove it Currently there is no need for this functionality. Since it depends on Thread and ThreadData (which will be removed) let's drop this as well. If need would arise for such functionality a new implementation with a modern API should be written. --- src/CMakeLists.txt | 1 - src/zm_timer.cpp | 119 --------------------------------------------- src/zm_timer.h | 110 ----------------------------------------- 3 files changed, 230 deletions(-) delete mode 100644 src/zm_timer.cpp delete mode 100644 src/zm_timer.h diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index b68077235..0aa52f240 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -69,7 +69,6 @@ set(ZM_BIN_SRC_FILES zm_swscale.cpp zm_thread.cpp zm_time.cpp - zm_timer.cpp zm_user.cpp zm_utils.cpp zm_video.cpp diff --git a/src/zm_timer.cpp b/src/zm_timer.cpp deleted file mode 100644 index e6d6d16da..000000000 --- a/src/zm_timer.cpp +++ /dev/null @@ -1,119 +0,0 @@ -// -// ZoneMinder Timer Class Implementation, $Date$, $Revision$ -// Copyright (C) 2001-2008 Philip Coombes -// -// This program is free software; you can redistribute it and/or -// modify it under the terms of the GNU General Public License -// as published by the Free Software Foundation; either version 2 -// of the License, or (at your option) any later version. -// -// This program is distributed in the hope that it will be useful, -// but WITHOUT ANY WARRANTY; without even the implied warranty of -// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -// GNU General Public License for more details. -// -// You should have received a copy of the GNU General Public License -// along with this program; if not, write to the Free Software -// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. -// - -#include "zm_timer.h" - -#include "zm_logger.h" - -int Timer::TimerThread::mNextTimerId = 0; - -Timer::TimerThread::TimerThread( Timer &timer, int duration, bool repeat ) : - mTimerId( 0 ), - mTimer( timer ), - mDuration( duration ), - mRepeat( repeat ), - mReset( false ), - mExpiryFlag( true ) -{ - mAccessMutex.lock(); - mTimerId = mNextTimerId++; - Debug( 5, "Creating timer %d for %d seconds%s", mTimerId, mDuration, mRepeat?", repeating":"" ); - mAccessMutex.unlock(); -} - -Timer::TimerThread::~TimerThread() -{ - cancel(); -} - -void Timer::TimerThread::cancel() -{ - mAccessMutex.lock(); - if ( mRunning ) - { - Debug( 4, "Cancelling timer %d", mTimerId ); - mRepeat = false; - mReset = false; - mExpiryFlag.updateValueSignal( false ); - } - mAccessMutex.unlock(); -} - -void Timer::TimerThread::reset() -{ - mAccessMutex.lock(); - if ( mRunning ) - { - Debug( 4, "Resetting timer" ); - mReset = true; - mExpiryFlag.updateValueSignal( false ); - } - else - { - Error( "Attempting to reset expired timer %d", mTimerId ); - } - mAccessMutex.unlock(); -} - -int Timer::TimerThread::run() -{ - Debug( 4, "Starting timer %d for %d seconds", mTimerId, mDuration ); - bool timerExpired = false; - do - { - mAccessMutex.lock(); - mReset = false; - mExpiryFlag.setValue( true ); - mAccessMutex.unlock(); - timerExpired = mExpiryFlag.getUpdatedValue( mDuration ); - mAccessMutex.lock(); - if ( timerExpired ) - { - Debug( 4, "Timer %d expired", mTimerId ); - mTimer.expire(); - } - else - { - Debug( 4, "Timer %d %s", mTimerId, mReset?"reset":"cancelled" ); - } - mAccessMutex.unlock(); - } while ( mRepeat || (mReset && !timerExpired) ); - return( timerExpired ); -} - -Timer::Timer( int timeout, bool repeat ) : mTimerThread( *this, timeout, repeat ) -{ - mTimerThread.start(); -} - -Timer::~Timer() -{ - //cancel(); -} - -void Timer::Timer::cancel() -{ - mTimerThread.cancel(); -} - -void Timer::Timer::reset() -{ - mTimerThread.reset(); -} - diff --git a/src/zm_timer.h b/src/zm_timer.h deleted file mode 100644 index f5db16f3a..000000000 --- a/src/zm_timer.h +++ /dev/null @@ -1,110 +0,0 @@ -// -// ZoneMinder Timer Class Interface, $Date$, $Revision$ -// Copyright (C) 2001-2008 Philip Coombes -// -// This program is free software; you can redistribute it and/or -// modify it under the terms of the GNU General Public License -// as published by the Free Software Foundation; either version 2 -// of the License, or (at your option) any later version. -// -// This program is distributed in the hope that it will be useful, -// but WITHOUT ANY WARRANTY; without even the implied warranty of -// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -// GNU General Public License for more details. -// -// You should have received a copy of the GNU General Public License -// along with this program; if not, write to the Free Software -// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. -// - -#ifndef ZM_TIMER_H -#define ZM_TIMER_H - -#include "zm_exception.h" -#include "zm_thread.h" -#include "zm_utils.h" - -#ifdef HAVE_SYS_SYSCALL_H -#include -#endif // HAVE_SYS_SYSCALL_H - -class Timer -{ -private: - class TimerException : public Exception - { - private: -#ifndef SOLARIS - pid_t pid() { - pid_t tid; -#ifdef __FreeBSD__ - long lwpid; - thr_self(&lwpid); - tid = lwpid; -#else - #ifdef __FreeBSD_kernel__ - if ( (syscall(SYS_thr_self, &tid)) < 0 ) // Thread/Process id - #else - tid=syscall(SYS_gettid); - #endif -#endif - return tid; - } -#else - pthread_t pid() { return( pthread_self() ); } -#endif - public: - explicit TimerException( const std::string &message ) : Exception( stringtf("(%d) ", (long int)pid())+message ) { - } - }; - - class TimerThread : public Thread - { - private: - typedef ThreadData ExpiryFlag; - - private: - static int mNextTimerId; - - private: - int mTimerId; - Timer &mTimer; - int mDuration; - int mRepeat; - int mReset; - ExpiryFlag mExpiryFlag; - Mutex mAccessMutex; - - private: - void quit() - { - cancel(); - } - - public: - TimerThread( Timer &timer, int timeout, bool repeat ); - ~TimerThread(); - - void cancel(); - void reset(); - int run(); - }; - -protected: - TimerThread mTimerThread; - -protected: - Timer( int timeout, bool repeat=false ); - -public: - virtual ~Timer(); - -protected: - virtual void expire()=0; - -public: - void cancel(); - void reset(); -}; - -#endif // ZM_TIMER_H From d9568a98c0c9bf9c7af73f6032bb447778c461ff Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Wed, 3 Mar 2021 23:48:02 +0100 Subject: [PATCH 0113/1277] Drop zm_thread which has been replaced by STL implementations --- src/CMakeLists.txt | 1 - src/zm_buffer.cpp | 2 + src/zm_comms.h | 1 + src/zm_db.h | 1 - src/zm_event.cpp | 1 + src/zm_event.h | 1 + src/zm_eventstream.cpp | 2 + src/zm_fifo.cpp | 1 + src/zm_fifo_debug.cpp | 1 + src/zm_fifo_stream.cpp | 1 + src/zm_file_camera.cpp | 1 + src/zm_image.cpp | 1 + src/zm_logger.cpp | 2 +- src/zm_monitor.h | 1 + src/zm_monitorstream.cpp | 2 + src/zm_mpeg.cpp | 1 + src/zm_remote_camera_http.cpp | 1 + src/zm_remote_camera_nvsocket.cpp | 1 + src/zm_rtp_ctrl.h | 1 - src/zm_rtp_source.cpp | 1 + src/zm_rtp_source.h | 1 - src/zm_rtsp_server_thread.h | 1 - src/zm_sdp.cpp | 1 + src/zm_stream.cpp | 3 +- src/zm_thread.cpp | 317 ------------------------------ src/zm_thread.h | 266 ------------------------- src/zm_threaddata.cpp | 21 -- src/zmc.cpp | 1 + src/zms.cpp | 1 + src/zmu.cpp | 1 + 30 files changed, 26 insertions(+), 611 deletions(-) delete mode 100644 src/zm_thread.cpp delete mode 100644 src/zm_thread.h delete mode 100644 src/zm_threaddata.cpp diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index 0aa52f240..d40c76ffd 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -67,7 +67,6 @@ set(ZM_BIN_SRC_FILES zm_signal.cpp zm_stream.cpp zm_swscale.cpp - zm_thread.cpp zm_time.cpp zm_user.cpp zm_utils.cpp diff --git a/src/zm_buffer.cpp b/src/zm_buffer.cpp index e5d0d21c9..d8b4f6242 100644 --- a/src/zm_buffer.cpp +++ b/src/zm_buffer.cpp @@ -19,6 +19,8 @@ #include "zm_buffer.h" +#include + unsigned int Buffer::assign(const unsigned char *pStorage, unsigned int pSize) { if ( mAllocation < pSize ) { delete[] mStorage; diff --git a/src/zm_comms.h b/src/zm_comms.h index 30c217ddc..1b12d18a4 100644 --- a/src/zm_comms.h +++ b/src/zm_comms.h @@ -27,6 +27,7 @@ #include #include #include +#include #include #if defined(BSD) diff --git a/src/zm_db.h b/src/zm_db.h index 8ba2df4c7..eabbebca4 100644 --- a/src/zm_db.h +++ b/src/zm_db.h @@ -20,7 +20,6 @@ #ifndef ZM_DB_H #define ZM_DB_H -#include "zm_thread.h" #include #include #include diff --git a/src/zm_event.cpp b/src/zm_event.cpp index daf1dfd39..63cf327f4 100644 --- a/src/zm_event.cpp +++ b/src/zm_event.cpp @@ -28,6 +28,7 @@ #include "zm_videostore.h" #include #include +#include //#define USE_PREPARED_SQL 1 diff --git a/src/zm_event.h b/src/zm_event.h index 7ba4b03ba..1eb3f2a7f 100644 --- a/src/zm_event.h +++ b/src/zm_event.h @@ -20,6 +20,7 @@ #ifndef ZM_EVENT_H #define ZM_EVENT_H +#include "zm_config.h" #include "zm_define.h" #include "zm_storage.h" #include diff --git a/src/zm_eventstream.cpp b/src/zm_eventstream.cpp index da1263384..72fab520e 100644 --- a/src/zm_eventstream.cpp +++ b/src/zm_eventstream.cpp @@ -27,6 +27,8 @@ #include "zm_storage.h" #include #include +#include + #ifdef __FreeBSD__ #include #endif diff --git a/src/zm_fifo.cpp b/src/zm_fifo.cpp index 7f7632ba1..09b35062a 100644 --- a/src/zm_fifo.cpp +++ b/src/zm_fifo.cpp @@ -24,6 +24,7 @@ #include #include #include +#include #define RAW_BUFFER 512 diff --git a/src/zm_fifo_debug.cpp b/src/zm_fifo_debug.cpp index 9903fda2b..fe2113c63 100644 --- a/src/zm_fifo_debug.cpp +++ b/src/zm_fifo_debug.cpp @@ -23,6 +23,7 @@ #include "zm_signal.h" #include #include +#include #define RAW_BUFFER 512 static bool zm_fifodbg_inited = false; diff --git a/src/zm_fifo_stream.cpp b/src/zm_fifo_stream.cpp index 5734b142d..d88cff484 100644 --- a/src/zm_fifo_stream.cpp +++ b/src/zm_fifo_stream.cpp @@ -24,6 +24,7 @@ #include #include #include +#include #define RAW_BUFFER 512 bool FifoStream::sendRAWFrames() { diff --git a/src/zm_file_camera.cpp b/src/zm_file_camera.cpp index 24b9e74ca..af494ce38 100644 --- a/src/zm_file_camera.cpp +++ b/src/zm_file_camera.cpp @@ -21,6 +21,7 @@ #include "zm_packet.h" #include +#include FileCamera::FileCamera( const Monitor *monitor, diff --git a/src/zm_image.cpp b/src/zm_image.cpp index a06a23c5b..014ae783c 100644 --- a/src/zm_image.cpp +++ b/src/zm_image.cpp @@ -25,6 +25,7 @@ #include #include #include +#include static unsigned char y_table_global[] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20, 22, 23, 24, 25, 26, 27, 29, 30, 31, 32, 33, 34, 36, 37, 38, 39, 40, 41, 43, 44, 45, 46, 47, 48, 50, 51, 52, 53, 54, 55, 57, 58, 59, 60, 61, 62, 64, 65, 66, 67, 68, 69, 71, 72, 73, 74, 75, 76, 78, 79, 80, 81, 82, 83, 85, 86, 87, 88, 89, 90, 91, 93, 94, 95, 96, 97, 98, 100, 101, 102, 103, 104, 105, 107, 108, 109, 110, 111, 112, 114, 115, 116, 117, 118, 119, 121, 122, 123, 124, 125, 126, 128, 129, 130, 131, 132, 133, 135, 136, 137, 138, 139, 140, 142, 143, 144, 145, 146, 147, 149, 150, 151, 152, 153, 154, 156, 157, 158, 159, 160, 161, 163, 164, 165, 166, 167, 168, 170, 171, 172, 173, 174, 175, 176, 178, 179, 180, 181, 182, 183, 185, 186, 187, 188, 189, 190, 192, 193, 194, 195, 196, 197, 199, 200, 201, 202, 203, 204, 206, 207, 208, 209, 210, 211, 213, 214, 215, 216, 217, 218, 220, 221, 222, 223, 224, 225, 227, 228, 229, 230, 231, 232, 234, 235, 236, 237, 238, 239, 241, 242, 243, 244, 245, 246, 248, 249, 250, 251, 252, 253, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}; diff --git a/src/zm_logger.cpp b/src/zm_logger.cpp index 303500e20..3de84a180 100644 --- a/src/zm_logger.cpp +++ b/src/zm_logger.cpp @@ -21,10 +21,10 @@ #include "zm_db.h" #include "zm_utils.h" - #include #include #include +#include #ifdef __FreeBSD__ #include diff --git a/src/zm_monitor.h b/src/zm_monitor.h index f26e60e03..29863f998 100644 --- a/src/zm_monitor.h +++ b/src/zm_monitor.h @@ -27,6 +27,7 @@ #include "zm_image.h" #include "zm_packet.h" #include "zm_packetqueue.h" +#include "zm_utils.h" #include "zm_video.h" #include #include diff --git a/src/zm_monitorstream.cpp b/src/zm_monitorstream.cpp index 83524b2ff..fd618800e 100644 --- a/src/zm_monitorstream.cpp +++ b/src/zm_monitorstream.cpp @@ -26,6 +26,8 @@ #include #include #include +#include + #ifdef __FreeBSD__ #include #endif diff --git a/src/zm_mpeg.cpp b/src/zm_mpeg.cpp index 3b298705b..c0b60ed2a 100644 --- a/src/zm_mpeg.cpp +++ b/src/zm_mpeg.cpp @@ -22,6 +22,7 @@ #include "zm_logger.h" #include "zm_rgb.h" #include +#include #if HAVE_LIBAVCODEC extern "C" { diff --git a/src/zm_remote_camera_http.cpp b/src/zm_remote_camera_http.cpp index 8342c2766..e7458b251 100644 --- a/src/zm_remote_camera_http.cpp +++ b/src/zm_remote_camera_http.cpp @@ -27,6 +27,7 @@ #include #include #include +#include #ifdef SOLARIS #include // FIONREAD and friends diff --git a/src/zm_remote_camera_nvsocket.cpp b/src/zm_remote_camera_nvsocket.cpp index 4adc2d957..f66d31fcb 100644 --- a/src/zm_remote_camera_nvsocket.cpp +++ b/src/zm_remote_camera_nvsocket.cpp @@ -24,6 +24,7 @@ #include #include #include +#include #ifdef SOLARIS #include // FIONREAD and friends diff --git a/src/zm_rtp_ctrl.h b/src/zm_rtp_ctrl.h index 3c3706ef6..896b125a1 100644 --- a/src/zm_rtp_ctrl.h +++ b/src/zm_rtp_ctrl.h @@ -20,7 +20,6 @@ #ifndef ZM_RTP_CTRL_H #define ZM_RTP_CTRL_H -#include "zm_thread.h" #include #include diff --git a/src/zm_rtp_source.cpp b/src/zm_rtp_source.cpp index 1515c8eb4..0136dfb44 100644 --- a/src/zm_rtp_source.cpp +++ b/src/zm_rtp_source.cpp @@ -23,6 +23,7 @@ #include "zm_rtp_data.h" #include "zm_utils.h" #include +#include #if HAVE_LIBAVCODEC diff --git a/src/zm_rtp_source.h b/src/zm_rtp_source.h index 7869d016d..0d1bdee83 100644 --- a/src/zm_rtp_source.h +++ b/src/zm_rtp_source.h @@ -24,7 +24,6 @@ #include "zm_config.h" #include "zm_define.h" #include "zm_ffmpeg.h" -#include "zm_thread.h" #include #include #include diff --git a/src/zm_rtsp_server_thread.h b/src/zm_rtsp_server_thread.h index 5921ebe0d..9d02ec2a7 100644 --- a/src/zm_rtsp_server_thread.h +++ b/src/zm_rtsp_server_thread.h @@ -3,7 +3,6 @@ #include "zm_config.h" #include "zm_ffmpeg.h" -#include "zm_thread.h" #include "zm_rtsp_server_server_media_subsession.h" #include "zm_rtsp_server_fifo_source.h" #include diff --git a/src/zm_sdp.cpp b/src/zm_sdp.cpp index 247ce82cb..d8bea2802 100644 --- a/src/zm_sdp.cpp +++ b/src/zm_sdp.cpp @@ -20,6 +20,7 @@ #include "zm_sdp.h" #include "zm_config.h" +#include "zm_exception.h" #include "zm_logger.h" #if HAVE_LIBAVFORMAT diff --git a/src/zm_stream.cpp b/src/zm_stream.cpp index ecbed9434..1f141ec65 100644 --- a/src/zm_stream.cpp +++ b/src/zm_stream.cpp @@ -21,10 +21,11 @@ #include "zm_box.h" #include "zm_monitor.h" +#include #include #include #include -#include +#include StreamBase::~StreamBase() { #if HAVE_LIBAVCODEC diff --git a/src/zm_thread.cpp b/src/zm_thread.cpp deleted file mode 100644 index 63a8bdbb2..000000000 --- a/src/zm_thread.cpp +++ /dev/null @@ -1,317 +0,0 @@ -// -// ZoneMinder Thread Class Implementation, $Date$, $Revision$ -// Copyright (C) 2001-2008 Philip Coombes -// -// This program is free software; you can redistribute it and/or -// modify it under the terms of the GNU General Public License -// as published by the Free Software Foundation; either version 2 -// of the License, or (at your option) any later version. -// -// This program is distributed in the hope that it will be useful, -// but WITHOUT ANY WARRANTY; without even the implied warranty of -// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -// GNU General Public License for more details. -// -// You should have received a copy of the GNU General Public License -// along with this program; if not, write to the Free Software -// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. -// - -#include "zm_thread.h" - -#include "zm_logger.h" -#include "zm_utils.h" -#include -#include -#include -#include - -struct timespec getTimeout( int secs ) { - struct timespec timeout; - struct timeval temp_timeout; - gettimeofday(&temp_timeout, nullptr); - timeout.tv_sec = temp_timeout.tv_sec + secs; - timeout.tv_nsec = temp_timeout.tv_usec*1000; - return timeout; -} - -struct timespec getTimeout( double secs ) { - struct timespec timeout; - struct timeval temp_timeout; - gettimeofday( &temp_timeout, nullptr ); - timeout.tv_sec = temp_timeout.tv_sec + int(secs); - timeout.tv_nsec = temp_timeout.tv_usec += (long int)(1000000000.0*(secs-int(secs))); - if ( timeout.tv_nsec > 1000000000 ) { - timeout.tv_sec += 1; - timeout.tv_nsec -= 1000000000; - } - return timeout; -} - -Mutex::Mutex() { - if ( pthread_mutex_init(&mMutex, nullptr) < 0 ) - Error("Unable to create pthread mutex: %s", strerror(errno)); -} - -Mutex::~Mutex() { - if ( locked() ) - Warning("Destroying mutex when locked"); - if ( pthread_mutex_destroy(&mMutex) < 0 ) - Error("Unable to destroy pthread mutex: %s", strerror(errno)); -} - -int Mutex::try_lock() { - return pthread_mutex_trylock(&mMutex); -} -void Mutex::lock() { - if ( pthread_mutex_lock(&mMutex) < 0 ) - throw ThreadException(stringtf("Unable to lock pthread mutex: %s", strerror(errno))); - //Debug(3, "Lock"); -} - -bool Mutex::try_lock_for(int secs) { - struct timespec timeout = getTimeout(secs); - return pthread_mutex_timedlock(&mMutex, &timeout) == 0; -} - -bool Mutex::try_lock_for(double secs) { - struct timespec timeout = getTimeout(secs); - return pthread_mutex_timedlock(&mMutex, &timeout) == 0; -} - -void Mutex::unlock() { - if ( pthread_mutex_unlock(&mMutex) < 0 ) - throw ThreadException(stringtf("Unable to unlock pthread mutex: %s", strerror(errno))); - //Debug(3, "unLock"); -} - -bool Mutex::locked() { - int state = pthread_mutex_trylock(&mMutex); - if ( (state != 0) && (state != EBUSY) ) - throw ThreadException(stringtf("Unable to trylock pthread mutex: %s", strerror(errno))); - if ( state != EBUSY ) - unlock(); - return (state == EBUSY); -} - -RecursiveMutex::RecursiveMutex() { - pthread_mutexattr_t attr; - pthread_mutexattr_init(&attr); - pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE); - - if ( pthread_mutex_init(&mMutex, &attr) < 0 ) - Error("Unable to create pthread mutex: %s", strerror(errno)); -} - -Condition::Condition( Mutex &mutex ) : mMutex( mutex ) { - if ( pthread_cond_init(&mCondition, nullptr) < 0 ) - throw ThreadException(stringtf("Unable to create pthread condition: %s", strerror(errno))); -} - -Condition::~Condition() { - if ( pthread_cond_destroy( &mCondition ) < 0 ) - Error("Unable to destroy pthread condition: %s", strerror(errno)); -} - -void Condition::wait() { - // Locking done outside of this function - if ( pthread_cond_wait(&mCondition, mMutex.getMutex()) < 0 ) - throw ThreadException(stringtf("Unable to wait pthread condition: %s", strerror(errno))); -} - -bool Condition::wait(int secs) { - // Locking done outside of this function - Debug(8, "Waiting for %d seconds", secs); - struct timespec timeout = getTimeout(secs); - if ( - ( pthread_cond_timedwait(&mCondition, mMutex.getMutex(), &timeout) < 0 ) - && - ( errno != ETIMEDOUT ) - ) - throw ThreadException(stringtf("Unable to timedwait pthread condition: %s", strerror(errno))); - return errno != ETIMEDOUT; -} - -bool Condition::wait( double secs ) { - // Locking done outside of this function - struct timespec timeout = getTimeout( secs ); - if ( - (pthread_cond_timedwait( &mCondition, mMutex.getMutex(), &timeout ) < 0) - && - (errno != ETIMEDOUT) ) - throw ThreadException( stringtf( "Unable to timedwait pthread condition: %s", strerror(errno) ) ); - return errno != ETIMEDOUT; -} - -void Condition::signal() { - if ( pthread_cond_signal( &mCondition ) < 0 ) - throw ThreadException( stringtf( "Unable to signal pthread condition: %s", strerror(errno) ) ); -} - -void Condition::broadcast() { - if ( pthread_cond_broadcast( &mCondition ) < 0 ) - throw ThreadException( stringtf( "Unable to broadcast pthread condition: %s", strerror(errno) ) ); -} - -template const T ThreadData::getValue() const { - mMutex.lock(); - const T valueCopy = mValue; - mMutex.unlock(); - return valueCopy; -} - -template T ThreadData::setValue(const T value) { - mMutex.lock(); - const T valueCopy = mValue = value; - mMutex.unlock(); - return valueCopy; -} - -template const T ThreadData::getUpdatedValue() const { - Debug(8, "Waiting for value update, %p", this); - mMutex.lock(); - mChanged = false; - mCondition.wait(); - const T valueCopy = mValue; - mMutex.unlock(); - Debug(9, "Got value update, %p", this); - return valueCopy; -} - -template const T ThreadData::getUpdatedValue(double secs) const { - Debug(8, "Waiting for value update, %.2f secs, %p", secs, this); - mMutex.lock(); - mChanged = false; - //do { - mCondition.wait(secs); - //} while ( !mChanged ); - const T valueCopy = mValue; - mMutex.unlock(); - Debug(9, "Got value update, %p", this); - return valueCopy; -} - -template const T ThreadData::getUpdatedValue(int secs) const { - Debug(8, "Waiting for value update, %d secs, %p", secs, this); - mMutex.lock(); - mChanged = false; - //do { - mCondition.wait(secs); - //} while ( !mChanged ); - const T valueCopy = mValue; - mMutex.unlock(); - Debug(9, "Got value update, %p", this); - return valueCopy; -} - -template void ThreadData::updateValueSignal(const T value) { - Debug(8, "Updating value with signal, %p", this); - mMutex.lock(); - mValue = value; - mChanged = true; - mCondition.signal(); - mMutex.unlock(); - Debug(9, "Updated value, %p", this); -} - -template void ThreadData::updateValueBroadcast( const T value ) { - Debug(8, "Updating value with broadcast, %p", this); - mMutex.lock(); - mValue = value; - mChanged = true; - mCondition.broadcast(); - mMutex.unlock(); - Debug(9, "Updated value, %p", this); -} - -Thread::Thread() : - mThreadCondition( mThreadMutex ), - mPid( -1 ), - mStarted( false ), - mRunning( false ) -{ - Debug( 1, "Creating thread" ); -} - -Thread::~Thread() { - Debug( 1, "Destroying thread %d", mPid ); - if ( mStarted ) { - Warning("You should really join the thread before destroying it"); - join(); - } -} - -void *Thread::mThreadFunc( void *arg ) { - Debug( 2, "Invoking thread" ); - - Thread *thisPtr = (Thread *)arg; - thisPtr->status = 0; - try { - thisPtr->mThreadMutex.lock(); - thisPtr->mPid = thisPtr->id(); - thisPtr->mThreadCondition.signal(); - thisPtr->mThreadMutex.unlock(); - thisPtr->mRunning = true; - Debug(2,"Runnning"); - thisPtr->status = thisPtr->run(); - thisPtr->mRunning = false; - Debug( 2, "Exiting thread, status %p", (void *)&(thisPtr->status) ); - return (void *)&(thisPtr->status); - } catch ( const ThreadException &e ) { - Error( "%s", e.getMessage().c_str() ); - thisPtr->mRunning = false; - Debug( 2, "Exiting thread after exception, status %p", (void *)-1 ); - return (void *)-1; - } -} - -void Thread::start() { - Debug(4, "Starting thread" ); - if ( isThread() ) - throw ThreadException("Can't self start thread"); - mThreadMutex.lock(); - if ( !mStarted ) { - pthread_attr_t threadAttrs; - pthread_attr_init( &threadAttrs ); - pthread_attr_setscope( &threadAttrs, PTHREAD_SCOPE_SYSTEM ); - - mStarted = true; - if ( pthread_create( &mThread, &threadAttrs, mThreadFunc, this ) < 0 ) - throw ThreadException( stringtf( "Can't create thread: %s", strerror(errno) ) ); - pthread_attr_destroy( &threadAttrs ); - } else { - Error( "Attempt to start already running thread %d", mPid ); - } - mThreadCondition.wait(); - mThreadMutex.unlock(); - Debug(4, "Started thread %d", mPid); -} - -void Thread::join() { - Debug(1, "Joining thread %d", mPid); - if ( isThread() ) - throw ThreadException( "Can't self join thread" ); - mThreadMutex.lock(); - if ( mPid >= 0 ) { - if ( mStarted ) { - void *threadStatus = 0; - if ( pthread_join( mThread, &threadStatus ) < 0 ) - throw ThreadException( stringtf( "Can't join sender thread: %s", strerror(errno) ) ); - mStarted = false; - Debug( 1, "Thread %d exited, status %p", mPid, threadStatus ); - } else { - Warning( "Attempt to join already finished thread %d", mPid ); - } - } else { - Warning( "Attempt to join non-started thread %d", mPid ); - } - mThreadMutex.unlock(); - Debug( 1, "Joined thread %d", mPid ); -} - -void Thread::kill( int signal ) { - pthread_kill( mThread, signal ); -} - -// Some explicit template instantiations -#include "zm_threaddata.cpp" diff --git a/src/zm_thread.h b/src/zm_thread.h deleted file mode 100644 index 4cdadb370..000000000 --- a/src/zm_thread.h +++ /dev/null @@ -1,266 +0,0 @@ -// -// ZoneMinder Thread Class Interface, $Date$, $Revision$ -// Copyright (C) 2001-2008 Philip Coombes -// -// This program is free software; you can redistribute it and/or -// modify it under the terms of the GNU General Public License -// as published by the Free Software Foundation; either version 2 -// of the License, or (at your option) any later version. -// -// This program is distributed in the hope that it will be useful, -// but WITHOUT ANY WARRANTY; without even the implied warranty of -// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -// GNU General Public License for more details. -// -// You should have received a copy of the GNU General Public License -// along with this program; if not, write to the Free Software -// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. -// - -#ifndef ZM_THREAD_H -#define ZM_THREAD_H - -#include "zm_config.h" -#include "zm_exception.h" -#include "zm_utils.h" -#include -#include - -#ifdef HAVE_SYS_SYSCALL_H -#include -#endif // HAVE_SYS_SYSCALL_H - -#ifdef __FreeBSD__ -#include -#endif - -class ThreadException : public Exception { -private: -#ifndef SOLARIS - pid_t pid() { - pid_t tid; -#ifdef __FreeBSD__ - long lwpid; - thr_self(&lwpid); - tid = lwpid; -#else - #ifdef __FreeBSD_kernel__ - if ( (syscall(SYS_thr_self, &tid)) < 0 ) // Thread/Process id - # else - tid=syscall(SYS_gettid); - #endif -#endif - return tid; - } -#else - pthread_t pid() { return( pthread_self() ); } -#endif -public: - explicit ThreadException(const std::string &message) : - Exception(stringtf("(%d) ", (long int)pid())+message) - { - } -}; - -class Mutex { - friend class Condition; - - protected: - pthread_mutex_t mMutex; - - public: - Mutex(); - ~Mutex(); - - private: - pthread_mutex_t *getMutex() { - return &mMutex; - } - - public: - int try_lock(); - void lock(); - bool try_lock_for(int secs); - bool try_lock_for(double secs); - void unlock(); - bool locked(); -}; - -class RecursiveMutex : public Mutex { - public: - RecursiveMutex(); -}; - -class ScopedMutex { -private: - Mutex &mMutex; - -public: - explicit ScopedMutex( Mutex &mutex ) : mMutex( mutex ) { - mMutex.lock(); - } - ~ScopedMutex() { - mMutex.unlock(); - } - -private: - ScopedMutex( const ScopedMutex & ); -}; - -class Condition { -private: - Mutex &mMutex; - pthread_cond_t mCondition; - -public: - explicit Condition(Mutex &mutex); - ~Condition(); - - void wait(); - bool wait(int secs); - bool wait(double secs); - void signal(); - void broadcast(); -}; - -class Semaphore : public Condition { -private: - Mutex mMutex; - -public: - Semaphore() : Condition(mMutex) { - } - - void wait() { - mMutex.lock(); - Condition::wait(); - mMutex.unlock(); - } - bool wait(int secs) { - mMutex.lock(); - bool result = Condition::wait(secs); - mMutex.unlock(); - return result; - } - bool wait(double secs) { - mMutex.lock(); - bool result = Condition::wait(secs); - mMutex.unlock(); - return result; - } - void signal() { - mMutex.lock(); - Condition::signal(); - mMutex.unlock(); - } - void broadcast() { - mMutex.lock(); - Condition::broadcast(); - mMutex.unlock(); - } -}; - -template class ThreadData { -private: - T mValue; - mutable bool mChanged; - mutable Mutex mMutex; - mutable Condition mCondition; - -public: - __attribute__((used)) ThreadData() : - mValue(0), mCondition(mMutex) - { - mChanged = false; - } - explicit __attribute__((used)) ThreadData(T value) : - mValue(value), mCondition(mMutex) { - mChanged = false; - } - - __attribute__((used)) operator T() const { - return getValue(); - } - __attribute__((used)) const T operator=( const T value ) { - return setValue(value); - } - - __attribute__((used)) const T getValueImmediate() const { - return mValue; - } - __attribute__((used)) T setValueImmediate( const T value ) { - return mValue = value; - } - __attribute__((used)) const T getValue() const; - __attribute__((used)) T setValue( const T value ); - __attribute__((used)) const T getUpdatedValue() const; - __attribute__((used)) const T getUpdatedValue(double secs) const; - __attribute__((used)) const T getUpdatedValue(int secs) const; - __attribute__((used)) void updateValueSignal(const T value); - __attribute__((used)) void updateValueBroadcast(const T value); -}; - -class Thread { -public: - typedef void *(*ThreadFunc)( void * ); - -protected: - pthread_t mThread; - - Mutex mThreadMutex; - Condition mThreadCondition; -#ifndef SOLARIS - pid_t mPid; -#else - pthread_t mPid; -#endif - bool mStarted; - bool mRunning; - int status; // Used in various functions to get around return a local variable - -protected: - Thread(); - virtual ~Thread(); - -#ifndef SOLARIS - pid_t id() const { - pid_t tid; -#ifdef __FreeBSD__ - long lwpid; - thr_self(&lwpid); - tid = lwpid; -#else - #ifdef __FreeBSD_kernel__ - if ( (syscall(SYS_thr_self, &tid)) < 0 ) // Thread/Process id - - #else - tid=syscall(SYS_gettid); - #endif -#endif - return tid; - } -#else - pthread_t id() const { - return pthread_self(); - } -#endif - void exit( int p_status = 0 ) { - //INFO( "Exiting" ); - pthread_exit( (void *)&p_status ); - } - static void *mThreadFunc( void *arg ); - -public: - virtual int run() = 0; - - void start(); - void join(); - void kill( int signal ); - bool isThread() { - return( mPid > -1 && pthread_equal( pthread_self(), mThread ) ); - } - bool isStarted() const { return mStarted; } - bool isRunning() const { return mRunning; } -}; - -#endif // ZM_THREAD_H diff --git a/src/zm_threaddata.cpp b/src/zm_threaddata.cpp deleted file mode 100644 index 6b25d5714..000000000 --- a/src/zm_threaddata.cpp +++ /dev/null @@ -1,21 +0,0 @@ -// -// ZoneMinder Explicit Thread Template Class Instantiations, $Date$, $Revision$ -// Copyright (C) 2001-2008 Philip Coombes -// -// This program is free software; you can redistribute it and/or -// modify it under the terms of the GNU General Public License -// as published by the Free Software Foundation; either version 2 -// of the License, or (at your option) any later version. -// -// This program is distributed in the hope that it will be useful, -// but WITHOUT ANY WARRANTY; without even the implied warranty of -// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -// GNU General Public License for more details. -// -// You should have received a copy of the GNU General Public License -// along with this program; if not, write to the Free Software -// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. -// - -template class ThreadData; -template class ThreadData; diff --git a/src/zmc.cpp b/src/zmc.cpp index 268aca845..0d3b03145 100644 --- a/src/zmc.cpp +++ b/src/zmc.cpp @@ -66,6 +66,7 @@ possible, this should run at more or less constant speed. #include "zm_utils.h" #include #include +#include void Usage() { fprintf(stderr, "zmc -d or -r -H -P -p or -f or -m \n"); diff --git a/src/zms.cpp b/src/zms.cpp index b7d88d35b..89b20e30c 100644 --- a/src/zms.cpp +++ b/src/zms.cpp @@ -25,6 +25,7 @@ #include "zm_eventstream.h" #include "zm_fifo_stream.h" #include +#include bool ValidateAccess(User *user, int mon_id) { bool allowed = true; diff --git a/src/zmu.cpp b/src/zmu.cpp index 472f05043..cb4eb5efb 100644 --- a/src/zmu.cpp +++ b/src/zmu.cpp @@ -93,6 +93,7 @@ Options for use with monitors: #include "zm_monitor.h" #include "zm_local_camera.h" #include +#include void Usage(int status=-1) { fputs( From f986b6a5e214b65d149a26d7c409f80398d98b5f Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 4 Mar 2021 07:46:39 -0500 Subject: [PATCH 0114/1277] Clear packet counts on clear so that camera restarts don't incremenet the stream_id's and cause memory consumption --- src/zm_packetqueue.cpp | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/src/zm_packetqueue.cpp b/src/zm_packetqueue.cpp index 588355287..625f0e2ad 100644 --- a/src/zm_packetqueue.cpp +++ b/src/zm_packetqueue.cpp @@ -130,6 +130,10 @@ void PacketQueue::clearPackets(ZMPacket *add_packet) { *(pktQueue.begin()) != add_packet ) ) { + Debug(3, "stream index %d ?= video_stream_id %d, keyframe %d, counts %d > max %d at begin %d", + add_packet->packet.stream_index, video_stream_id, add_packet->keyframe, packet_counts[video_stream_id], max_video_packet_count, + ( *(pktQueue.begin()) != add_packet ) + ); return; } std::unique_lock lck(mutex); @@ -315,7 +319,6 @@ void PacketQueue::clear() { ZMPacket *packet = pktQueue.front(); // Someone might have this packet, but not for very long and since we have locked the queue they won't be able to get another one packet->lock(); - packet_counts[packet->packet.stream_index] -= 1; pktQueue.pop_front(); packet->unlock(); delete packet; @@ -329,6 +332,11 @@ void PacketQueue::clear() { packetqueue_iterator *iterator_it = *iterators_it; *iterator_it = pktQueue.begin(); } // end foreach iterator + + if ( packet_counts ) delete[] packet_counts; + packet_counts = nullptr; + max_stream_id = -1; + condition.notify_all(); } From 5259b7806517b3e4bc2f5b2a933096ba35584c66 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 4 Mar 2021 11:12:27 -0500 Subject: [PATCH 0115/1277] Fix event notes not getting populated. --- src/zm_event.cpp | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/src/zm_event.cpp b/src/zm_event.cpp index daf1dfd39..a190f94ab 100644 --- a/src/zm_event.cpp +++ b/src/zm_event.cpp @@ -273,20 +273,16 @@ Event::~Event() { } // Event::~Event() void Event::createNotes(std::string ¬es) { - if ( !notes.empty() ) { - notes.clear(); - for ( StringSetMap::const_iterator mapIter = noteSetMap.begin(); mapIter != noteSetMap.end(); ++mapIter ) { - notes += mapIter->first; - notes += ": "; - const StringSet &stringSet = mapIter->second; - for ( StringSet::const_iterator setIter = stringSet.begin(); setIter != stringSet.end(); ++setIter ) { - if ( setIter != stringSet.begin() ) - notes += ", "; - notes += *setIter; - } + notes.clear(); + for ( StringSetMap::const_iterator mapIter = noteSetMap.begin(); mapIter != noteSetMap.end(); ++mapIter ) { + notes += mapIter->first; + notes += ": "; + const StringSet &stringSet = mapIter->second; + for ( StringSet::const_iterator setIter = stringSet.begin(); setIter != stringSet.end(); ++setIter ) { + if ( setIter != stringSet.begin() ) + notes += ", "; + notes += *setIter; } - } else { - notes = ""; } } // void Event::createNotes(std::string ¬es) From 5ffaebf70da180c4cd1c0bfc3658d4441ca350fa Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Thu, 4 Mar 2021 19:21:09 +0100 Subject: [PATCH 0116/1277] RtpCtrlThread: Add a missing include --- src/zm_rtp_ctrl.h | 1 + 1 file changed, 1 insertion(+) diff --git a/src/zm_rtp_ctrl.h b/src/zm_rtp_ctrl.h index 896b125a1..9346496f7 100644 --- a/src/zm_rtp_ctrl.h +++ b/src/zm_rtp_ctrl.h @@ -21,6 +21,7 @@ #define ZM_RTP_CTRL_H #include +#include #include // Defined in ffmpeg rtp.h From 1adeda6241b7d542c3ebf680a52a79cf7aba776c Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 4 Mar 2021 13:24:05 -0500 Subject: [PATCH 0117/1277] Add debugging of sql --- src/zm_db.cpp | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/zm_db.cpp b/src/zm_db.cpp index e2b23c7e7..77bb20094 100644 --- a/src/zm_db.cpp +++ b/src/zm_db.cpp @@ -178,6 +178,7 @@ int zmDbDo(const char *query) { return rc; } } + Debug(1, "Success running sql query %s", query); return 1; } @@ -191,6 +192,7 @@ int zmDbDoInsert(const char *query) { return 0; } int id = mysql_insert_id(&dbconn); + Debug(1, "Success running sql insert %s. Resulting id is %d", query, id); return id; } From eb36c9e919d9022bb665d571426a7f45d77fc930 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 4 Mar 2021 13:24:26 -0500 Subject: [PATCH 0118/1277] add getFrequencyIndex and use it to correctly set the frequency index --- src/zm_rtsp_server_fifo_audio_source.cpp | 8 +++++--- src/zm_rtsp_server_fifo_audio_source.h | 8 ++++++-- 2 files changed, 11 insertions(+), 5 deletions(-) diff --git a/src/zm_rtsp_server_fifo_audio_source.cpp b/src/zm_rtsp_server_fifo_audio_source.cpp index 371ef6985..25e10f7f0 100644 --- a/src/zm_rtsp_server_fifo_audio_source.cpp +++ b/src/zm_rtsp_server_fifo_audio_source.cpp @@ -6,11 +6,8 @@ ** ** -------------------------------------------------------------------------*/ -#include "zm_logger.h" #include "zm_rtsp_server_fifo_audio_source.h" -#include - #if HAVE_RTSP_SERVER static unsigned const samplingFrequencyTable[16] = { @@ -35,4 +32,9 @@ ZoneMinderFifoAudioSource::ZoneMinderFifoAudioSource( channels(1) { } +int ZoneMinderFifoAudioSource::getFrequencyIndex() { + for (int i=0; i<16; i++) + if (samplingFrequencyTable[i] == frequency) return i; + return -1; +} #endif // HAVE_RTSP_SERVER diff --git a/src/zm_rtsp_server_fifo_audio_source.h b/src/zm_rtsp_server_fifo_audio_source.h index 84827237f..691652f8f 100644 --- a/src/zm_rtsp_server_fifo_audio_source.h +++ b/src/zm_rtsp_server_fifo_audio_source.h @@ -39,8 +39,12 @@ class ZoneMinderFifoAudioSource : public ZoneMinderFifoSource { virtual ~ZoneMinderFifoAudioSource() {} public: - void setFrequency(int p_frequency) { frequency = p_frequency; }; + void setFrequency(int p_frequency) { + frequency = p_frequency; + samplingFrequencyIndex = getFrequencyIndex(); + }; int getFrequency() { return frequency; }; + int getFrequencyIndex(); const char *configStr() const { return config.c_str(); }; void setChannels(int p_channels) { channels = p_channels; }; int getChannels() const { return channels; }; @@ -48,7 +52,7 @@ class ZoneMinderFifoAudioSource : public ZoneMinderFifoSource { protected: std::string config; int samplingFrequencyIndex; - int frequency; + unsigned int frequency; int channels; }; #endif // HAVE_RTSP_SERVER From 7da1e4845674217778b1e3fbb7b546a436a63963 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 4 Mar 2021 13:25:42 -0500 Subject: [PATCH 0119/1277] Fix delete=>delete[] on header. If we read_into the buffer more all our pointers can be invalidated. So use offsets instead --- src/zm_rtsp_server_fifo_source.cpp | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/src/zm_rtsp_server_fifo_source.cpp b/src/zm_rtsp_server_fifo_source.cpp index ee23d8167..cf603ae46 100644 --- a/src/zm_rtsp_server_fifo_source.cpp +++ b/src/zm_rtsp_server_fifo_source.cpp @@ -192,7 +192,7 @@ int ZoneMinderFifoSource::getNextFrame() { pts_ptr ++; data_size = atoi(content_length_ptr); pts = strtoll(pts_ptr, nullptr, 10); - delete header; + delete[] header; } else { Debug(1, "ZM header not found."); return -1; @@ -202,8 +202,9 @@ int ZoneMinderFifoSource::getNextFrame() { Debug(4, "ZM Packet didn't start at beginning of buffer %u. %c%c", header_start-m_buffer.head(), m_buffer[0], m_buffer[1]); } - unsigned char *packet_start = header_end+1; - unsigned int header_size = packet_start - m_buffer.head(); // includes any bytes before header + + // read_into may invalidate packet_start + unsigned int header_size = (header_end+1) /*packet_start*/ - m_buffer.head(); // includes any bytes before header int bytes_needed = data_size - (m_buffer.size() - header_size); if (bytes_needed > 0) { @@ -212,6 +213,7 @@ int ZoneMinderFifoSource::getNextFrame() { if ( bytes_read != bytes_needed ) return -1; } + unsigned char *packet_start = m_buffer.head() + header_size; // splitFrames modifies so make a copy unsigned int bytes_remaining = data_size; @@ -234,11 +236,12 @@ int ZoneMinderFifoSource::getNextFrame() { if (m_captureQueue.size() > 25) { // 1 sec at 25 fps NAL_Frame * f = m_captureQueue.front(); while (m_captureQueue.size() and ((tv.tv_sec - f->m_timestamp.tv_sec) > 2)) { - Debug(1, "Deleting Front NAL is %d seconds old", (tv.tv_sec - f->m_timestamp.tv_sec)); + Debug(1, "Deleting Front NAL is %d seconds old", (tv.tv_sec - f->m_timestamp.tv_sec)); m_captureQueue.pop_front(); delete f; f = m_captureQueue.front(); } + Debug(1, "Done clearing"); } m_captureQueue.push_back(frame); pthread_mutex_unlock(&m_mutex); From d20521569d3f7d8e589a1908d2166f604080a632 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 4 Mar 2021 13:25:57 -0500 Subject: [PATCH 0120/1277] Only add width and height if they have a value --- src/zm_rtsp_server_h264_fifo_source.cpp | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/zm_rtsp_server_h264_fifo_source.cpp b/src/zm_rtsp_server_h264_fifo_source.cpp index 979bca18d..9f72eea02 100644 --- a/src/zm_rtsp_server_h264_fifo_source.cpp +++ b/src/zm_rtsp_server_h264_fifo_source.cpp @@ -74,8 +74,9 @@ std::list< std::pair > H264_ZoneMinderFifoSource::splitF std::ostringstream os; os << "profile-level-id=" << std::hex << std::setw(6) << std::setfill('0') << profile_level_id; - os << ";sprop-parameter-sets=" << sps_base64 << "," << pps_base64; - os << "\r\n" << "a=x-dimensions:" << m_width << "," << m_height << "\r\n"; + os << ";sprop-parameter-sets=" << sps_base64 << "," << pps_base64 << "\r\n"; + if (!(m_width and m_height)) + os << "a=x-dimensions:" << m_width << "," << m_height << "\r\n"; m_auxLine.assign(os.str()); Debug(3, "auxLine: %s", m_auxLine.c_str()); From 5f476df194e65427d0d6743fd996fb083ba63c18 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 4 Mar 2021 13:35:39 -0500 Subject: [PATCH 0121/1277] Fix invalid read when no mAudioStream --- src/zm_ffmpeg_camera.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/zm_ffmpeg_camera.cpp b/src/zm_ffmpeg_camera.cpp index 621bb578f..a1d8fb418 100644 --- a/src/zm_ffmpeg_camera.cpp +++ b/src/zm_ffmpeg_camera.cpp @@ -203,8 +203,8 @@ int FfmpegCamera::Capture(ZMPacket &zm_packet) { Debug(1, "Using audio input"); } else { mFormatContextPtr = mFormatContext; - Debug(1, "Using video input beacuse %" PRId64 " >= %" PRId64, - av_rescale_q(mLastAudioPTS, mAudioStream->time_base, AV_TIME_BASE_Q), + Debug(1, "Using video input because %" PRId64 " >= %" PRId64, + (mAudioStream?av_rescale_q(mLastAudioPTS, mAudioStream->time_base, AV_TIME_BASE_Q):0), av_rescale_q(mLastVideoPTS, mVideoStream->time_base, AV_TIME_BASE_Q) ); } From fa22129966f93735698e2efa61476432b8ab66e2 Mon Sep 17 00:00:00 2001 From: Admin Date: Thu, 4 Mar 2021 20:20:02 +0100 Subject: [PATCH 0122/1277] Make last_alarm_count last alarmed frame while only in ALARM state to make post_event_count counts since here not alone alarmed frames that don't shot alarms. --- src/zm_monitor.cpp | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 7d00cf4d5..ef0ee05b2 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -2138,7 +2138,9 @@ bool Monitor::Analyse() { Debug(1, "Staying in %s", State_Strings[state].c_str()); } - last_alarm_count = analysis_image_count; + if ( state == ALARM ) { + last_alarm_count = analysis_image_count; + } // This is needed so post_event_count counts after last alarmed frames while in ALARM not single alarmed frames while ALERT } else { // no score? alert_to_alarm_frame_count = alarm_frame_count; // load same value configured for alarm_frame_count if (state == ALARM) { From 1ff4e5bc8dcc31ca2914083637bffcf8db645691 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 5 Mar 2021 10:02:12 -0500 Subject: [PATCH 0123/1277] Fix not keeping enough video packets in packetqueue to satisfy pre_event_count --- src/zm_packetqueue.cpp | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/src/zm_packetqueue.cpp b/src/zm_packetqueue.cpp index 625f0e2ad..40aeff3bd 100644 --- a/src/zm_packetqueue.cpp +++ b/src/zm_packetqueue.cpp @@ -140,6 +140,7 @@ void PacketQueue::clearPackets(ZMPacket *add_packet) { packetqueue_iterator it = pktQueue.begin(); packetqueue_iterator next_front = pktQueue.begin(); + int video_packets_to_delete = 0; // This is a count of how many packets we will delete so we know when to stop looking // First packet is special because we know it is a video keyframe and only need to check for lock ZMPacket *zm_packet = *it; @@ -162,9 +163,15 @@ void PacketQueue::clearPackets(ZMPacket *add_packet) { if ( zm_packet->packet.stream_index == video_stream_id ) { if ( zm_packet->keyframe ) { - Debug(1, "Have a video keyframe so setting next front to it"); + Debug(3, "Have a video keyframe so setting next front to it"); next_front = it; } + ++video_packets_to_delete; + Debug(4, "Counted %d video packets. Which would leave %d in packetqueue", + video_packets_to_delete, packet_counts[video_stream_id]-video_packets_to_delete); + if (packet_counts[video_stream_id] - video_packets_to_delete <= max_video_packet_count) { + break; + } } it++; } // end while @@ -526,7 +533,7 @@ packetqueue_iterator *PacketQueue::get_event_start_packet_it( // Step one count back pre_event_count frames as the minimum // Do not assume that snapshot_it is video // snapshot it might already point to the beginning - while ( ( (*it) != pktQueue.begin() ) and pre_event_count ) { + while (( (*it) != pktQueue.begin() ) and pre_event_count) { Debug(1, "Previous packet pre_event_count %d stream_index %d keyframe %d", pre_event_count, (*(*it))->packet.stream_index, (*(*it))->keyframe); ZM_DUMP_PACKET((*(*it))->packet, ""); From 0af68a091443ad37336982a7b7fa8d302cc6bf0a Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 5 Mar 2021 14:12:44 -0500 Subject: [PATCH 0124/1277] Implement read_into with a timeout --- src/zm_buffer.cpp | 24 ++++++++++++++++++++++++ src/zm_buffer.h | 1 + 2 files changed, 25 insertions(+) diff --git a/src/zm_buffer.cpp b/src/zm_buffer.cpp index d8b4f6242..fe1e01895 100644 --- a/src/zm_buffer.cpp +++ b/src/zm_buffer.cpp @@ -69,3 +69,27 @@ int Buffer::read_into(int sd, unsigned int bytes) { } return bytes_read; } + +int Buffer::read_into(int sd, unsigned int bytes, struct timeval timeout) { + // Make sure there is enough space + this->expand(bytes); + + fd_set set; + FD_ZERO(&set); /* clear the set */ + FD_SET(sd, &set); /* add our file descriptor to the set */ + + int rv = select(sd + 1, &set, NULL, NULL, &timeout); + if (rv == -1) { + Error("Error %d %s from select", errno, strerror(errno)); + return rv; + } else if (rv == 0) { + printf("timeout"); /* a timeout occured */ + return 0; + } + int bytes_read = read(sd, mTail, bytes); + if ( bytes_read > 0 ) { + mTail += bytes_read; + mSize += bytes_read; + } + return bytes_read; +} diff --git a/src/zm_buffer.h b/src/zm_buffer.h index d4df93a83..f15097fdc 100644 --- a/src/zm_buffer.h +++ b/src/zm_buffer.h @@ -182,6 +182,7 @@ class Buffer { return static_cast(mSize); } int read_into(int sd, unsigned int bytes); + int read_into(int sd, unsigned int bytes, struct timeval timeout); }; #endif // ZM_BUFFER_H From f99b8896ecb659828ecc5e868b1559518900925e Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 5 Mar 2021 14:13:10 -0500 Subject: [PATCH 0125/1277] Don't unlock the mutex around openFile. It is a recursive mutex so we should be able to stay locked --- src/zm_logger.cpp | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/zm_logger.cpp b/src/zm_logger.cpp index 3de84a180..914f9eefc 100644 --- a/src/zm_logger.cpp +++ b/src/zm_logger.cpp @@ -510,10 +510,11 @@ void Logger::logPrint(bool hex, const char * const filepath, const int line, con if (level <= mFileLevel) { if (!mLogFileFP) { // FIXME unlocking here is a problem. Another thread could sneak in. - log_mutex.unlock(); + // We are using a recursive mutex so unlocking shouldn't be neccessary + //log_mutex.unlock(); // We do this here so that we only create the file if we ever write to it. openFile(); - log_mutex.lock(); + //log_mutex.lock(); } if (mLogFileFP) { fputs(logString, mLogFileFP); From 07339e443b7f230c82ca299917276fa06262c88e Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 5 Mar 2021 14:13:24 -0500 Subject: [PATCH 0126/1277] Add RTSP Server shutdown code --- src/zm_rtsp_server.cpp | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/zm_rtsp_server.cpp b/src/zm_rtsp_server.cpp index 376a35a01..9e4b59152 100644 --- a/src/zm_rtsp_server.cpp +++ b/src/zm_rtsp_server.cpp @@ -223,6 +223,16 @@ int main(int argc, char *argv[]) { zm_reload = false; } // end if zm_reload } // end while ! zm_terminate + Info("RTSP Server shutting down"); + + for (size_t i = 0; i < monitors.size(); i++) { + if (sessions[i]) { + Debug(1, "Removing session for %s", monitors[i]->Name()); + rtsp_server_thread->removeSession(sessions[i]); + sessions[i] = nullptr; + } + } // end foreach monitor + rtsp_server_thread->Stop(); delete[] sessions; sessions = nullptr; From ebd29a3cb9851c7817eff9fc2b73e280d55b5567 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 5 Mar 2021 14:18:12 -0500 Subject: [PATCH 0127/1277] use the timeout version of read_into so that we don't stay blocked while we have been told to exit. If getNextFrame returns -1 sleep for a second. --- src/zm_rtsp_server_fifo_source.cpp | 36 +++++++++++++++++------------- 1 file changed, 21 insertions(+), 15 deletions(-) diff --git a/src/zm_rtsp_server_fifo_source.cpp b/src/zm_rtsp_server_fifo_source.cpp index cf603ae46..81bd42f30 100644 --- a/src/zm_rtsp_server_fifo_source.cpp +++ b/src/zm_rtsp_server_fifo_source.cpp @@ -46,6 +46,7 @@ ZoneMinderFifoSource::~ZoneMinderFifoSource() { m_captureQueue.pop_front(); delete f; } + Debug(1, "Deleting Fifo Source done"); pthread_mutex_destroy(&m_mutex); } @@ -54,7 +55,8 @@ void* ZoneMinderFifoSource::thread() { stop = 0; while (!stop) { - getNextFrame(); + if ( getNextFrame() < 0 ) + sleep(1); } return nullptr; } @@ -119,6 +121,7 @@ void ZoneMinderFifoSource::deliverFrame() { // FrameSource callback on read event void ZoneMinderFifoSource::incomingPacketHandler() { + Debug(1, "incomingPacketHandler"); if (this->getNextFrame() <= 0) { handleClosure(this); } @@ -126,7 +129,10 @@ void ZoneMinderFifoSource::incomingPacketHandler() { // read from monitor int ZoneMinderFifoSource::getNextFrame() { - if (zm_terminate) return -1; + if (zm_terminate or stop) { + Debug(1, "Terminating %d %d", zm_terminate, stop); + return -1; + } if (m_fd == -1) { Debug(1, "Opening fifo %s", m_fifo.c_str()); @@ -137,7 +143,7 @@ int ZoneMinderFifoSource::getNextFrame() { } } - int bytes_read = m_buffer.read_into(m_fd, 4096); + int bytes_read = m_buffer.read_into(m_fd, 4096, {1,0}); if (bytes_read == 0) { Debug(3, "No bytes read"); sleep(1); @@ -152,7 +158,6 @@ int ZoneMinderFifoSource::getNextFrame() { Debug(4, "%s bytes read %d bytes, buffer size %u", m_fifo.c_str(), bytes_read, m_buffer.size()); while (m_buffer.size()) { - unsigned int data_size = 0; int64_t pts; unsigned char *header_end = nullptr; @@ -164,7 +169,7 @@ int ZoneMinderFifoSource::getNextFrame() { if (!header_end) { // Must not have enough data. So... keep all. Debug(1, "Didn't find newline"); - return -1; + return 0; } unsigned int header_size = header_end-header_start; @@ -176,8 +181,8 @@ int ZoneMinderFifoSource::getNextFrame() { if (!content_length_ptr) { Debug(1, "Didn't find space delineating size in %s", header); m_buffer.consume(header_start-m_buffer.head() + 2); - delete header; - return -1; + delete[] header; + return 0; } *content_length_ptr = '\0'; content_length_ptr ++; @@ -185,8 +190,8 @@ int ZoneMinderFifoSource::getNextFrame() { if (!pts_ptr) { m_buffer.consume(header_start-m_buffer.head() + 2); Debug(1, "Didn't find space delineating pts in %s", header); - delete header; - return -1; + delete[] header; + return 0; } *pts_ptr = '\0'; pts_ptr ++; @@ -194,8 +199,8 @@ int ZoneMinderFifoSource::getNextFrame() { pts = strtoll(pts_ptr, nullptr, 10); delete[] header; } else { - Debug(1, "ZM header not found."); - return -1; + Debug(1, "ZM header not found %s.",m_buffer.head()); + return 0; } Debug(4, "ZM Packet size %u pts %" PRId64, data_size, pts); if (header_start != m_buffer) { @@ -209,9 +214,11 @@ int ZoneMinderFifoSource::getNextFrame() { int bytes_needed = data_size - (m_buffer.size() - header_size); if (bytes_needed > 0) { Debug(4, "Need another %d bytes. Trying to read them", bytes_needed); - int bytes_read = m_buffer.read_into(m_fd, bytes_needed); - if ( bytes_read != bytes_needed ) + int bytes_read = m_buffer.read_into(m_fd, bytes_needed, {1,0}); + if ( bytes_read != bytes_needed ) { + Debug(4, "Failed to read another %d bytes.", bytes_needed); return -1; + } } unsigned char *packet_start = m_buffer.head() + header_size; @@ -236,12 +243,11 @@ int ZoneMinderFifoSource::getNextFrame() { if (m_captureQueue.size() > 25) { // 1 sec at 25 fps NAL_Frame * f = m_captureQueue.front(); while (m_captureQueue.size() and ((tv.tv_sec - f->m_timestamp.tv_sec) > 2)) { - Debug(1, "Deleting Front NAL is %d seconds old", (tv.tv_sec - f->m_timestamp.tv_sec)); m_captureQueue.pop_front(); delete f; f = m_captureQueue.front(); } - Debug(1, "Done clearing"); + Debug(3, "Done clearing. Queue size is now %d", m_captureQueue.size()); } m_captureQueue.push_back(frame); pthread_mutex_unlock(&m_mutex); From cfb8e062c17087f8f49b604474a70da4eab790a9 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 5 Mar 2021 14:18:51 -0500 Subject: [PATCH 0128/1277] Add deleting sources to RTSPServerThread::Stop(). Delete redundant terminate --- src/zm_rtsp_server_thread.cpp | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/src/zm_rtsp_server_thread.cpp b/src/zm_rtsp_server_thread.cpp index c337d669f..c8fdd9fdb 100644 --- a/src/zm_rtsp_server_thread.cpp +++ b/src/zm_rtsp_server_thread.cpp @@ -12,7 +12,7 @@ #include RTSPServerThread::RTSPServerThread(int port) : - terminate_(false), scheduler_watch_var_(0) + scheduler_watch_var_(0) { //unsigned short rtsp_over_http_port = 0; //const char *realm = "ZoneMinder"; @@ -64,7 +64,6 @@ void RTSPServerThread::Run() { void RTSPServerThread::Stop() { Debug(1, "RTSPServerThread::stop()"); - terminate_ = true; { std::lock_guard lck(scheduler_watch_var_mutex_); @@ -72,8 +71,15 @@ void RTSPServerThread::Stop() { } for ( std::list::iterator it = sources.begin(); it != sources.end(); ++it ) { + Debug(1, "RTSPServerThread::stopping source"); (*it)->stopGettingFrames(); } + while ( sources.size() ) { + Debug(1, "RTSPServerThread::stop closing source"); + FramedSource *source = sources.front(); + sources.pop_front(); + Medium::close(source); + } } ServerMediaSession *RTSPServerThread::addSession(std::string &streamname) { From c96cb1dd8d912adb42273ea6b6b4026d837fe357 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 5 Mar 2021 14:21:18 -0500 Subject: [PATCH 0129/1277] Put back terminate_ --- src/zm_rtsp_server_thread.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/zm_rtsp_server_thread.cpp b/src/zm_rtsp_server_thread.cpp index c8fdd9fdb..7f989f987 100644 --- a/src/zm_rtsp_server_thread.cpp +++ b/src/zm_rtsp_server_thread.cpp @@ -12,7 +12,7 @@ #include RTSPServerThread::RTSPServerThread(int port) : - scheduler_watch_var_(0) + terminate_(false), scheduler_watch_var_(0) { //unsigned short rtsp_over_http_port = 0; //const char *realm = "ZoneMinder"; @@ -64,6 +64,7 @@ void RTSPServerThread::Run() { void RTSPServerThread::Stop() { Debug(1, "RTSPServerThread::stop()"); + terminate_ = true; { std::lock_guard lck(scheduler_watch_var_mutex_); From 9e77324de4e65deb6b39371afb99ce345e8ab8fa Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Fri, 5 Mar 2021 09:31:10 +0100 Subject: [PATCH 0130/1277] Replace raw mysql_query calls with the zmDb* functions With this we can make sure we have proper locking of our DB connection at all times. --- src/zm_config.cpp | 12 ++----- src/zm_eventstream.cpp | 75 ++++++++++++++---------------------------- src/zm_monitor.cpp | 14 +++----- src/zm_user.cpp | 71 +++++++++++++-------------------------- src/zm_zone.cpp | 26 +++++---------- src/zmu.cpp | 14 +++----- 6 files changed, 67 insertions(+), 145 deletions(-) diff --git a/src/zm_config.cpp b/src/zm_config.cpp index f7cd609bf..97e2f8c2a 100644 --- a/src/zm_config.cpp +++ b/src/zm_config.cpp @@ -340,16 +340,11 @@ Config::~Config() { } void Config::Load() { - if ( mysql_query(&dbconn, "SELECT `Name`, `Value`, `Type` FROM `Config` ORDER BY `Id`") ) { - Error("Can't run query: %s", mysql_error(&dbconn)); - exit(mysql_errno(&dbconn)); + MYSQL_RES *result = zmDbFetch("SELECT `Name`, `Value`, `Type` FROM `Config` ORDER BY `Id`"); + if (!result) { + exit(-1); } - MYSQL_RES *result = mysql_store_result(&dbconn); - if ( !result ) { - Error("Can't use query result: %s", mysql_error(&dbconn)); - exit(mysql_errno(&dbconn)); - } n_items = mysql_num_rows(result); if ( n_items <= ZM_MAX_CFG_ID ) { @@ -362,7 +357,6 @@ void Config::Load() { items[i] = new ConfigItem(dbrow[0], dbrow[1], dbrow[2]); } mysql_free_result(result); - result = nullptr; } void Config::Assign() { diff --git a/src/zm_eventstream.cpp b/src/zm_eventstream.cpp index 72fab520e..5ed424b7f 100644 --- a/src/zm_eventstream.cpp +++ b/src/zm_eventstream.cpp @@ -41,22 +41,14 @@ const std::string EventStream::StreamMode_Strings[4] = { }; bool EventStream::loadInitialEventData(int monitor_id, time_t event_time) { - static char sql[ZM_SQL_SML_BUFSIZ]; + std::string sql = stringtf("SELECT `Id` FROM `Events` WHERE " + "`MonitorId` = %d AND unix_timestamp(`EndDateTime`) > %ld " + "ORDER BY `Id` ASC LIMIT 1", monitor_id, event_time); - snprintf(sql, sizeof(sql), "SELECT `Id` FROM `Events` WHERE " - "`MonitorId` = %d AND unix_timestamp(`EndDateTime`) > %ld " - "ORDER BY `Id` ASC LIMIT 1", monitor_id, event_time); + MYSQL_RES *result = zmDbFetch(sql.c_str()); + if (!result) + exit(-1); - if ( mysql_query(&dbconn, sql) ) { - Error("Can't run query: %s", mysql_error(&dbconn)); - exit(mysql_errno(&dbconn)); - } - - MYSQL_RES *result = mysql_store_result(&dbconn); - if ( !result ) { - Error("Can't use query result: %s", mysql_error(&dbconn)); - exit(mysql_errno(&dbconn)); - } MYSQL_ROW dbrow = mysql_fetch_row(result); if ( mysql_errno(&dbconn) ) { @@ -115,23 +107,15 @@ bool EventStream::loadInitialEventData( } bool EventStream::loadEventData(uint64_t event_id) { - static char sql[ZM_SQL_MED_BUFSIZ]; - - snprintf(sql, sizeof(sql), + std::string sql = stringtf( "SELECT `MonitorId`, `StorageId`, `Frames`, unix_timestamp( `StartDateTime` ) AS StartTimestamp, " "unix_timestamp( `EndDateTime` ) AS EndTimestamp, " "(SELECT max(`Delta`)-min(`Delta`) FROM `Frames` WHERE `EventId`=`Events`.`Id`) AS FramesDuration, " "`DefaultVideo`, `Scheme`, `SaveJPEGs`, `Orientation`+0 FROM `Events` WHERE `Id` = %" PRIu64, event_id); - if ( mysql_query(&dbconn, sql) ) { - Error("Can't run query: %s", mysql_error(&dbconn)); - exit(mysql_errno(&dbconn)); - } - - MYSQL_RES *result = mysql_store_result(&dbconn); - if ( !result ) { - Error("Can't use query result: %s", mysql_error(&dbconn)); - exit(mysql_errno(&dbconn)); + MYSQL_RES *result = zmDbFetch(sql.c_str()); + if (!result) { + exit(-1); } if ( !mysql_num_rows(result) ) { @@ -228,17 +212,12 @@ bool EventStream::loadEventData(uint64_t event_id) { updateFrameRate((event_data->frame_count and event_data->duration) ? (double)event_data->frame_count/event_data->duration : 1); - snprintf(sql, sizeof(sql), "SELECT `FrameId`, unix_timestamp(`TimeStamp`), `Delta` " - "FROM `Frames` WHERE `EventId` = %" PRIu64 " ORDER BY `FrameId` ASC", event_id); - if ( mysql_query(&dbconn, sql) ) { - Error("Can't run query: %s", mysql_error(&dbconn)); - exit(mysql_errno(&dbconn)); - } + sql = stringtf("SELECT `FrameId`, unix_timestamp(`TimeStamp`), `Delta` " + "FROM `Frames` WHERE `EventId` = %" PRIu64 " ORDER BY `FrameId` ASC", event_id); - result = mysql_store_result(&dbconn); - if ( !result ) { - Error("Can't use query result: %s", mysql_error(&dbconn)); - exit(mysql_errno(&dbconn)); + result = zmDbFetch(sql.c_str()); + if (!result) { + exit(-1); } event_data->n_frames = mysql_num_rows(result); @@ -604,10 +583,10 @@ void EventStream::processCommand(const CmdMsg *msg) { } // void EventStream::processCommand(const CmdMsg *msg) bool EventStream::checkEventLoaded() { - static char sql[ZM_SQL_SML_BUFSIZ]; + std::string sql; if ( curr_frame_id <= 0 ) { - snprintf(sql, sizeof(sql), + sql = stringtf( "SELECT `Id` FROM `Events` WHERE `MonitorId` = %d AND `Id` < %" PRIu64 " ORDER BY `Id` DESC LIMIT 1", event_data->monitor_id, event_data->event_id); } else if ( (unsigned int)curr_frame_id > event_data->last_frame_id ) { @@ -618,7 +597,7 @@ bool EventStream::checkEventLoaded() { curr_frame_id = event_data->last_frame_id; return false; } - snprintf(sql, sizeof(sql), + sql = stringtf( "SELECT `Id` FROM `Events` WHERE `MonitorId` = %d AND `Id` > %" PRIu64 " ORDER BY `Id` ASC LIMIT 1", event_data->monitor_id, event_data->event_id); } else { @@ -630,19 +609,15 @@ bool EventStream::checkEventLoaded() { // Event change required. if ( forceEventChange || ( (mode != MODE_SINGLE) && (mode != MODE_NONE) ) ) { - Debug(1, "Checking for next event %s", sql); - if ( mysql_query(&dbconn, sql) ) { - Error("Can't run query: %s", mysql_error(&dbconn)); - exit(mysql_errno(&dbconn)); + Debug(1, "Checking for next event %s", sql.c_str()); + + MYSQL_RES *result = zmDbFetch(sql.c_str()); + if (!result) { + exit(-1); } - MYSQL_RES *result = mysql_store_result(&dbconn); - if ( !result ) { - Error("Can't use query result: %s", mysql_error(&dbconn)); - exit(mysql_errno(&dbconn)); - } if ( mysql_num_rows(result) != 1 ) { - Debug(1, "No rows returned for %s", sql); + Debug(1, "No rows returned for %s", sql.c_str()); } MYSQL_ROW dbrow = mysql_fetch_row(result); @@ -664,7 +639,7 @@ bool EventStream::checkEventLoaded() { Debug(2, "New frame id = %d", curr_frame_id); return true; } else { - Debug(2, "No next event loaded using %s. Pausing", sql); + Debug(2, "No next event loaded using %s. Pausing", sql.c_str()); if ( curr_frame_id <= 0 ) curr_frame_id = 1; else diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index ef0ee05b2..eab019ab5 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -2391,25 +2391,19 @@ void Monitor::ReloadLinkedMonitors(const char *p_linked_monitors) { for ( int i = 0; i < n_link_ids; i++ ) { Debug(1, "Checking linked monitor %d", link_ids[i]); - std::lock_guard lck(db_mutex); - static char sql[ZM_SQL_SML_BUFSIZ]; - snprintf(sql, sizeof(sql), + std::string sql = stringtf( "SELECT `Id`, `Name` FROM `Monitors`" " WHERE `Id` = %d" " AND `Function` != 'None'" " AND `Function` != 'Monitor'" " AND `Enabled`=1", link_ids[i]); - if (mysql_query(&dbconn, sql)) { - Error("Can't run query: %s", mysql_error(&dbconn)); + + MYSQL_RES *result = zmDbFetch(sql.c_str()); + if (!result) { continue; } - MYSQL_RES *result = mysql_store_result(&dbconn); - if ( !result ) { - Error("Can't use query result: %s", mysql_error(&dbconn)); - continue; - } int n_monitors = mysql_num_rows(result); if ( n_monitors == 1 ) { MYSQL_ROW dbrow = mysql_fetch_row(result); diff --git a/src/zm_user.cpp b/src/zm_user.cpp index dfba0d51d..512718ccb 100644 --- a/src/zm_user.cpp +++ b/src/zm_user.cpp @@ -94,33 +94,25 @@ bool User::canAccess(int monitor_id) { // Function to load a user from username and password // Please note that in auth relay mode = none, password is NULL User *zmLoadUser(const char *username, const char *password) { - char sql[ZM_SQL_MED_BUFSIZ] = ""; int username_length = strlen(username); - char *safer_username = new char[(username_length * 2) + 1]; // According to docs, size of safer_whatever must be 2*length+1 // due to unicode conversions + null terminator. - mysql_real_escape_string(&dbconn, safer_username, username, username_length); + std::string escaped_username((username_length * 2) + 1, '\0'); - snprintf(sql, sizeof(sql), - "SELECT `Id`, `Username`, `Password`, `Enabled`," - " `Stream`+0, `Events`+0, `Control`+0, `Monitors`+0, `System`+0," - " `MonitorIds`" - " FROM `Users` WHERE `Username` = '%s' AND `Enabled` = 1", - safer_username); - delete[] safer_username; - safer_username = nullptr; - if ( mysql_query(&dbconn, sql) ) { - Error("Can't run query: %s", mysql_error(&dbconn)); - exit(mysql_errno(&dbconn)); - } + size_t escaped_len = mysql_real_escape_string(&dbconn, &escaped_username[0], username, username_length); + escaped_username.resize(escaped_len); - MYSQL_RES *result = mysql_store_result(&dbconn); - if ( !result ) { - Error("Can't use query result: %s", mysql_error(&dbconn)); - exit(mysql_errno(&dbconn)); - } + std::string sql = stringtf("SELECT `Id`, `Username`, `Password`, `Enabled`," + " `Stream`+0, `Events`+0, `Control`+0, `Monitors`+0, `System`+0," + " `MonitorIds`" + " FROM `Users` WHERE `Username` = '%s' AND `Enabled` = 1", + escaped_username.c_str()); + + MYSQL_RES *result = zmDbFetch(sql.c_str()); + if (!result) + return nullptr; if ( mysql_num_rows(result) == 1 ) { MYSQL_ROW dbrow = mysql_fetch_row(result); @@ -165,22 +157,13 @@ User *zmLoadTokenUser(std::string jwt_token_str, bool use_remote_addr) { return nullptr; } - char sql[ZM_SQL_MED_BUFSIZ] = ""; - snprintf(sql, sizeof(sql), - "SELECT `Id`, `Username`, `Password`, `Enabled`, `Stream`+0, `Events`+0," - " `Control`+0, `Monitors`+0, `System`+0, `MonitorIds`, `TokenMinExpiry`" - " FROM `Users` WHERE `Username` = '%s' AND `Enabled` = 1", username.c_str()); + std::string sql = stringtf("SELECT `Id`, `Username`, `Password`, `Enabled`, `Stream`+0, `Events`+0," + " `Control`+0, `Monitors`+0, `System`+0, `MonitorIds`, `TokenMinExpiry`" + " FROM `Users` WHERE `Username` = '%s' AND `Enabled` = 1", username.c_str()); - if ( mysql_query(&dbconn, sql) ) { - Error("Can't run query: %s", mysql_error(&dbconn)); + MYSQL_RES *result = zmDbFetch(sql.c_str()); + if (!result) return nullptr; - } - - MYSQL_RES *result = mysql_store_result(&dbconn); - if ( !result ) { - Error("Can't use query result: %s", mysql_error(&dbconn)); - return nullptr; - } int n_users = mysql_num_rows(result); if ( n_users != 1 ) { @@ -227,22 +210,14 @@ User *zmLoadAuthUser(const char *auth, bool use_remote_addr) { } Debug(1, "Attempting to authenticate user from auth string '%s', remote addr(%s)", auth, remote_addr); - char sql[ZM_SQL_SML_BUFSIZ] = ""; - snprintf(sql, sizeof(sql), - "SELECT `Id`, `Username`, `Password`, `Enabled`," - " `Stream`+0, `Events`+0, `Control`+0, `Monitors`+0, `System`+0," - " `MonitorIds` FROM `Users` WHERE `Enabled` = 1"); + std::string sql = "SELECT `Id`, `Username`, `Password`, `Enabled`," + " `Stream`+0, `Events`+0, `Control`+0, `Monitors`+0, `System`+0," + " `MonitorIds` FROM `Users` WHERE `Enabled` = 1"; - if ( mysql_query(&dbconn, sql) ) { - Error("Can't run query: %s", mysql_error(&dbconn)); - exit(mysql_errno(&dbconn)); - } - - MYSQL_RES *result = mysql_store_result(&dbconn); - if ( !result ) { - Error("Can't use query result: %s", mysql_error(&dbconn)); + MYSQL_RES *result = zmDbFetch(sql.c_str()); + if (!result) return nullptr; - } + int n_users = mysql_num_rows(result); if ( n_users < 1 ) { mysql_free_result(result); diff --git a/src/zm_zone.cpp b/src/zm_zone.cpp index 749fff387..77703745c 100644 --- a/src/zm_zone.cpp +++ b/src/zm_zone.cpp @@ -819,28 +819,18 @@ bool Zone::ParseZoneString(const char *zone_string, int &zone_id, int &colour, P } // end bool Zone::ParseZoneString(const char *zone_string, int &zone_id, int &colour, Polygon &polygon) int Zone::Load(Monitor *monitor, Zone **&zones) { - static char sql[ZM_SQL_MED_BUFSIZ]; - MYSQL_RES *result; + std::string sql = stringtf("SELECT Id,Name,Type+0,Units,Coords,AlarmRGB,CheckMethod+0," + "MinPixelThreshold,MaxPixelThreshold,MinAlarmPixels,MaxAlarmPixels," + "FilterX,FilterY,MinFilterPixels,MaxFilterPixels," + "MinBlobPixels,MaxBlobPixels,MinBlobs,MaxBlobs," + "OverloadFrames,ExtendAlarmFrames" + " FROM Zones WHERE MonitorId = %d ORDER BY Type, Id", monitor->Id()); - { // scope for lock - std::lock_guard lck(db_mutex); - snprintf(sql, sizeof(sql), "SELECT Id,Name,Type+0,Units,Coords,AlarmRGB,CheckMethod+0," - "MinPixelThreshold,MaxPixelThreshold,MinAlarmPixels,MaxAlarmPixels," - "FilterX,FilterY,MinFilterPixels,MaxFilterPixels," - "MinBlobPixels,MaxBlobPixels,MinBlobs,MaxBlobs," - "OverloadFrames,ExtendAlarmFrames" - " FROM Zones WHERE MonitorId = %d ORDER BY Type, Id", monitor->Id()); - if ( mysql_query(&dbconn, sql) ) { - Error("Can't run query: %s", mysql_error(&dbconn)); - return 0; - } - - result = mysql_store_result(&dbconn); - } + MYSQL_RES *result = zmDbFetch(sql.c_str()); if (!result) { - Error("Can't use query result: %s", mysql_error(&dbconn)); return 0; } + int n_zones = mysql_num_rows(result); Debug(1, "Got %d zones for monitor %s", n_zones, monitor->Name()); delete[] zones; diff --git a/src/zmu.cpp b/src/zmu.cpp index cb4eb5efb..e6c328cca 100644 --- a/src/zmu.cpp +++ b/src/zmu.cpp @@ -721,20 +721,14 @@ int main(int argc, char *argv[]) { if ( function & ZMU_LIST ) { std::string sql = "SELECT `Id`, `Function`+0 FROM `Monitors`"; - if ( !verbose ) { + if (!verbose) { sql += "WHERE `Function` != 'None'"; } sql += " ORDER BY Id ASC"; - if ( mysql_query(&dbconn, sql.c_str()) ) { - Error("Can't run query: %s", mysql_error(&dbconn)); - exit_zmu(mysql_errno(&dbconn)); - } - - MYSQL_RES *result = mysql_store_result(&dbconn); - if ( !result ) { - Error("Can't use query result: %s", mysql_error(&dbconn)); - exit_zmu(mysql_errno(&dbconn)); + MYSQL_RES *result = zmDbFetch(sql.c_str()); + if (!result) { + exit_zmu(-1); } Debug(1, "Got %d monitors", mysql_num_rows(result)); From 0796a2262e43936ba50a631bb899a841652c2f76 Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Fri, 5 Mar 2021 22:24:33 +0100 Subject: [PATCH 0131/1277] Utils: Replace stringtf with a type-safe version that can't overflow --- cmake/compiler/gcc/settings.cmake | 1 + src/zm_remote_camera_http.cpp | 6 +++--- src/zm_utils.cpp | 28 ---------------------------- src/zm_utils.h | 15 ++++++++++++--- 4 files changed, 16 insertions(+), 34 deletions(-) diff --git a/cmake/compiler/gcc/settings.cmake b/cmake/compiler/gcc/settings.cmake index b037a4ea6..87ff6e939 100644 --- a/cmake/compiler/gcc/settings.cmake +++ b/cmake/compiler/gcc/settings.cmake @@ -2,6 +2,7 @@ target_compile_options(zm-warning-interface INTERFACE -Wall -Wextra + -Wformat-security -Wno-cast-function-type -Wno-type-limits -Wno-unused-parameter) diff --git a/src/zm_remote_camera_http.cpp b/src/zm_remote_camera_http.cpp index e7458b251..d3ba95df0 100644 --- a/src/zm_remote_camera_http.cpp +++ b/src/zm_remote_camera_http.cpp @@ -105,7 +105,7 @@ void RemoteCameraHttp::Initialise() { request += stringtf( "User-Agent: %s/%s\r\n", config.http_ua, ZM_VERSION ); request += stringtf( "Host: %s\r\n", host.c_str()); if ( strcmp( config.http_version, "1.0" ) == 0 ) - request += stringtf( "Connection: Keep-Alive\r\n" ); + request += "Connection: Keep-Alive\r\n"; if ( !auth.empty() ) request += stringtf( "Authorization: Basic %s\r\n", auth64.c_str() ); request += "\r\n"; @@ -362,7 +362,7 @@ int RemoteCameraHttp::GetResponse() { request += stringtf( "User-Agent: %s/%s\r\n", config.http_ua, ZM_VERSION ); request += stringtf( "Host: %s\r\n", host.c_str()); if ( strcmp( config.http_version, "1.0" ) == 0 ) - request += stringtf( "Connection: Keep-Alive\r\n" ); + request += "Connection: Keep-Alive\r\n"; request += mAuthenticator->getAuthHeader( "GET", path.c_str() ); request += "\r\n"; @@ -738,7 +738,7 @@ int RemoteCameraHttp::GetResponse() { request += stringtf("User-Agent: %s/%s\r\n", config.http_ua, ZM_VERSION); request += stringtf("Host: %s\r\n", host.c_str()); if ( strcmp(config.http_version, "1.0") == 0 ) - request += stringtf("Connection: Keep-Alive\r\n"); + request += "Connection: Keep-Alive\r\n"; request += mAuthenticator->getAuthHeader("GET", path.c_str()); request += "\r\n"; diff --git a/src/zm_utils.cpp b/src/zm_utils.cpp index de5748e80..49aedffcc 100644 --- a/src/zm_utils.cpp +++ b/src/zm_utils.cpp @@ -65,34 +65,6 @@ std::string replaceAll(std::string str, std::string from, std::string to) { return str; } -const std::string stringtf( const char *format, ... ) { - va_list ap; - char tempBuffer[8192]; - std::string tempString; - - va_start(ap, format); - vsnprintf(tempBuffer, sizeof(tempBuffer), format , ap); - va_end(ap); - - tempString = tempBuffer; - - return tempString; -} - -const std::string stringtf(const std::string format, ...) { - va_list ap; - char tempBuffer[8192]; - std::string tempString; - - va_start(ap, format); - vsnprintf(tempBuffer, sizeof(tempBuffer), format.c_str(), ap); - va_end(ap); - - tempString = tempBuffer; - - return tempString; -} - bool startsWith(const std::string &haystack, const std::string &needle) { return ( haystack.substr(0, needle.length()) == needle ); } diff --git a/src/zm_utils.h b/src/zm_utils.h index f29bae2e3..ccd73bf2b 100644 --- a/src/zm_utils.h +++ b/src/zm_utils.h @@ -23,8 +23,9 @@ #include #include #include -#include +#include #include +#include #include typedef std::vector StringVector; @@ -33,8 +34,16 @@ std::string trimSpaces(const std::string &str); std::string trimSet(std::string str, std::string trimset); std::string replaceAll(std::string str, std::string from, std::string to); -const std::string stringtf( const char *format, ... ); -const std::string stringtf( const std::string &format, ... ); +template +std::string stringtf(const std::string &format, Args... args) { + int size = snprintf(nullptr, 0, format.c_str(), args...) + 1; // Extra space for '\0' + if (size <= 0) { + throw std::runtime_error("Error during formatting."); + } + std::unique_ptr buf(new char[size]); + snprintf(buf.get(), size, format.c_str(), args...); + return std::string(buf.get(), buf.get() + size - 1); // We don't want the '\0' inside +} bool startsWith( const std::string &haystack, const std::string &needle ); StringVector split( const std::string &string, const std::string &chars, int limit=0 ); From 32f8bc8e31c9c2318b7488f78cc40a9fc04f309a Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Sat, 6 Mar 2021 23:39:59 +0100 Subject: [PATCH 0132/1277] Fifo: Fix a crash on shutdown outfile can be undefined if the fifo couldn't be opened correctly. Only try to close outfile on shutdown when it is valid. --- src/zm_fifo.cpp | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/zm_fifo.cpp b/src/zm_fifo.cpp index 09b35062a..e7ff145af 100644 --- a/src/zm_fifo.cpp +++ b/src/zm_fifo.cpp @@ -94,7 +94,10 @@ bool Fifo::open() { } bool Fifo::close() { - fclose(outfile); + if (outfile) { + fclose(outfile); + } + return true; } From 8ebaee998aa6b1de0123753a0df86b240235fa33 Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Sun, 7 Mar 2021 17:43:48 +0100 Subject: [PATCH 0133/1277] CI/Cirrus: Show Catch2 output on test failure --- .cirrus.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.cirrus.yml b/.cirrus.yml index 98047298e..775bacefb 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -19,4 +19,4 @@ task: test_script: - cd build - - make test + - CTEST_OUTPUT_ON_FAILURE=1 make test From 9660448e5a161233459cab03de61a1498f7e4641 Mon Sep 17 00:00:00 2001 From: Daniel Schetritt Date: Sun, 7 Mar 2021 10:37:21 -0800 Subject: [PATCH 0134/1277] Fix rendering of RST codeblock in documentation This fixes the rendering of a codeblock in reStructuredText. I think the parser couldn't render a codeblock containing a URL using the shorthand `::` so an explicit `.. codeblock::` directive is used instead. --- docs/installationguide/debian.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/installationguide/debian.rst b/docs/installationguide/debian.rst index d09423bd9..2244177e6 100644 --- a/docs/installationguide/debian.rst +++ b/docs/installationguide/debian.rst @@ -60,7 +60,8 @@ Add the following to the /etc/apt/sources.list.d/zoneminder.list file You can do this using: -:: +.. code-block:: + echo "deb https://zmrepo.zoneminder.com/debian/release-1.34 buster/" | sudo tee /etc/apt/sources.list.d/zoneminder.list Because ZoneMinder's package repository provides a secure connection through HTTPS, apt must be enabled for HTTPS. From e38e8a2775102e814746548ad7194567641aab1b Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 8 Mar 2021 09:30:53 -0500 Subject: [PATCH 0135/1277] Actually use zmc_heartbeat_time. Set it on every capture and use it in ShmValid to determine if zmc has gone away. --- src/zm_monitor.cpp | 1 + src/zm_monitor.h | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index eab019ab5..86b359156 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -2505,6 +2505,7 @@ int Monitor::Capture() { packet->timestamp = new struct timeval; packet->image_index = image_count; gettimeofday(packet->timestamp, nullptr); + shared_data->zmc_heartbeat_time = packet->timestamp->tv_sec; Image* capture_image = image_buffer[index].image; int captureResult = 0; diff --git a/src/zm_monitor.h b/src/zm_monitor.h index 29863f998..740a18268 100644 --- a/src/zm_monitor.h +++ b/src/zm_monitor.h @@ -410,7 +410,9 @@ public: bool disconnect(); inline int ShmValid() const { - return shared_data && shared_data->valid; + struct timeval now; + gettimeofday(&now, nullptr); + return shared_data && shared_data->valid && ((now.tv_sec - shared_data->zmc_heartbeat_time) > config.watch_max_delay); } From 46bf765f80039425a46f52b4b0b3b68bac81dc3a Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 8 Mar 2021 18:31:01 -0500 Subject: [PATCH 0136/1277] Set heartbeat on startup. Fix logic in ShmValid --- src/zm_monitor.h | 12 ++++++++---- src/zmc.cpp | 1 + 2 files changed, 9 insertions(+), 4 deletions(-) diff --git a/src/zm_monitor.h b/src/zm_monitor.h index 740a18268..0f1f125d4 100644 --- a/src/zm_monitor.h +++ b/src/zm_monitor.h @@ -410,12 +410,15 @@ public: bool disconnect(); inline int ShmValid() const { - struct timeval now; - gettimeofday(&now, nullptr); - return shared_data && shared_data->valid && ((now.tv_sec - shared_data->zmc_heartbeat_time) > config.watch_max_delay); + if ( shared_data && shared_data->valid ) { + struct timeval now; + gettimeofday(&now, nullptr); + if ((now.tv_sec - shared_data->zmc_heartbeat_time) < config.watch_max_delay) + return true; + } + return false; } - inline unsigned int Id() const { return id; } inline const char *Name() const { return name; } inline unsigned int ServerId() { return server_id; } @@ -520,6 +523,7 @@ public: TriggerState GetTriggerState() const { return trigger_data ? trigger_data->trigger_state : TRIGGER_CANCEL; } inline time_t getStartupTime() const { return shared_data->startup_time; } inline void setStartupTime( time_t p_time ) { shared_data->startup_time = p_time; } + inline void setHeartbeatTime( time_t p_time ) { shared_data->zmc_heartbeat_time = p_time; } void get_ref_image(); int LabelSize() const { return label_size; } diff --git a/src/zmc.cpp b/src/zmc.cpp index 0d3b03145..c4e12c7a0 100644 --- a/src/zmc.cpp +++ b/src/zmc.cpp @@ -245,6 +245,7 @@ int main(int argc, char *argv[]) { } time_t now = (time_t)time(nullptr); monitor->setStartupTime(now); + monitor->setHeartbeatTime(now); snprintf(sql, sizeof(sql), "INSERT INTO Monitor_Status (MonitorId,Status,CaptureFPS,AnalysisFPS)" From 41085c9e5cbd569ce2e2998abf971bc3f2a6ea69 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 8 Mar 2021 22:14:13 -0500 Subject: [PATCH 0137/1277] Remove final bits of zm_video writer. --- src/CMakeLists.txt | 1 - src/zm_monitor.cpp | 2 - src/zm_monitor.h | 3 - src/zm_rtsp_server.cpp | 1 + src/zm_video.cpp | 588 ----------------------------------------- src/zm_video.h | 1 - 6 files changed, 1 insertion(+), 595 deletions(-) delete mode 100644 src/zm_video.cpp diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index d40c76ffd..ea5ab4d30 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -70,7 +70,6 @@ set(ZM_BIN_SRC_FILES zm_time.cpp zm_user.cpp zm_utils.cpp - zm_video.cpp zm_videostore.cpp zm_zone.cpp zm_storage.cpp) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 86b359156..5618314d0 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -534,8 +534,6 @@ void Monitor::Load(MYSQL_ROW dbrow, bool load_zones=true, Purpose p = QUERY) { savejpegs = atoi(dbrow[col]); col++; videowriter = (VideoWriter)atoi(dbrow[col]); col++; encoderparams = dbrow[col] ? dbrow[col] : ""; col++; - /* Parse encoder parameters */ - ParseEncoderParameters(encoderparams.c_str(), &encoderparamsvec); output_codec = dbrow[col] ? atoi(dbrow[col]) : 0; col++; encoder = dbrow[col] ? dbrow[col] : ""; col++; diff --git a/src/zm_monitor.h b/src/zm_monitor.h index 0f1f125d4..20a59d79c 100644 --- a/src/zm_monitor.h +++ b/src/zm_monitor.h @@ -28,7 +28,6 @@ #include "zm_packet.h" #include "zm_packetqueue.h" #include "zm_utils.h" -#include "zm_video.h" #include #include #include @@ -277,7 +276,6 @@ protected: int output_codec; std::string encoder; std::string output_container; - std::vector encoderparamsvec; _AVPIXELFORMAT imagePixFormat; unsigned int subpixelorder; bool record_audio; // Whether to store the audio that we receive @@ -476,7 +474,6 @@ public: int GetOptSaveJPEGs() const { return savejpegs; } VideoWriter GetOptVideoWriter() const { return videowriter; } - //const std::vector* GetEncoderParams() const { return &encoderparamsvec; } const std::string &GetEncoderOptions() const { return encoderparams; } int OutputCodec() const { return output_codec; } const std::string &Encoder() const { return encoder; } diff --git a/src/zm_rtsp_server.cpp b/src/zm_rtsp_server.cpp index 9e4b59152..f8c0e9426 100644 --- a/src/zm_rtsp_server.cpp +++ b/src/zm_rtsp_server.cpp @@ -179,6 +179,7 @@ int main(int argc, char *argv[]) { rtsp_server_thread->removeSession(sessions[i]); sessions[i] = nullptr; } + monitor->disconnect(); continue; } Debug(1, "monitor %d is connected", monitor->Id()); diff --git a/src/zm_video.cpp b/src/zm_video.cpp deleted file mode 100644 index c2a9e161e..000000000 --- a/src/zm_video.cpp +++ /dev/null @@ -1,588 +0,0 @@ -// Copyright (C) 2001-2017 ZoneMinder LLC -// This program is free software; you can redistribute it and/or -// modify it under the terms of the GNU General Public License -// as published by the Free Software Foundation; either version 2 -// of the License, or (at your option) any later version. -// -// This program is distributed in the hope that it will be useful, -// but WITHOUT ANY WARRANTY; without even the implied warranty of -// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -// GNU General Public License for more details. -// -// You should have received a copy of the GNU General Public License -// along with this program; if not, write to the Free Software -// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. -// - -#include "zm_video.h" - -#include -#include -#include - -VideoWriter::VideoWriter( - const char* p_container, - const char* p_codec, - const char* p_path, - const unsigned int p_width, - const unsigned int p_height, - const unsigned int p_colours, - const unsigned int p_subpixelorder) : - container(p_container), - codec(p_codec), - path(p_path), - width(p_width), - height(p_height), - colours(p_colours), - subpixelorder(p_subpixelorder), - frame_count(0) { - Debug(7, "Video object created"); - - /* Parameter checking */ - if ( path.empty() ) { - Error("Invalid file path"); - } - if ( !width || !height ) { - Error("Invalid width or height"); - } -} - -VideoWriter::~VideoWriter() { - Debug(7, "Video object destroyed"); -} - -int VideoWriter::Reset(const char* new_path) { - /* Common variables reset */ - - /* If there is a new path, use it */ - if ( new_path != nullptr ) { - path = new_path; - } - - /* Reset frame counter */ - frame_count = 0; - - return 0; -} - - -#if ZM_HAVE_VIDEOWRITER_X264MP4 -X264MP4Writer::X264MP4Writer( - const char* p_path, - const unsigned int p_width, - const unsigned int p_height, - const unsigned int p_colours, - const unsigned int p_subpixelorder, - const std::vector* p_user_params) : - VideoWriter( - "mp4", - "h264", - p_path, - p_width, - p_height, - p_colours, - p_subpixelorder), - bOpen(false), - bGotH264AVCInfo(false), - bFirstFrame(true) { - /* Initialize ffmpeg if it hasn't been initialized yet */ - FFMPEGInit(); - - /* Initialize swscale */ - zm_pf = GetFFMPEGPixelFormat(colours, subpixelorder); - if ( zm_pf == 0 ) { - Error("Unable to match ffmpeg pixelformat"); - } - codec_pf = AV_PIX_FMT_YUV420P; - - if ( ! swscaleobj.init() ) { - Error("Failed init swscaleobj"); - return; - } - - swscaleobj.SetDefaults(zm_pf, codec_pf, width, height); - - /* Calculate the image sizes. We will need this for parameter checking */ - zm_imgsize = colours * width * height; -#if LIBAVUTIL_VERSION_CHECK(54, 6, 0, 6, 0) - codec_imgsize = av_image_get_buffer_size(codec_pf, width, height, 1); -#else - codec_imgsize = avpicture_get_size(codec_pf, width, height); -#endif - if ( !codec_imgsize ) { - Error("Failed calculating codec pixel format image size"); - } - - /* If supplied with user parameters to the encoder, copy them */ - if ( p_user_params != nullptr ) { - user_params = *p_user_params; - } - - /* Setup x264 parameters */ - if ( x264config() < 0 ) { - Error("Failed setting x264 parameters"); - } - - /* Allocate x264 input picture */ - x264_picture_alloc( - &x264picin, - X264_CSP_I420, - x264params.i_width, - x264params.i_height); -} - -X264MP4Writer::~X264MP4Writer() { - /* Free x264 input picture */ - x264_picture_clean(&x264picin); - - if ( bOpen ) - Close(); -} - -int X264MP4Writer::Open() { - /* Open the encoder */ - x264enc = x264_encoder_open(&x264params); - if ( x264enc == nullptr ) { - Error("Failed opening x264 encoder"); - return -1; - } - - // Debug(4,"x264 maximum delayed frames: %d", - // x264_encoder_maximum_delayed_frames(x264enc)); - - x264_nal_t* nals; - int i_nals; - if ( !x264_encoder_headers(x264enc, &nals, &i_nals) ) { - Error("Failed getting encoder headers"); - return -2; - } - - /* Search SPS NAL for AVC information */ - for ( int i = 0; i < i_nals; i++ ) { - if ( nals[i].i_type == NAL_SPS ) { - x264_profleindication = nals[i].p_payload[5]; - x264_profilecompat = nals[i].p_payload[6]; - x264_levelindication = nals[i].p_payload[7]; - bGotH264AVCInfo = true; - break; - } - } - if ( !bGotH264AVCInfo ) { - Warning("Missing AVC information"); - } - - /* Create the file */ - mp4h = MP4Create((path + ".incomplete").c_str()); - if ( mp4h == MP4_INVALID_FILE_HANDLE ) { - Error("Failed creating mp4 file: %s", path.c_str()); - return -10; - } - - /* Set the global timescale */ - if ( !MP4SetTimeScale(mp4h, 1000) ) { - Error("Failed setting timescale"); - return -11; - } - - /* Set the global video profile */ - /* I am a bit confused about this one. - I couldn't find what the value should be - Some use 0x15 while others use 0x7f. */ - MP4SetVideoProfileLevel(mp4h, 0x7f); - - /* Add H264 video track */ - mp4vtid = MP4AddH264VideoTrack( - mp4h, - 1000, - MP4_INVALID_DURATION, - width, - height, - x264_profleindication, - x264_profilecompat, - x264_levelindication, - 3); - if ( mp4vtid == MP4_INVALID_TRACK_ID ) { - Error("Failed adding H264 video track"); - return -12; - } - - bOpen = true; - - return 0; -} - -int X264MP4Writer::Close() { - /* Flush all pending frames */ - for ( int i = (x264_encoder_delayed_frames(x264enc) + 1); i > 0; i-- ) { - Debug(1, "Encoding delayed frame"); - if ( x264encodeloop(true) < 0 ) - break; - } - - /* Close the encoder */ - x264_encoder_close(x264enc); - - /* Close MP4 handle */ - MP4Close(mp4h); - - Debug(1, "Optimising"); - /* Required for proper HTTP streaming */ - MP4Optimize((path + ".incomplete").c_str(), path.c_str()); - - /* Delete the temporary file */ - unlink((path + ".incomplete").c_str()); - - bOpen = false; - - Debug(1, "Video closed. Total frames: %d", frame_count); - - return 0; -} - -int X264MP4Writer::Reset(const char* new_path) { - /* Close the encoder and file */ - if ( bOpen ) - Close(); - - /* Reset common variables */ - VideoWriter::Reset(new_path); - - /* Reset local variables */ - bFirstFrame = true; - bGotH264AVCInfo = false; - prevnals.clear(); - prevpayload.clear(); - - /* Reset x264 parameters */ - x264config(); - - /* Open the encoder */ - Open(); - - return 0; -} - -int X264MP4Writer::Encode( - const uint8_t* data, - const size_t data_size, - const unsigned int frame_time) { - /* Parameter checking */ - if ( data == nullptr ) { - Error("NULL buffer"); - return -1; - } - - if ( data_size != zm_imgsize ) { - Error("The data buffer size (%d) != expected (%d)", data_size, zm_imgsize); - return -2; - } - - if ( !bOpen ) { - Warning("The encoder was not initialized, initializing now"); - Open(); - } - - /* Convert the image into the x264 input picture */ - if ( swscaleobj.ConvertDefaults(data, data_size, x264picin.img.plane[0], codec_imgsize) < 0 ) { - Error("Image conversion failed"); - return -3; - } - - /* Set PTS */ - x264picin.i_pts = frame_time; - - /* Do the encoding */ - x264encodeloop(); - - /* Increment frame counter */ - frame_count++; - - return 0; -} - -int X264MP4Writer::Encode(const Image* img, const unsigned int frame_time) { - if ( img->Width() != width ) { - Error("Source image width differs. Source: %d Output: %d", img->Width(), width); - return -12; - } - - if ( img->Height() != height ) { - Error("Source image height differs. Source: %d Output: %d", img->Height(), height); - return -13; - } - - return Encode(img->Buffer(), img->Size(), frame_time); -} - -int X264MP4Writer::x264config() { - /* Sets up the encoder configuration */ - - int x264ret; - - /* Defaults */ - const char* preset = "veryfast"; - const char* tune = "stillimage"; - const char* profile = "main"; - - /* Search the user parameters for preset, tune and profile */ - for ( unsigned int i = 0; i < user_params.size(); i++ ) { - if ( strcmp(user_params[i].pname, "preset") == 0 ) { - /* Got preset */ - preset = user_params[i].pvalue; - } else if ( strcmp(user_params[i].pname, "tune") == 0 ) { - /* Got tune */ - tune = user_params[i].pvalue; - } else if ( strcmp(user_params[i].pname, "profile") == 0 ) { - /* Got profile */ - profile = user_params[i].pvalue; - } - } - - /* Set the defaults and preset and tune */ - x264ret = x264_param_default_preset(&x264params, preset, tune); - if ( x264ret != 0 ) { - Error("Failed setting x264 preset %s and tune %s : %d", preset, tune, x264ret); - } - - /* Set the profile */ - x264ret = x264_param_apply_profile(&x264params, profile); - if ( x264ret != 0 ) { - Error("Failed setting x264 profile %s : %d", profile, x264ret); - } - - /* Input format */ - x264params.i_width = width; - x264params.i_height = height; - x264params.i_csp = X264_CSP_I420; - - /* Quality control */ - x264params.rc.i_rc_method = X264_RC_CRF; - x264params.rc.f_rf_constant = 23.0; - - /* Enable b-frames */ - x264params.i_bframe = 16; - x264params.i_bframe_adaptive = 1; - - /* Timebase */ - x264params.i_timebase_num = 1; - x264params.i_timebase_den = 1000; - - /* Enable variable frame rate */ - x264params.b_vfr_input = 1; - - /* Disable annex-b (start codes) */ - x264params.b_annexb = 0; - - /* TODO: Setup error handler */ - if ( logDebugging() ) - x264params.i_log_level = X264_LOG_DEBUG; - else - x264params.i_log_level = X264_LOG_NONE; - - /* Process user parameters (excluding preset, tune and profile) */ - for ( unsigned int i = 0; i < user_params.size(); i++ ) { - /* Skip preset, tune and profile */ - if ( - (strcmp(user_params[i].pname, "preset") == 0) || - (strcmp(user_params[i].pname, "tune") == 0) || - (strcmp(user_params[i].pname, "profile") == 0) ) { - continue; - } - - /* Pass the name and value to x264 */ - x264ret = x264_param_parse(&x264params, user_params[i].pname, user_params[i].pvalue); - - /* Error checking */ - if ( x264ret != 0 ) { - if ( x264ret == X264_PARAM_BAD_NAME ) { - Error("Failed processing x264 user parameter %s=%s : Bad name", - user_params[i].pname, user_params[i].pvalue); - } else if ( x264ret == X264_PARAM_BAD_VALUE ) { - Error("Failed processing x264 user parameter %s=%s : Bad value", - user_params[i].pname, user_params[i].pvalue); - } else { - Error("Failed processing x264 user parameter %s=%s : Unknown error (%d)", - user_params[i].pname, user_params[i].pvalue, x264ret); - } - } - } - - return 0; -} - -int X264MP4Writer::x264encodeloop(bool bFlush) { - x264_nal_t* nals; - int i_nals; - int frame_size; - - if ( bFlush ) { - frame_size = x264_encoder_encode(x264enc, &nals, &i_nals, nullptr, &x264picout); - } else { - frame_size = x264_encoder_encode(x264enc, &nals, &i_nals, &x264picin, &x264picout); - } - - if ( frame_size > 0 || bFlush ) { - Debug(1, "x264 Frame: %d PTS: %d DTS: %d Size: %d\n", - frame_count, x264picout.i_pts, x264picout.i_dts, frame_size); - - /* Handle the previous frame */ - if ( !bFirstFrame ) { - buffer.clear(); - - /* Process the NALs for the previous frame */ - for ( unsigned int i = 0; i < prevnals.size(); i++ ) { - Debug(9, "Processing NAL: Type %d Size %d", - prevnals[i].i_type, - prevnals[i].i_payload); - - switch ( prevnals[i].i_type ) { - case NAL_PPS: - /* PPS NAL */ - MP4AddH264PictureParameterSet(mp4h, mp4vtid, prevnals[i].p_payload+4, prevnals[i].i_payload-4); - break; - case NAL_SPS: - /* SPS NAL */ - MP4AddH264SequenceParameterSet(mp4h, mp4vtid, prevnals[i].p_payload+4, prevnals[i].i_payload-4); - break; - default: - /* Anything else, hopefully frames, so copy it into the sample */ - buffer.append(prevnals[i].p_payload, prevnals[i].i_payload); - } - } - - /* Calculate frame duration and offset */ - int duration = x264picout.i_dts - prevDTS; - int offset = prevPTS - prevDTS; - - /* Write the sample */ - if ( !buffer.empty() ) { - unsigned int bufSize = buffer.size(); - if ( !MP4WriteSample( - mp4h, - mp4vtid, - buffer.extract(bufSize), - bufSize, - duration, - offset, - prevKeyframe) ) { - Error("Failed writing sample"); - } - } - - /* Cleanup */ - prevnals.clear(); - prevpayload.clear(); - } - - /* Got a frame. Copy this new frame into the previous frame */ - if ( frame_size > 0 ) { - /* Copy the NALs and the payloads */ - for ( int i = 0; i < i_nals; i++ ) { - prevnals.push_back(nals[i]); - prevpayload.append(nals[i].p_payload, nals[i].i_payload); - } - - /* Update the payload pointers */ - /* This is done in a separate loop because the previous loop might reallocate memory when appending, - making the pointers invalid */ - unsigned int payload_offset = 0; - for ( unsigned int i = 0; i < prevnals.size(); i++ ) { - prevnals[i].p_payload = prevpayload.head() + payload_offset; - payload_offset += nals[i].i_payload; - } - - /* We need this for the next frame */ - prevPTS = x264picout.i_pts; - prevDTS = x264picout.i_dts; - prevKeyframe = x264picout.b_keyframe; - - bFirstFrame = false; - } - - } else if ( frame_size == 0 ) { - Debug(1, "x264 encode returned zero. Delayed frames: %d", - x264_encoder_delayed_frames(x264enc)); - } else { - Error("x264 encode failed: %d", frame_size); - } - return frame_size; -} -#endif // ZM_VIDEOWRITER_X264MP4 - -int ParseEncoderParameters( - const char* str, - std::vector* vec - ) { - if ( vec == nullptr ) { - Error("NULL Encoder parameters vector pointer"); - return -1; - } - - if ( str == nullptr ) { - Error("NULL Encoder parameters string"); - return -2; - } - - vec->clear(); - - if ( str[0] == 0 ) { - /* Empty */ - return 0; - } - - std::string line; - std::stringstream ss(str); - size_t valueoffset; - size_t valuelen; - unsigned int lineno = 0; - EncoderParameter_t param; - - while ( std::getline(ss, line) ) { - lineno++; - - /* Remove CR if exists */ - if ( line.length() >= 1 && line[line.length()-1] == '\r' ) { - line.erase(line.length() - 1); - } - - /* Skip comments and empty lines */ - if ( line.empty() || line[0] == '#' ) { - continue; - } - - valueoffset = line.find('='); - if ( valueoffset == std::string::npos || (valueoffset+1 >= line.length()) || (valueoffset == 0) ) { - Warning("Failed parsing encoder parameters line %d %s: Invalid pair", lineno, line.c_str()); - continue; - } - - if ( valueoffset > (sizeof(param.pname) - 1 ) ) { - Warning("Failed parsing encoder parameters line %d: Name too long", lineno); - continue; - } - - valuelen = line.length() - (valueoffset+1); - - if ( valuelen > (sizeof(param.pvalue) - 1 ) ) { - Warning("Failed parsing encoder parameters line %d: Value too long", lineno); - continue; - } - - /* Copy and NULL terminate */ - line.copy(param.pname, valueoffset, 0); - line.copy(param.pvalue, valuelen, valueoffset+1); - param.pname[valueoffset] = 0; - param.pvalue[valuelen] = 0; - - /* Push to the vector */ - vec->push_back(param); - - Debug(7, "Parsed encoder parameter: %s = %s", param.pname, param.pvalue); - } - - Debug(7, "Parsed %d lines", lineno); - - return 0; -} - - diff --git a/src/zm_video.h b/src/zm_video.h index 89e5e33af..1b8087bc0 100644 --- a/src/zm_video.h +++ b/src/zm_video.h @@ -51,7 +51,6 @@ extern "C" { struct EncoderParameter_t { char pname[48]; char pvalue[48]; - }; int ParseEncoderParameters(const char* str, std::vector* vec); From 831cf8af5614f21b74b490e018ab38c2d3c7a0de Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 8 Mar 2021 22:14:47 -0500 Subject: [PATCH 0138/1277] remove zm_video.h which is no longer relevant --- src/zm_video.h | 170 ------------------------------------------------- 1 file changed, 170 deletions(-) delete mode 100644 src/zm_video.h diff --git a/src/zm_video.h b/src/zm_video.h deleted file mode 100644 index 1b8087bc0..000000000 --- a/src/zm_video.h +++ /dev/null @@ -1,170 +0,0 @@ -// This program is free software; you can redistribute it and/or -// modify it under the terms of the GNU General Public License -// as published by the Free Software Foundation; either version 2 -// of the License, or (at your option) any later version. -// -// This program is distributed in the hope that it will be useful, -// but WITHOUT ANY WARRANTY; without even the implied warranty of -// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -// GNU General Public License for more details. -// -// You should have received a copy of the GNU General Public License -// along with this program; if not, write to the Free Software -// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. -// - -#ifndef ZM_VIDEO_H -#define ZM_VIDEO_H - -#include "zm_buffer.h" -#include "zm_config.h" -#include "zm_swscale.h" - -/* -#define HAVE_LIBX264 1 -#define HAVE_LIBMP4V2 1 -#define HAVE_X264_H 1 -#define HAVE_MP4_H 1 -*/ - -#if HAVE_MP4V2_MP4V2_H -#include -#endif -#if HAVE_MP4V2_H -#include -#endif -#if HAVE_MP4_H -#include -#endif - -#if HAVE_X264_H -#ifdef __cplusplus -extern "C" { -#endif -#include -#ifdef __cplusplus -} -#endif -#endif - -/* Structure for user parameters to the encoder */ -struct EncoderParameter_t { - char pname[48]; - char pvalue[48]; -}; -int ParseEncoderParameters(const char* str, std::vector* vec); - -/* VideoWriter is a generic interface that ZM uses to save events as videos */ -/* It is relatively simple and the functions are pure virtual, so they must be implemented by the deriving class */ - -class VideoWriter { - -protected: - std::string container; - std::string codec; - std::string path; - unsigned int width; - unsigned int height; - unsigned int colours; - unsigned int subpixelorder; - - unsigned int frame_count; - -public: - VideoWriter(const char* p_container, const char* p_codec, const char* p_path, const unsigned int p_width, const unsigned int p_height, const unsigned int p_colours, const unsigned int p_subpixelorder); - virtual ~VideoWriter(); - virtual int Encode(const uint8_t* data, const size_t data_size, const unsigned int frame_time) = 0; - virtual int Encode(const Image* img, const unsigned int frame_time) = 0; - virtual int Open() = 0; - virtual int Close() = 0; - virtual int Reset(const char* new_path = nullptr); - - const char* GetContainer() const { - return container.c_str(); - } - const char* GetCodec() const { - return codec.c_str(); - } - const char* GetPath() const { - return path.c_str(); - } - unsigned int GetWidth() const { - return width; - } - unsigned int GetHeight() const { - return height; - } - unsigned int GetColours() const { - return colours; - } - unsigned int GetSubpixelorder () const { - return subpixelorder; - } - unsigned int GetFrameCount() const { - return frame_count; - } -}; - -#if HAVE_LIBX264 && HAVE_LIBMP4V2 && HAVE_LIBAVUTIL && HAVE_LIBSWSCALE -#define ZM_HAVE_VIDEOWRITER_X264MP4 1 -class X264MP4Writer : public VideoWriter { - -protected: - - bool bOpen; - bool bGotH264AVCInfo; - bool bFirstFrame; - - /* SWScale */ - SWScale swscaleobj; - enum _AVPIXELFORMAT zm_pf; - enum _AVPIXELFORMAT codec_pf; - size_t codec_imgsize; - size_t zm_imgsize; - - /* User parameters */ - std::vector user_params; - - /* AVC Information */ - uint8_t x264_profleindication; - uint8_t x264_profilecompat; - uint8_t x264_levelindication; - - /* NALs */ - Buffer buffer; - - /* Previous frame */ - int prevPTS; - int prevDTS; - bool prevKeyframe; - Buffer prevpayload; - std::vector prevnals; - - /* Internal functions */ - int x264config(); - int x264encodeloop(bool bFlush = false); - - /* x264 objects */ - x264_t* x264enc; - x264_param_t x264params; - x264_picture_t x264picin; - x264_picture_t x264picout; - - /* MP4v2 objects */ - MP4FileHandle mp4h; - MP4TrackId mp4vtid; - - -public: - X264MP4Writer(const char* p_path, const unsigned int p_width, const unsigned int p_height, const unsigned int p_colours, const unsigned int p_subpixelorder, const std::vector* p_user_params = nullptr); - ~X264MP4Writer(); - int Encode(const uint8_t* data, const size_t data_size, const unsigned int frame_time); - int Encode(const Image* img, const unsigned int frame_time); - int Open(); - int Close(); - int Reset(const char* new_path = nullptr); - -}; -#endif // HAVE_LIBX264 && HAVE_LIBMP4V2 && HAVE_LIBAVUTIL && HAVE_LIBSWSCALE - -#endif // ZM_VIDEO_H From f4cb4ec5b3d9859d166eabc808cb1a7abdb33f64 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 9 Mar 2021 09:10:56 -0500 Subject: [PATCH 0139/1277] Remove mp4v2-dev dependency --- .github/workflows/codeql-analysis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml index aed08288d..8f1541031 100644 --- a/.github/workflows/codeql-analysis.yml +++ b/.github/workflows/codeql-analysis.yml @@ -52,7 +52,7 @@ jobs: git submodule init git submodule update --init --recursive sudo apt-get update - sudo apt-get install libx264-dev libmp4v2-dev libavdevice-dev libavcodec-dev libavformat-dev libavutil-dev libswresample-dev libswscale-dev + sudo apt-get install libavdevice-dev libavcodec-dev libavformat-dev libavutil-dev libswresample-dev libswscale-dev sudo apt-get install libbz2-dev libgcrypt20-dev libcurl4-gnutls-dev libjpeg-turbo8-dev libturbojpeg0-dev sudo apt-get install default-libmysqlclient-dev libpcre3-dev libpolkit-gobject-1-dev libv4l-dev libvlc-dev sudo apt-get install libdate-manip-perl libdbd-mysql-perl libphp-serialization-perl libsys-mmap-perl From b1f6eb127b957df2dd544310e38c3ee73ea3352d Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 10 Mar 2021 11:01:04 -0500 Subject: [PATCH 0140/1277] Switch from live555 to PHZ76/RtspServer --- src/CMakeLists.txt | 11 +- src/zm_rtsp_server.cpp | 110 +++++++-- ...pp => zm_rtsp_server_fifo_adts_source.cpp} | 19 +- ...ce.h => zm_rtsp_server_fifo_adts_source.h} | 19 +- src/zm_rtsp_server_fifo_audio_source.cpp | 24 +- src/zm_rtsp_server_fifo_audio_source.h | 22 +- ...pp => zm_rtsp_server_fifo_h264_source.cpp} | 44 ++-- ...ce.h => zm_rtsp_server_fifo_h264_source.h} | 64 ++---- src/zm_rtsp_server_fifo_source.cpp | 135 ++--------- src/zm_rtsp_server_fifo_source.h | 57 ++--- src/zm_rtsp_server_fifo_video_source.cpp | 21 +- src/zm_rtsp_server_fifo_video_source.h | 11 +- src/zm_rtsp_server_thread.cpp | 210 ++++-------------- src/zm_rtsp_server_thread.h | 28 +-- 14 files changed, 301 insertions(+), 474 deletions(-) rename src/{zm_rtsp_server_adts_fifo_source.cpp => zm_rtsp_server_fifo_adts_source.cpp} (78%) rename src/{zm_rtsp_server_adts_fifo_source.h => zm_rtsp_server_fifo_adts_source.h} (71%) rename src/{zm_rtsp_server_h264_fifo_source.cpp => zm_rtsp_server_fifo_h264_source.cpp} (90%) rename src/{zm_rtsp_server_h264_fifo_source.h => zm_rtsp_server_fifo_h264_source.h} (56%) diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index ea5ab4d30..c6a9c04c7 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -52,17 +52,11 @@ set(ZM_BIN_SRC_FILES zm_rtp_source.cpp zm_rtsp.cpp zm_rtsp_auth.cpp - zm_rtsp_server_thread.cpp - zm_rtsp_server_adts_source.cpp - zm_rtsp_server_adts_fifo_source.cpp - zm_rtsp_server_h264_device_source.cpp - zm_rtsp_server_h264_fifo_source.cpp - zm_rtsp_server_device_source.cpp zm_rtsp_server_fifo_source.cpp + zm_rtsp_server_fifo_adts_source.cpp + zm_rtsp_server_fifo_h264_source.cpp zm_rtsp_server_fifo_audio_source.cpp zm_rtsp_server_fifo_video_source.cpp - zm_rtsp_server_server_media_subsession.cpp - zm_rtsp_server_unicast_server_media_subsession.cpp zm_sdp.cpp zm_signal.cpp zm_stream.cpp @@ -85,6 +79,7 @@ target_link_libraries(zm PUBLIC libbcrypt::bcrypt jwt-cpp::jwt-cpp + RtspServer::RtspServer PRIVATE zm-core-interface) diff --git a/src/zm_rtsp_server.cpp b/src/zm_rtsp_server.cpp index f8c0e9426..e97ea5677 100644 --- a/src/zm_rtsp_server.cpp +++ b/src/zm_rtsp_server.cpp @@ -51,15 +51,16 @@ and provide that stream over rtsp #include "zm_db.h" #include "zm_define.h" #include "zm_monitor.h" -#include "zm_rtsp_server_thread.h" -#include "zm_rtsp_server_fifo_audio_source.h" -#include "zm_rtsp_server_fifo_video_source.h" +#include "zm_rtsp_server_fifo_h264_source.h" +#include "zm_rtsp_server_fifo_adts_source.h" #include "zm_signal.h" #include "zm_time.h" #include "zm_utils.h" #include #include -#include + + +#include "xop/RtspServer.h" void Usage() { fprintf(stderr, "zm_rtsp_server -m \n"); @@ -157,17 +158,23 @@ int main(int argc, char *argv[]) { sigaddset(&block_set, SIGUSR1); sigaddset(&block_set, SIGUSR2); - std::unique_ptr rtsp_server_thread; - if (config.min_rtsp_port) { - rtsp_server_thread = ZM::make_unique(config.min_rtsp_port); - Debug(1, "Starting RTSP server because min_rtsp_port is set"); - } else { + if (!config.min_rtsp_port) { Debug(1, "Not starting RTSP server because min_rtsp_port not set"); exit(-1); } - ServerMediaSession **sessions = new ServerMediaSession *[monitors.size()]; + + std::shared_ptr eventLoop(new xop::EventLoop()); + std::shared_ptr rtspServer = xop::RtspServer::Create(eventLoop.get()); + if ( !rtspServer->Start("0.0.0.0", config.min_rtsp_port) ) { + Debug(1, "Failed starting RTSP server on port %d", config.min_rtsp_port); + exit(-1); + } + + xop::MediaSession **sessions = new xop::MediaSession *[monitors.size()]; for (size_t i = 0; i < monitors.size(); i++) sessions[i] = nullptr; + std::list sources; + while (!zm_terminate) { for (size_t i = 0; i < monitors.size(); i++) { @@ -176,13 +183,12 @@ int main(int argc, char *argv[]) { if (!(monitor->ShmValid() or monitor->connect())) { Warning("Couldn't connect to monitor %d", monitor->Id()); if (sessions[i]) { - rtsp_server_thread->removeSession(sessions[i]); + rtspServer->RemoveSession(sessions[i]->GetMediaSessionId()); sessions[i] = nullptr; } monitor->disconnect(); continue; } - Debug(1, "monitor %d is connected", monitor->Id()); if (!sessions[i]) { std::string videoFifoPath = monitor->GetVideoFifoPath(); @@ -191,14 +197,49 @@ int main(int argc, char *argv[]) { continue; } std::string streamname = monitor->GetRTSPStreamName(); - Debug(1, "Adding session for %s", streamname.c_str()); - ServerMediaSession *sms = sessions[i] = rtsp_server_thread->addSession(streamname); + + xop::MediaSession *session = sessions[i] = xop::MediaSession::CreateNew(streamname); + if (session) { + session->AddNotifyConnectedCallback([] (xop::MediaSessionId sessionId, std::string peer_ip, uint16_t peer_port){ + Debug(1, "RTSP client connect, ip=%s, port=%hu \n", peer_ip.c_str(), peer_port); + }); + + session->AddNotifyDisconnectedCallback([](xop::MediaSessionId sessionId, std::string peer_ip, uint16_t peer_port) { + Debug(1, "RTSP client disconnect, ip=%s, port=%hu \n", peer_ip.c_str(), peer_port); + }); + + rtspServer->AddSession(session); + //char *url = rtspServer->rtspURL(session); + //Debug(1, "url is %s for stream %s", url, streamname.c_str()); + //delete[] url; + } + Debug(1, "Adding video fifo %s", videoFifoPath.c_str()); - ZoneMinderFifoVideoSource *video_source = static_cast(rtsp_server_thread->addFifo(sms, videoFifoPath)); + ZoneMinderFifoVideoSource *videoSource = nullptr; + + if (std::string::npos != videoFifoPath.find("h264")) { + session->AddSource(xop::channel_0, xop::H264Source::CreateNew()); + videoSource = new H264_ZoneMinderFifoSource(rtspServer, session->GetMediaSessionId(), xop::channel_0, videoFifoPath); + } else if ( + std::string::npos != videoFifoPath.find("hevc") + or + std::string::npos != videoFifoPath.find("h265")) { + session->AddSource(xop::channel_0, xop::H265Source::CreateNew()); + videoSource = new H265_ZoneMinderFifoSource(rtspServer, session->GetMediaSessionId(), xop::channel_0, videoFifoPath); + } else { + Warning("Unknown format in %s", videoFifoPath.c_str()); + } + if (videoSource == nullptr) { + Error("Unable to create source"); + } + sources.push_back(videoSource); + +#if 0 if (video_source) { video_source->setWidth(monitor->Width()); video_source->setHeight(monitor->Height()); } +#endif std::string audioFifoPath = monitor->GetAudioFifoPath(); if (audioFifoPath.empty()) { @@ -206,13 +247,28 @@ int main(int argc, char *argv[]) { continue; } Debug(1, "Adding audio fifo %s", audioFifoPath.c_str()); - ZoneMinderFifoAudioSource *audio_source = static_cast(rtsp_server_thread->addFifo(sms, audioFifoPath)); - if (audio_source) { - audio_source->setFrequency(monitor->GetAudioFrequency()); - audio_source->setChannels(monitor->GetAudioChannels()); + + ZoneMinderFifoAudioSource *audioSource = nullptr; + + if (std::string::npos != audioFifoPath.find("aac")) { + Debug(1, "Adding aac source at %dHz %d channels", monitor->GetAudioFrequency(), monitor->GetAudioChannels()); + session->AddSource(xop::channel_1, xop::AACSource::CreateNew( + monitor->GetAudioFrequency(), + monitor->GetAudioChannels(), + false /* has_adts */)); + audioSource = new ADTS_ZoneMinderFifoSource(rtspServer, session->GetMediaSessionId(), xop::channel_1, audioFifoPath); + audioSource->setFrequency(monitor->GetAudioFrequency()); + audioSource->setChannels(monitor->GetAudioChannels()); + } else { + Warning("Unknown format in %s", audioFifoPath.c_str()); } + if (audioSource == nullptr) { + Error("Unable to create source"); + } + sources.push_back(audioSource); } // end if ! sessions[i] } // end foreach monitor + sleep(1); if (zm_reload) { @@ -229,11 +285,23 @@ int main(int argc, char *argv[]) { for (size_t i = 0; i < monitors.size(); i++) { if (sessions[i]) { Debug(1, "Removing session for %s", monitors[i]->Name()); - rtsp_server_thread->removeSession(sessions[i]); + rtspServer->RemoveSession(sessions[i]->GetMediaSessionId()); sessions[i] = nullptr; } } // end foreach monitor - rtsp_server_thread->Stop(); + + for ( std::list::iterator it = sources.begin(); it != sources.end(); ++it ) { + Debug(1, "RTSPServerThread::stopping source"); + (*it)->Stop(); + } + while (sources.size()) { + Debug(1, "RTSPServerThread::stop closing source"); + ZoneMinderFifoSource *source = sources.front(); + sources.pop_front(); + delete source; + } + + rtspServer->Stop(); delete[] sessions; sessions = nullptr; diff --git a/src/zm_rtsp_server_adts_fifo_source.cpp b/src/zm_rtsp_server_fifo_adts_source.cpp similarity index 78% rename from src/zm_rtsp_server_adts_fifo_source.cpp rename to src/zm_rtsp_server_fifo_adts_source.cpp index d62b04df1..6c01371e5 100644 --- a/src/zm_rtsp_server_adts_fifo_source.cpp +++ b/src/zm_rtsp_server_fifo_adts_source.cpp @@ -7,32 +7,27 @@ ** -------------------------------------------------------------------------*/ #include "zm_logger.h" -#include "zm_rtsp_server_adts_fifo_source.h" +#include "zm_rtsp_server_fifo_adts_source.h" #include #include #if HAVE_RTSP_SERVER -static unsigned const samplingFrequencyTable[16] = { - 96000, 88200, 64000, 48000, - 44100, 32000, 24000, 22050, - 16000, 12000, 11025, 8000, - 7350, 0, 0, 0 -}; // --------------------------------- // ADTS ZoneMinder FramedSource // --------------------------------- // ADTS_ZoneMinderFifoSource::ADTS_ZoneMinderFifoSource( - UsageEnvironment& env, - std::string fifo, - unsigned int queueSize + std::shared_ptr& rtspServer, + xop::MediaSessionId sessionId, + xop::MediaChannelId channelId, + std::string fifo ) : - ZoneMinderFifoAudioSource(env, fifo, queueSize) + ZoneMinderFifoAudioSource(rtspServer, sessionId, channelId, fifo) { -#if 1 +#if 0 int profile = 0; unsigned char audioSpecificConfig[2]; diff --git a/src/zm_rtsp_server_adts_fifo_source.h b/src/zm_rtsp_server_fifo_adts_source.h similarity index 71% rename from src/zm_rtsp_server_adts_fifo_source.h rename to src/zm_rtsp_server_fifo_adts_source.h index df4abd417..0bbe0890c 100644 --- a/src/zm_rtsp_server_adts_fifo_source.h +++ b/src/zm_rtsp_server_fifo_adts_source.h @@ -21,19 +21,12 @@ // --------------------------------- class ADTS_ZoneMinderFifoSource : public ZoneMinderFifoAudioSource { - public: - static ADTS_ZoneMinderFifoSource* createNew( - UsageEnvironment& env, - std::string fifo, - unsigned int queueSize - ) { - return new ADTS_ZoneMinderFifoSource(env, fifo, queueSize); - }; - protected: - ADTS_ZoneMinderFifoSource( - UsageEnvironment& env, - std::string fifo, - unsigned int queueSize + public: + ADTS_ZoneMinderFifoSource( + std::shared_ptr& rtspServer, + xop::MediaSessionId sessionId, + xop::MediaChannelId channelId, + std::string fifo ); virtual ~ADTS_ZoneMinderFifoSource() {} diff --git a/src/zm_rtsp_server_fifo_audio_source.cpp b/src/zm_rtsp_server_fifo_audio_source.cpp index 25e10f7f0..f28c5951f 100644 --- a/src/zm_rtsp_server_fifo_audio_source.cpp +++ b/src/zm_rtsp_server_fifo_audio_source.cpp @@ -10,7 +10,7 @@ #if HAVE_RTSP_SERVER -static unsigned const samplingFrequencyTable[16] = { +static const int samplingFrequencyTable[16] = { 96000, 88200, 64000, 48000, 44100, 32000, 24000, 22050, 16000, 12000, 11025, 8000, @@ -21,12 +21,13 @@ static unsigned const samplingFrequencyTable[16] = { // --------------------------------- // ZoneMinderFifoAudioSource::ZoneMinderFifoAudioSource( - UsageEnvironment& env, - std::string fifo, - unsigned int queueSize + std::shared_ptr& rtspServer, + xop::MediaSessionId sessionId, + xop::MediaChannelId channelId, + std::string fifo ) : - ZoneMinderFifoSource(env, fifo, queueSize), + ZoneMinderFifoSource(rtspServer, sessionId, channelId, fifo), samplingFrequencyIndex(-1), frequency(-1), channels(1) @@ -37,4 +38,17 @@ int ZoneMinderFifoAudioSource::getFrequencyIndex() { if (samplingFrequencyTable[i] == frequency) return i; return -1; } + +void ZoneMinderFifoAudioSource::PushFrame(const uint8_t *data, size_t size, int64_t pts) { + + Debug(1, "Pushing audio frame to session %d channel %d pts %" PRId64, m_sessionId, m_channelId, pts); + xop::AVFrame frame = {0}; + frame.type = xop::AUDIO_FRAME; + frame.size = size; + frame.timestamp = av_rescale_q(pts, AV_TIME_BASE_Q, m_timeBase); + frame.buffer.reset(new uint8_t[size]); + memcpy(frame.buffer.get(), data, size); + m_rtspServer->PushFrame(m_sessionId, m_channelId, frame); + +} #endif // HAVE_RTSP_SERVER diff --git a/src/zm_rtsp_server_fifo_audio_source.h b/src/zm_rtsp_server_fifo_audio_source.h index 691652f8f..83cb8c709 100644 --- a/src/zm_rtsp_server_fifo_audio_source.h +++ b/src/zm_rtsp_server_fifo_audio_source.h @@ -22,26 +22,19 @@ class ZoneMinderFifoAudioSource : public ZoneMinderFifoSource { public: - static ZoneMinderFifoAudioSource* createNew( - UsageEnvironment& env, - std::string fifo, - unsigned int queueSize - ) { - return new ZoneMinderFifoAudioSource(env, fifo, queueSize); - }; - protected: ZoneMinderFifoAudioSource( - UsageEnvironment& env, - std::string fifo, - unsigned int queueSize + std::shared_ptr& rtspServer, + xop::MediaSessionId sessionId, + xop::MediaChannelId channelId, + std::string fifo ); virtual ~ZoneMinderFifoAudioSource() {} - public: void setFrequency(int p_frequency) { frequency = p_frequency; samplingFrequencyIndex = getFrequencyIndex(); + m_timeBase = {1, frequency}; }; int getFrequency() { return frequency; }; int getFrequencyIndex(); @@ -49,10 +42,13 @@ class ZoneMinderFifoAudioSource : public ZoneMinderFifoSource { void setChannels(int p_channels) { channels = p_channels; }; int getChannels() const { return channels; }; + protected: + void PushFrame(const uint8_t *data, size_t size, int64_t pts); + protected: std::string config; int samplingFrequencyIndex; - unsigned int frequency; + int frequency; int channels; }; #endif // HAVE_RTSP_SERVER diff --git a/src/zm_rtsp_server_h264_fifo_source.cpp b/src/zm_rtsp_server_fifo_h264_source.cpp similarity index 90% rename from src/zm_rtsp_server_h264_fifo_source.cpp rename to src/zm_rtsp_server_fifo_h264_source.cpp index 9f72eea02..6d8a5393e 100644 --- a/src/zm_rtsp_server_h264_fifo_source.cpp +++ b/src/zm_rtsp_server_fifo_h264_source.cpp @@ -6,11 +6,10 @@ ** ** -------------------------------------------------------------------------*/ -#include "zm_rtsp_server_h264_fifo_source.h" +#include "zm_rtsp_server_fifo_h264_source.h" #include "zm_config.h" #include "zm_logger.h" -#include "zm_rtsp_server_frame.h" #include #include @@ -23,26 +22,27 @@ // --------------------------------- // H264_ZoneMinderFifoSource::H264_ZoneMinderFifoSource( - UsageEnvironment& env, - std::string fifo, - unsigned int queueSize, - bool repeatConfig, - bool keepMarker) - : H26X_ZoneMinderFifoSource(env, fifo, queueSize, repeatConfig, keepMarker) + std::shared_ptr& rtspServer, + xop::MediaSessionId sessionId, + xop::MediaChannelId channelId, + std::string fifo + ) + : H26X_ZoneMinderFifoSource(rtspServer, sessionId, channelId, fifo) { // extradata appears to simply be the SPS and PPS NAL's //this->splitFrames(m_stream->codecpar->extradata, m_stream->codecpar->extradata_size); } // split packet into frames -std::list< std::pair > H264_ZoneMinderFifoSource::splitFrames(unsigned char* frame, unsigned &frameSize) { +std::list< std::pair > H264_ZoneMinderFifoSource::splitFrames(unsigned char* frame, size_t &frameSize) { std::list< std::pair > frameList; size_t bufSize = frameSize; size_t size = 0; unsigned char* buffer = this->extractFrame(frame, bufSize, size); - bool updateAux = false; while ( buffer != nullptr ) { +#if 0 + bool updateAux = false; switch ( m_frameType & 0x1F ) { case 7: Debug(4, "SPS_Size: %d bufSize %d", size, bufSize); @@ -83,6 +83,7 @@ std::list< std::pair > H264_ZoneMinderFifoSource::splitF delete [] sps_base64; delete [] pps_base64; } +#endif frameList.push_back(std::pair(buffer, size)); buffer = this->extractFrame(&buffer[size], bufSize, size); @@ -92,12 +93,12 @@ std::list< std::pair > H264_ZoneMinderFifoSource::splitF } H265_ZoneMinderFifoSource::H265_ZoneMinderFifoSource( - UsageEnvironment& env, - std::string fifo, - unsigned int queueSize, - bool repeatConfig, - bool keepMarker) - : H26X_ZoneMinderFifoSource(env, fifo, queueSize, repeatConfig, keepMarker) + std::shared_ptr& rtspServer, + xop::MediaSessionId sessionId, + xop::MediaChannelId channelId, + std::string fifo + ) + : H26X_ZoneMinderFifoSource(rtspServer, sessionId, channelId, fifo) { // extradata appears to simply be the SPS and PPS NAL's // this->splitFrames(m_stream->codecpar->extradata, m_stream->codecpar->extradata_size); @@ -105,14 +106,15 @@ H265_ZoneMinderFifoSource::H265_ZoneMinderFifoSource( // split packet in frames std::list< std::pair > -H265_ZoneMinderFifoSource::splitFrames(unsigned char* frame, unsigned &frameSize) { - std::list< std::pair > frameList; +H265_ZoneMinderFifoSource::splitFrames(unsigned char* frame, size_t &frameSize) { + std::list< std::pair > frameList; size_t bufSize = frameSize; size_t size = 0; unsigned char* buffer = this->extractFrame(frame, bufSize, size); - bool updateAux = false; while ( buffer != nullptr ) { +#if 0 + bool updateAux = false; switch ((m_frameType&0x7E)>>1) { case 32: Debug(4, "VPS_Size: %d bufSize %d", size, bufSize); @@ -160,13 +162,11 @@ H265_ZoneMinderFifoSource::splitFrames(unsigned char* frame, unsigned &frameSize delete [] sps_base64; delete [] pps_base64; } +#endif frameList.push_back(std::pair(buffer, size)); buffer = this->extractFrame(&buffer[size], bufSize, size); } // end while buffer - if ( bufSize ) { - Debug(1, "%d bytes remaining", bufSize); - } frameSize = bufSize; return frameList; } // end H265_ZoneMinderFifoSource::splitFrames(unsigned char* frame, unsigned frameSize) diff --git a/src/zm_rtsp_server_h264_fifo_source.h b/src/zm_rtsp_server_fifo_h264_source.h similarity index 56% rename from src/zm_rtsp_server_h264_fifo_source.h rename to src/zm_rtsp_server_fifo_h264_source.h index 27ca2ef71..d73aa1730 100644 --- a/src/zm_rtsp_server_h264_fifo_source.h +++ b/src/zm_rtsp_server_fifo_h264_source.h @@ -19,18 +19,19 @@ // H264 ZoneMinder FramedSource // --------------------------------- #if HAVE_RTSP_SERVER +const char H264marker[] = {0,0,0,1}; +const char H264shortmarker[] = {0,0,1}; class H26X_ZoneMinderFifoSource : public ZoneMinderFifoVideoSource { - protected: + public: H26X_ZoneMinderFifoSource( - UsageEnvironment& env, - std::string fifo, - unsigned int queueSize, - bool repeatConfig, - bool keepMarker) + std::shared_ptr& rtspServer, + xop::MediaSessionId sessionId, + xop::MediaChannelId channelId, + std::string fifo + ) : - ZoneMinderFifoVideoSource(env, fifo, queueSize), - m_repeatConfig(repeatConfig), - m_keepMarker(keepMarker), + ZoneMinderFifoVideoSource(rtspServer, sessionId, channelId, fifo), + m_keepMarker(false), m_frameType(0) { } virtual ~H26X_ZoneMinderFifoSource() {} @@ -41,55 +42,34 @@ class H26X_ZoneMinderFifoSource : public ZoneMinderFifoVideoSource { protected: std::string m_sps; std::string m_pps; - bool m_repeatConfig; bool m_keepMarker; int m_frameType; }; class H264_ZoneMinderFifoSource : public H26X_ZoneMinderFifoSource { public: - static H264_ZoneMinderFifoSource* createNew( - UsageEnvironment& env, - std::string fifo, - unsigned int queueSize, - bool repeatConfig, - bool keepMarker) { - return new H264_ZoneMinderFifoSource(env, fifo, queueSize, repeatConfig, keepMarker); - } - - protected: H264_ZoneMinderFifoSource( - UsageEnvironment& env, - std::string fifo, - unsigned int queueSize, - bool repeatConfig, - bool keepMarker); + std::shared_ptr& rtspServer, + xop::MediaSessionId sessionId, + xop::MediaChannelId channelId, + std::string fifo + ); // overide ZoneMinderFifoSource - virtual std::list< std::pair > splitFrames(unsigned char* frame, unsigned &frameSize); + virtual std::list< std::pair > splitFrames(unsigned char* frame, size_t &frameSize); }; class H265_ZoneMinderFifoSource : public H26X_ZoneMinderFifoSource { public: - static H265_ZoneMinderFifoSource* createNew( - UsageEnvironment& env, - std::string fifo, - unsigned int queueSize, - bool repeatConfig, - bool keepMarker) { - return new H265_ZoneMinderFifoSource(env, fifo, queueSize, repeatConfig, keepMarker); - } - - protected: H265_ZoneMinderFifoSource( - UsageEnvironment& env, - std::string fifo, - unsigned int queueSize, - bool repeatConfig, - bool keepMarker); + std::shared_ptr& rtspServer, + xop::MediaSessionId sessionId, + xop::MediaChannelId channelId, + std::string fifo + ); // overide ZoneMinderFifoSource - virtual std::list< std::pair > splitFrames(unsigned char* frame, unsigned &frameSize); + virtual std::list< std::pair > splitFrames(unsigned char* frame, size_t &frameSize); protected: std::string m_vps; diff --git a/src/zm_rtsp_server_fifo_source.cpp b/src/zm_rtsp_server_fifo_source.cpp index 81bd42f30..cc3d443b1 100644 --- a/src/zm_rtsp_server_fifo_source.cpp +++ b/src/zm_rtsp_server_fifo_source.cpp @@ -11,8 +11,8 @@ #include "zm_rtsp_server_fifo_source.h" #include "zm_config.h" +#include "zm_ffmpeg.h" #include "zm_logger.h" -#include "zm_rtsp_server_frame.h" #include "zm_signal.h" #include @@ -20,16 +20,17 @@ #if HAVE_RTSP_SERVER ZoneMinderFifoSource::ZoneMinderFifoSource( - UsageEnvironment& env, - std::string fifo, - unsigned int queueSize + std::shared_ptr& rtspServer, + xop::MediaSessionId sessionId, + xop::MediaChannelId channelId, + std::string fifo ) : - FramedSource(env), + m_rtspServer(rtspServer), + m_sessionId(sessionId), + m_channelId(channelId), m_fifo(fifo), - m_queueSize(queueSize), m_fd(-1) { - m_eventTriggerId = envir().taskScheduler().createEventTrigger(ZoneMinderFifoSource::deliverFrameStub); memset(&m_thid, 0, sizeof(m_thid)); memset(&m_mutex, 0, sizeof(m_mutex)); pthread_mutex_init(&m_mutex, nullptr); @@ -39,13 +40,7 @@ ZoneMinderFifoSource::ZoneMinderFifoSource( ZoneMinderFifoSource::~ZoneMinderFifoSource() { Debug(1, "Deleting Fifo Source"); stop = 1; - envir().taskScheduler().deleteEventTrigger(m_eventTriggerId); pthread_join(m_thid, nullptr); - while (m_captureQueue.size()) { - NAL_Frame * f = m_captureQueue.front(); - m_captureQueue.pop_front(); - delete f; - } Debug(1, "Deleting Fifo Source done"); pthread_mutex_destroy(&m_mutex); } @@ -55,78 +50,11 @@ void* ZoneMinderFifoSource::thread() { stop = 0; while (!stop) { - if ( getNextFrame() < 0 ) - sleep(1); + if (getNextFrame() < 0) sleep(1); } return nullptr; } -// getting FrameSource callback -void ZoneMinderFifoSource::doGetNextFrame() { - deliverFrame(); -} - -// stopping FrameSource callback -void ZoneMinderFifoSource::doStopGettingFrames() { - //stop = 1; - Debug(1, "ZoneMinderFifoSource::doStopGettingFrames"); - FramedSource::doStopGettingFrames(); -} - -// deliver frame to the sink -void ZoneMinderFifoSource::deliverFrame() { - if (!isCurrentlyAwaitingData()) { - Debug(5, "not awaiting data"); - return; - } - - pthread_mutex_lock(&m_mutex); - if (m_captureQueue.empty()) { - Debug(5, "Queue is empty"); - pthread_mutex_unlock(&m_mutex); - return; - } - - NAL_Frame *frame = m_captureQueue.front(); - m_captureQueue.pop_front(); - pthread_mutex_unlock(&m_mutex); - - fDurationInMicroseconds = 0; - fFrameSize = 0; - - unsigned int nal_size = frame->size(); - - if (nal_size > fMaxSize) { - fFrameSize = fMaxSize; - fNumTruncatedBytes = nal_size - fMaxSize; - } else { - fFrameSize = nal_size; - } - - Debug(4, "deliverFrame timestamp: %ld.%06ld size: %d queuesize: %d", - frame->m_timestamp.tv_sec, frame->m_timestamp.tv_usec, - fFrameSize, - m_captureQueue.size() - ); - - fPresentationTime = frame->m_timestamp; - memcpy(fTo, frame->buffer(), fFrameSize); - - if (fFrameSize > 0) { - // send Frame to the consumer - FramedSource::afterGetting(this); - } - delete frame; -} // end void ZoneMinderFifoSource::deliverFrame() - -// FrameSource callback on read event -void ZoneMinderFifoSource::incomingPacketHandler() { - Debug(1, "incomingPacketHandler"); - if (this->getNextFrame() <= 0) { - handleClosure(this); - } -} - // read from monitor int ZoneMinderFifoSource::getNextFrame() { if (zm_terminate or stop) { @@ -221,53 +149,28 @@ int ZoneMinderFifoSource::getNextFrame() { } } unsigned char *packet_start = m_buffer.head() + header_size; - - // splitFrames modifies so make a copy - unsigned int bytes_remaining = data_size; + size_t bytes_remaining = data_size; std::list< std::pair > framesList = this->splitFrames(packet_start, bytes_remaining); - Debug(3, "Got %d frames, consuming %d bytes", framesList.size(), header_size + data_size); + Debug(3, "Got %d frames, consuming %d bytes, remaining %d", framesList.size(), header_size + data_size, bytes_remaining); m_buffer.consume(header_size + data_size); - - timeval tv; - tv.tv_sec = pts / 1000000; - tv.tv_usec = pts % 1000000; - while (framesList.size()) { std::pair nal = framesList.front(); framesList.pop_front(); - NAL_Frame *frame = new NAL_Frame(nal.first, nal.second, tv); - Debug(3, "Got frame, size %d, queue_size %d", frame->size(), m_captureQueue.size()); - - pthread_mutex_lock(&m_mutex); - if (m_captureQueue.size() > 25) { // 1 sec at 25 fps - NAL_Frame * f = m_captureQueue.front(); - while (m_captureQueue.size() and ((tv.tv_sec - f->m_timestamp.tv_sec) > 2)) { - m_captureQueue.pop_front(); - delete f; - f = m_captureQueue.front(); - } - Debug(3, "Done clearing. Queue size is now %d", m_captureQueue.size()); - } - m_captureQueue.push_back(frame); - pthread_mutex_unlock(&m_mutex); - - // post an event to ask to deliver the frame - envir().taskScheduler().triggerEvent(m_eventTriggerId, this); - } // end while we get frame from data + PushFrame(nal.first, nal.second, pts); + } } // end while m_buffer.size() return 1; } // split packet in frames -std::list< std::pair > ZoneMinderFifoSource::splitFrames(unsigned char* frame, unsigned &frameSize) { - std::list< std::pair > frameList; - if (frame != nullptr) { - frameList.push_back(std::pair(frame, frameSize)); - } - // We consume it all +std::list< std::pair > ZoneMinderFifoSource::splitFrames(unsigned char* frame, size_t &frameSize) { + std::list< std::pair > frameList; + if ( frame != nullptr ) { + frameList.push_back(std::pair(frame, frameSize)); + } frameSize = 0; - return frameList; + return frameList; } // extract a frame diff --git a/src/zm_rtsp_server_fifo_source.h b/src/zm_rtsp_server_fifo_source.h index 990d32984..e1c60cfdf 100644 --- a/src/zm_rtsp_server_fifo_source.h +++ b/src/zm_rtsp_server_fifo_source.h @@ -11,70 +11,53 @@ #include "zm_buffer.h" #include "zm_config.h" +#include "zm_ffmpeg.h" #include "zm_define.h" #include #include #include #if HAVE_RTSP_SERVER -#include +#include "xop/RtspServer.h" -class NAL_Frame; - -class ZoneMinderFifoSource: public FramedSource { +class ZoneMinderFifoSource { public: - static ZoneMinderFifoSource* createNew( - UsageEnvironment& env, - std::string fifo, - unsigned int queueSize - ) { - return new ZoneMinderFifoSource(env, fifo, queueSize); - }; - std::string getAuxLine() { return m_auxLine; }; - int getWidth() { return m_width; }; - int getHeight() { return m_height; }; - int setWidth(int width) { return m_width=width; }; - int setHeight(int height) { return m_height=height; }; - protected: - ZoneMinderFifoSource(UsageEnvironment& env, std::string fifo, unsigned int queueSize); + void Stop() { stop=1; }; + + ZoneMinderFifoSource( + std::shared_ptr& rtspServer, + xop::MediaSessionId sessionId, + xop::MediaChannelId channelId, + std::string fifo + ); virtual ~ZoneMinderFifoSource(); protected: static void* threadStub(void* clientData) { return ((ZoneMinderFifoSource*) clientData)->thread();}; void* thread(); - static void deliverFrameStub(void* clientData) {((ZoneMinderFifoSource*) clientData)->deliverFrame();}; - void deliverFrame(); - static void incomingPacketHandlerStub(void* clientData, int mask) { ((ZoneMinderFifoSource*) clientData)->incomingPacketHandler(); }; - void incomingPacketHandler(); int getNextFrame(); - void processFrame(char * frame, int frameSize, const timeval &ref); - void queueFrame(char * frame, int frameSize, const timeval &tv); - - // split packet in frames - virtual std::list< std::pair > splitFrames(unsigned char* frame, unsigned &frameSize); - - // overide FramedSource - virtual void doGetNextFrame(); - virtual void doStopGettingFrames(); + virtual void PushFrame(const uint8_t *data, size_t size, int64_t pts) = 0; + // split packet in frames + virtual std::list< std::pair > splitFrames(unsigned char* frame, size_t &frameSize); virtual unsigned char *extractFrame(unsigned char *data, size_t& size, size_t& outsize); - + protected: - std::list m_captureQueue; - EventTriggerId m_eventTriggerId; - std::string m_fifo; int m_width; int m_height; - unsigned int m_queueSize; pthread_t m_thid; pthread_mutex_t m_mutex; - std::string m_auxLine; int stop; + std::shared_ptr& m_rtspServer; + xop::MediaSessionId m_sessionId; + xop::MediaChannelId m_channelId; + std::string m_fifo; int m_fd; Buffer m_buffer; + AVRational m_timeBase; }; #endif // HAVE_RTSP_SERVER diff --git a/src/zm_rtsp_server_fifo_video_source.cpp b/src/zm_rtsp_server_fifo_video_source.cpp index bcf1f8841..3fb21f78d 100644 --- a/src/zm_rtsp_server_fifo_video_source.cpp +++ b/src/zm_rtsp_server_fifo_video_source.cpp @@ -12,9 +12,26 @@ #if HAVE_RTSP_SERVER ZoneMinderFifoVideoSource::ZoneMinderFifoVideoSource( - UsageEnvironment& env, std::string fifo, unsigned int queueSize) : - ZoneMinderFifoSource(env,fifo,queueSize) + std::shared_ptr& rtspServer, + xop::MediaSessionId sessionId, + xop::MediaChannelId channelId, + std::string fifo + ) : + ZoneMinderFifoSource(rtspServer, sessionId, channelId, fifo) { + m_timeBase = {1, 90000}; +} + +void ZoneMinderFifoVideoSource::PushFrame(const uint8_t *data, size_t size, int64_t pts) { + + xop::AVFrame frame = {0}; + frame.type = 0; + frame.size = size; + frame.timestamp = av_rescale_q(pts, AV_TIME_BASE_Q, m_timeBase); + frame.buffer.reset(new uint8_t[size]); + memcpy(frame.buffer.get(), data, size); + m_rtspServer->PushFrame(m_sessionId, m_channelId, frame); + } #endif // HAVE_RTSP_SERVER diff --git a/src/zm_rtsp_server_fifo_video_source.h b/src/zm_rtsp_server_fifo_video_source.h index 444d0ca04..b06dbe5a7 100644 --- a/src/zm_rtsp_server_fifo_video_source.h +++ b/src/zm_rtsp_server_fifo_video_source.h @@ -21,9 +21,16 @@ class ZoneMinderFifoVideoSource: public ZoneMinderFifoSource { int setWidth(int width) { return m_width=width; }; int setHeight(int height) { return m_height=height; }; - protected: - ZoneMinderFifoVideoSource(UsageEnvironment& env, std::string fifo, unsigned int queueSize); + ZoneMinderFifoVideoSource( + std::shared_ptr& rtspServer, + xop::MediaSessionId sessionId, + xop::MediaChannelId channelId, + std::string fifo + ); + protected: + void PushFrame(const uint8_t *data, size_t size, int64_t pts); + protected: int m_width; int m_height; }; diff --git a/src/zm_rtsp_server_thread.cpp b/src/zm_rtsp_server_thread.cpp index 7f989f987..cbe7b905b 100644 --- a/src/zm_rtsp_server_thread.cpp +++ b/src/zm_rtsp_server_thread.cpp @@ -2,38 +2,22 @@ #include "zm_config.h" #include "zm_logger.h" -#include "zm_rtsp_server_adts_fifo_source.h" -#include "zm_rtsp_server_adts_source.h" -#include "zm_rtsp_server_h264_fifo_source.h" -#include "zm_rtsp_server_h264_device_source.h" -#include "zm_rtsp_server_unicast_server_media_subsession.h" #if HAVE_RTSP_SERVER -#include -RTSPServerThread::RTSPServerThread(int port) : - terminate_(false), scheduler_watch_var_(0) +RTSPServerThread::RTSPServerThread(int p_port) : + terminate_(false), scheduler_watch_var_(0), port(p_port) { //unsigned short rtsp_over_http_port = 0; //const char *realm = "ZoneMinder"; - //unsigned int timeout = 65; - OutPacketBuffer::maxSize = 2000000; - - scheduler = BasicTaskScheduler::createNew(); - env = BasicUsageEnvironment::createNew(*scheduler); - authDB = nullptr; - //authDB = new UserAuthenticationDatabase("ZoneMinder"); - //authDB->addUserRecord("username1", "password1"); // replace these with real strings - - portNumBits rtspServerPortNum = port; - rtspServer = RTSPServer::createNew(*env, rtspServerPortNum, authDB); + // + eventLoop = std::make_shared(); + rtspServer = xop::RtspServer::Create(eventLoop.get()); if ( rtspServer == nullptr ) { - Fatal("Failed to create rtspServer at port %d", rtspServerPortNum); + Fatal("Failed to create rtspServer"); return; } - const char *prefix = rtspServer->rtspURLPrefix(); - delete[] prefix; thread_ = std::thread(&RTSPServerThread::Run, this); } @@ -43,25 +27,24 @@ RTSPServerThread::~RTSPServerThread() { if (thread_.joinable()) thread_.join(); - if (rtspServer) { - Medium::close(rtspServer); - } // end if rtsp_server - while ( sources.size() ) { - FramedSource *source = sources.front(); - sources.pop_front(); - Medium::close(source); - } - env->reclaim(); - delete scheduler; } void RTSPServerThread::Run() { Debug(1, "RTSPServerThread::Run()"); - if (rtspServer) - env->taskScheduler().doEventLoop(&scheduler_watch_var_); // does not return + if (rtspServer) { + while (!scheduler_watch_var_) { + //if (clients > 0) { + sleep(1); + //} + } + } Debug(1, "RTSPServerThread::done()"); } +int RTSPServerThread::Start() { + return rtspServer->Start(std::string("0.0.0.0"), port); +} + void RTSPServerThread::Stop() { Debug(1, "RTSPServerThread::stop()"); terminate_ = true; @@ -71,174 +54,75 @@ void RTSPServerThread::Stop() { scheduler_watch_var_ = 1; } - for ( std::list::iterator it = sources.begin(); it != sources.end(); ++it ) { + for ( std::list::iterator it = sources.begin(); it != sources.end(); ++it ) { Debug(1, "RTSPServerThread::stopping source"); - (*it)->stopGettingFrames(); + (*it)->Stop(); } while ( sources.size() ) { Debug(1, "RTSPServerThread::stop closing source"); - FramedSource *source = sources.front(); + ZoneMinderFifoSource *source = sources.front(); sources.pop_front(); - Medium::close(source); + delete source; } } -ServerMediaSession *RTSPServerThread::addSession(std::string &streamname) { - ServerMediaSession *sms = ServerMediaSession::createNew(*env, streamname.c_str()); - if (sms) { - rtspServer->addServerMediaSession(sms); - char *url = rtspServer->rtspURL(sms); - Debug(1, "url is %s for stream %s", url, streamname.c_str()); - delete[] url; +xop::MediaSession *RTSPServerThread::addSession(std::string &streamname) { + + xop::MediaSession *session = xop::MediaSession::CreateNew(streamname); + if (session) { + session->AddNotifyConnectedCallback([] (xop::MediaSessionId sessionId, std::string peer_ip, uint16_t peer_port){ + Debug(1, "RTSP client connect, ip=%s, port=%hu \n", peer_ip.c_str(), peer_port); +}); + + session->AddNotifyDisconnectedCallback([](xop::MediaSessionId sessionId, std::string peer_ip, uint16_t peer_port) { + Debug(1, "RTSP client disconnect, ip=%s, port=%hu \n", peer_ip.c_str(), peer_port); +}); + + rtspServer->AddSession(session); + //char *url = rtspServer->rtspURL(session); + //Debug(1, "url is %s for stream %s", url, streamname.c_str()); + //delete[] url; } - return sms; + return session; } -void RTSPServerThread::removeSession(ServerMediaSession *sms) { - rtspServer->removeServerMediaSession(sms); +void RTSPServerThread::removeSession(xop::MediaSession *session) { + //rtspServer->removeServerMediaSession(session); } -FramedSource *RTSPServerThread::addFifo( - ServerMediaSession *sms, +ZoneMinderFifoSource *RTSPServerThread::addFifo( + xop::MediaSession *session, std::string fifo) { if (!rtspServer) return nullptr; - int queueSize = 60; - bool repeatConfig = false; - bool muxTS = false; - FramedSource *source = nullptr; + ZoneMinderFifoSource *source = nullptr; if (!fifo.empty()) { - StreamReplicator* replicator = nullptr; std::string rtpFormat; if (std::string::npos != fifo.find("h264")) { rtpFormat = "video/H264"; - source = H264_ZoneMinderFifoSource::createNew(*env, fifo, queueSize, repeatConfig, muxTS); + session->AddSource(xop::channel_0, xop::H264Source::CreateNew()); + source = new ZoneMinderFifoSource(rtspServer, session->GetMediaSessionId(), xop::channel_0, fifo); } else if ( std::string::npos != fifo.find("hevc") or std::string::npos != fifo.find("h265")) { rtpFormat = "video/H265"; - source = H265_ZoneMinderFifoSource::createNew(*env, fifo, queueSize, repeatConfig, muxTS); + session->AddSource(xop::channel_0, xop::H265Source::CreateNew()); + source = new ZoneMinderFifoSource(rtspServer, session->GetMediaSessionId(), xop::channel_0, fifo); } else if (std::string::npos != fifo.find("aac")) { - rtpFormat = "audio/AAC"; - source = ADTS_ZoneMinderFifoSource::createNew(*env, fifo, queueSize); Debug(1, "ADTS source %p", source); } else { Warning("Unknown format in %s", fifo.c_str()); } if (source == nullptr) { Error("Unable to create source"); - } else { - replicator = StreamReplicator::createNew(*env, source, false); } sources.push_back(source); - - if (replicator) { - sms->addSubsession(UnicastServerMediaSubsession::createNew(*env, replicator, rtpFormat)); - } } else { Debug(1, "Not Adding stream as fifo was empty"); } return source; } // end void addFifo -void RTSPServerThread::addStream(std::string &streamname, AVStream *video_stream, AVStream *audio_stream) { - if ( !rtspServer ) - return; - - int queueSize = 30; - bool repeatConfig = true; - bool muxTS = false; - ServerMediaSession *sms = nullptr; - - if ( video_stream ) { - StreamReplicator* videoReplicator = nullptr; - FramedSource *source = nullptr; - std::string rtpFormat = getRtpFormat( -#if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) - video_stream->codecpar->codec_id -#else - video_stream->codec->codec_id -#endif - , false); - if ( rtpFormat.empty() ) { - Error("No streaming format"); - return; - } - Debug(1, "RTSP: format %s", rtpFormat.c_str()); - if ( rtpFormat == "video/H264" ) { - source = H264_ZoneMinderDeviceSource::createNew(*env, monitor_, video_stream, queueSize, repeatConfig, muxTS); - } else if ( rtpFormat == "video/H265" ) { - source = H265_ZoneMinderDeviceSource::createNew(*env, monitor_, video_stream, queueSize, repeatConfig, muxTS); - } - if ( source == nullptr ) { - Error("Unable to create source"); - } else { - videoReplicator = StreamReplicator::createNew(*env, source, false); - } - sources.push_back(source); - - // Create Unicast Session - if ( videoReplicator ) { - if ( !sms ) - sms = ServerMediaSession::createNew(*env, streamname.c_str()); - sms->addSubsession(UnicastServerMediaSubsession::createNew(*env, videoReplicator, rtpFormat)); - } - } - if ( audio_stream ) { - StreamReplicator* replicator = nullptr; - FramedSource *source = nullptr; - std::string rtpFormat = getRtpFormat( -#if LIBAVCODEC_VERSION_CHECK(57, 64, 0, 64, 0) - audio_stream->codecpar->codec_id -#else - audio_stream->codec->codec_id -#endif - , false); - if ( rtpFormat == "audio/AAC" ) { - source = ADTS_ZoneMinderDeviceSource::createNew(*env, monitor_, audio_stream, queueSize); - Debug(1, "ADTS source %p", source); - } - if ( source ) { - replicator = StreamReplicator::createNew(*env, source, false /* deleteWhenLastReplicaDies */); - sources.push_back(source); - } - if ( replicator ) { - if ( !sms ) - sms = ServerMediaSession::createNew(*env, streamname.c_str()); - sms->addSubsession(UnicastServerMediaSubsession::createNew(*env, replicator, rtpFormat)); - } - } else { - Debug(1, "Not Adding audio stream"); - } - if ( sms ) { - rtspServer->addServerMediaSession(sms); - char *url = rtspServer->rtspURL(sms); - Debug(1, "url is %s", url); - delete[] url; - } -} // end void addStream - -// ----------------------------------------- -// convert V4L2 pix format to RTP mime -// ----------------------------------------- -const std::string RTSPServerThread::getRtpFormat(AVCodecID codec_id, bool muxTS) { - if ( muxTS ) { - return "video/MP2T"; - } else { - switch ( codec_id ) { - case AV_CODEC_ID_H265 : return "video/H265"; - case AV_CODEC_ID_H264 : return "video/H264"; - //case PIX_FMT_MJPEG: rtpFormat = "video/JPEG"; break; - //case PIX_FMT_JPEG : rtpFormat = "video/JPEG"; break; - //case AV_PIX_FMT_VP8 : rtpFormat = "video/VP8" ; break; - //case AV_PIX_FMT_VP9 : rtpFormat = "video/VP9" ; break; - case AV_CODEC_ID_AAC : return "audio/AAC"; - default: break; - } - } - - return ""; -} #endif // HAVE_RTSP_SERVER diff --git a/src/zm_rtsp_server_thread.h b/src/zm_rtsp_server_thread.h index 9d02ec2a7..393203831 100644 --- a/src/zm_rtsp_server_thread.h +++ b/src/zm_rtsp_server_thread.h @@ -3,15 +3,14 @@ #include "zm_config.h" #include "zm_ffmpeg.h" -#include "zm_rtsp_server_server_media_subsession.h" +#include "xop/RtspServer.h" + #include "zm_rtsp_server_fifo_source.h" #include #include #include #if HAVE_RTSP_SERVER -#include -#include class Monitor; @@ -24,29 +23,22 @@ class RTSPServerThread { std::mutex scheduler_watch_var_mutex_; char scheduler_watch_var_; - TaskScheduler* scheduler; - UsageEnvironment* env; - UserAuthenticationDatabase* authDB; + std::shared_ptr eventLoop; + std::shared_ptr rtspServer; - RTSPServer* rtspServer; - std::list sources; + std::list sources; + int port; public: explicit RTSPServerThread(int port); ~RTSPServerThread(); - ServerMediaSession *addSession(std::string &streamname); - void removeSession(ServerMediaSession *sms); - void addStream(std::string &streamname, AVStream *, AVStream *); - FramedSource *addFifo(ServerMediaSession *sms, std::string fifo); + xop::MediaSession *addSession(std::string &streamname); + void removeSession(xop::MediaSession *sms); + ZoneMinderFifoSource *addFifo(xop::MediaSession *sms, std::string fifo); void Run(); void Stop(); + int Start(); bool IsStopped() const { return terminate_; }; - private: - const std::string getRtpFormat(AVCodecID codec, bool muxTS); - int addSession( - const std::string & sessionName, - const std::list & subSession - ); }; #endif // HAVE_RTSP_SERVER From 24fb020686c8774964771420ffc338e6c446fac0 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 10 Mar 2021 11:01:52 -0500 Subject: [PATCH 0141/1277] Add PHZ76 in deps. Must be added as a submodule --- dep/CMakeLists.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/dep/CMakeLists.txt b/dep/CMakeLists.txt index 588f5d558..bb197570c 100644 --- a/dep/CMakeLists.txt +++ b/dep/CMakeLists.txt @@ -1,2 +1,3 @@ add_subdirectory(jwt-cpp) add_subdirectory(libbcrypt) +add_subdirectory(RtspServer) From a0957fbd10f13b82cb3c9610f7dfc87c73437782 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 10 Mar 2021 13:19:30 -0500 Subject: [PATCH 0142/1277] remove debug --- src/zm_rtsp_server_fifo_audio_source.cpp | 3 --- 1 file changed, 3 deletions(-) diff --git a/src/zm_rtsp_server_fifo_audio_source.cpp b/src/zm_rtsp_server_fifo_audio_source.cpp index f28c5951f..d0008600a 100644 --- a/src/zm_rtsp_server_fifo_audio_source.cpp +++ b/src/zm_rtsp_server_fifo_audio_source.cpp @@ -40,8 +40,6 @@ int ZoneMinderFifoAudioSource::getFrequencyIndex() { } void ZoneMinderFifoAudioSource::PushFrame(const uint8_t *data, size_t size, int64_t pts) { - - Debug(1, "Pushing audio frame to session %d channel %d pts %" PRId64, m_sessionId, m_channelId, pts); xop::AVFrame frame = {0}; frame.type = xop::AUDIO_FRAME; frame.size = size; @@ -49,6 +47,5 @@ void ZoneMinderFifoAudioSource::PushFrame(const uint8_t *data, size_t size, int6 frame.buffer.reset(new uint8_t[size]); memcpy(frame.buffer.get(), data, size); m_rtspServer->PushFrame(m_sessionId, m_channelId, frame); - } #endif // HAVE_RTSP_SERVER From 562e7c7e8024bbf1226a6eff4a5e3f639bb247b4 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 10 Mar 2021 13:19:41 -0500 Subject: [PATCH 0143/1277] Add authentication --- src/zm_rtsp_server.cpp | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/src/zm_rtsp_server.cpp b/src/zm_rtsp_server.cpp index e97ea5677..0fb5a8d15 100644 --- a/src/zm_rtsp_server.cpp +++ b/src/zm_rtsp_server.cpp @@ -51,6 +51,7 @@ and provide that stream over rtsp #include "zm_db.h" #include "zm_define.h" #include "zm_monitor.h" +#include "zm_rtsp_server_authenticator.h" #include "zm_rtsp_server_fifo_h264_source.h" #include "zm_rtsp_server_fifo_adts_source.h" #include "zm_signal.h" @@ -165,7 +166,13 @@ int main(int argc, char *argv[]) { std::shared_ptr eventLoop(new xop::EventLoop()); std::shared_ptr rtspServer = xop::RtspServer::Create(eventLoop.get()); - if ( !rtspServer->Start("0.0.0.0", config.min_rtsp_port) ) { + + if (config.opt_use_auth) { + std::shared_ptr authenticator(new ZM_RtspServer_Authenticator()); + rtspServer->SetAuthenticator(authenticator); + } + + if (!rtspServer->Start("0.0.0.0", config.min_rtsp_port)) { Debug(1, "Failed starting RTSP server on port %d", config.min_rtsp_port); exit(-1); } From aadee4ea65bc93404f8e40b50d0b34f377c83e8f Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 10 Mar 2021 13:20:14 -0500 Subject: [PATCH 0144/1277] add a Query String parse in zm_utils add an authenticator to use with RtspServer --- src/zm_rtsp_server_authenticator.h | 84 ++++++++++++++++++++++++++++++ src/zm_utils.cpp | 62 ++++++++++++++++++++++ src/zm_utils.h | 36 +++++++++++++ 3 files changed, 182 insertions(+) create mode 100644 src/zm_rtsp_server_authenticator.h diff --git a/src/zm_rtsp_server_authenticator.h b/src/zm_rtsp_server_authenticator.h new file mode 100644 index 000000000..5bbc0b38b --- /dev/null +++ b/src/zm_rtsp_server_authenticator.h @@ -0,0 +1,84 @@ +#ifndef ZM_RTSP_SERVER_D_H +#define ZM_RTSP_SERVER_AUTHENTICATOR_H + +#include "zm_config.h" +#include "zm_user.h" + +#include +#include + +#include "xop/Authenticator.h" +#include "xop/RtspMessage.h" + +#if HAVE_RTSP_SERVER + +class ZM_RtspServer_Authenticator : public xop::Authenticator { + public: + ZM_RtspServer_Authenticator() {}; + ~ZM_RtspServer_Authenticator() {}; + + virtual bool Authenticate(std::shared_ptr request, std::string &nonce) { + + if (!config.opt_use_auth) { + Debug(1, "Not doing auth"); + return true; + } + std::string url = request->GetRtspUrl(); + Debug(1, "Doing auth %s", url.c_str()); + + User *user = nullptr; + + size_t found = url.find("?"); + if (found == std::string::npos) return false; + + std::string queryString = url.substr(found+1, std::string::npos); + +#if 0 + found = suffix_string.find("/"); + if ( found != std::string::npos ) + suffix_string = suffix_string.substr(0, found); +#endif + + Debug(1, "suffix %s", queryString.c_str()); + std::istringstream requestStream(queryString); + QueryString query(requestStream); + + if (query.has("jwt_token")) { + const QueryParameter *jwt_token = query.get("jwt_token"); + user = zmLoadTokenUser(jwt_token->firstValue().c_str(), false); + } else if (strcmp(config.auth_relay, "none") == 0) { + if (query.has("username")) { + std::string username = query.get("username")->firstValue(); + if (checkUser(username.c_str())) { + user = zmLoadUser(username.c_str()); + } else { + Debug(1, "Bad username %s", username.c_str()); + } + } + } else { + if (query.has("auth")) { + std::string auth_hash = query.get("auth")->firstValue(); + if ( !auth_hash.empty() ) + user = zmLoadAuthUser(auth_hash.c_str(), config.auth_hash_ips); + } + Debug(1, "Query has username ? %d", query.has("username")); + if ((!user) and query.has("username") and query.has("password")) { + std::string username = query.get("username")->firstValue(); + std::string password = query.get("password")->firstValue(); + Debug(1, "username %s password %s", username.c_str(), password.c_str()); + user = zmLoadUser(username.c_str(), password.c_str()); + } + } // end if query string + + if (user) { + Debug(1, "Authenticated"); + delete user; + return true; + } + return false; + } +}; // end class ZM_RtspServer_Authenticator + +#endif // HAVE_RTSP_SERVER + +#endif // ZM_RTSP_SERVER_AUTHENTICATOR_H diff --git a/src/zm_utils.cpp b/src/zm_utils.cpp index 49aedffcc..905500b45 100644 --- a/src/zm_utils.cpp +++ b/src/zm_utils.cpp @@ -396,3 +396,65 @@ void touch(const char *pathname) { } } +QueryString::QueryString(std::istream &input) { + + while (!input.eof() && input.peek() > 0) { + //Should eat "param1=" + auto name = parseName(input); + //Should eat value1& + std::string value = parseValue(input); + + auto foundItr = parameters_.find(name); + if (foundItr == parameters_.end()) { + auto newParam = std::make_unique(name); + if (value.size() > 0) { + newParam->addValue(value); + } + parameters_.emplace(name, std::move(newParam)).first; + } else { + foundItr->second->addValue(value); + } + } // end while not the end +} + +std::vector QueryString::names() const { + std::vector names; + for (auto const& pair : parameters_) + names.push_back(pair.second->name()); + + return names; +} + +const QueryParameter *QueryString::get(const std::string &name) const { + auto itr = parameters_.find(name); + return itr == parameters_.end() ? nullptr : itr->second.get(); +}; + +std::string QueryString::parseName(std::istream &input) { + std::string name = ""; + + while (!input.eof() && input.peek()!= '=') + name.push_back(input.get()); + + //Eat the '=' + if (!input.eof()) input.get(); + + Debug(1, "namee: %s", name.c_str()); + return name; +} + +std::string QueryString::parseValue(std::istream &input) { + std::string urlEncodedValue; + + int c = input.get(); + while (c > 0 && c != '&') { + urlEncodedValue.push_back(c); + c = input.get(); + } + + Debug(1, "value: %s", urlEncodedValue.c_str()); + if (urlEncodedValue.size() == 0) + return ""; + + return UriDecode(urlEncodedValue); +} diff --git a/src/zm_utils.h b/src/zm_utils.h index ccd73bf2b..4eba32b6a 100644 --- a/src/zm_utils.h +++ b/src/zm_utils.h @@ -22,6 +22,7 @@ #include #include +#include #include #include #include @@ -94,4 +95,39 @@ typedef std::chrono::hours Hours; typedef std::chrono::steady_clock::time_point TimePoint; typedef std::chrono::system_clock::time_point SystemTimePoint; +class QueryParameter { + public: + const std::string &name() const { return name_; } + const std::string &firstValue() const { return values_[0]; } + + const std::vector &values() const { return values_; } + size_t size() const { return values_.size(); } + + QueryParameter(std::string name) : name_(std::move(name)) { } + + template void addValue(T&& value) { values_.emplace_back(std::forward(value)); } + private: + std::string name_; + std::vector values_; +}; + +class QueryString { + public: + QueryString(std::istream &input); + + size_t size() const { return parameters_.size(); } + bool has(const char *name) const { return parameters_.find(std::string(name)) != parameters_.end(); } + + std::vector names() const; + + const QueryParameter *get(const std::string &name) const; + const QueryParameter *get(const char* name) const { return get(std::string(name)); }; + + private: + + static std::string parseName(std::istream &input); + static std::string parseValue(std::istream &input); + + std::map> parameters_; +}; #endif // ZM_UTILS_H From 42972f9ab7b4d14a86ef8ee540a53b7f5775db43 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 10 Mar 2021 13:38:46 -0500 Subject: [PATCH 0145/1277] Add our fork of RtspServer as a module --- .gitmodules | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.gitmodules b/.gitmodules index 9f21bb62b..721bc1da1 100644 --- a/.gitmodules +++ b/.gitmodules @@ -5,3 +5,6 @@ [submodule "web/api/app/Plugin/CakePHP-Enum-Behavior"] path = web/api/app/Plugin/CakePHP-Enum-Behavior url = https://github.com/ZoneMinder/CakePHP-Enum-Behavior.git +[submodule "dep/RtspServer"] + path = dep/RtspServer + url = https://github.com/ZoneMinder/RtspServer.git From 15bd3276d229294d071587989bd0c409c7c73903 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 10 Mar 2021 13:50:42 -0500 Subject: [PATCH 0146/1277] fix define --- src/zm_rtsp_server_authenticator.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/zm_rtsp_server_authenticator.h b/src/zm_rtsp_server_authenticator.h index 5bbc0b38b..a1b2a5c45 100644 --- a/src/zm_rtsp_server_authenticator.h +++ b/src/zm_rtsp_server_authenticator.h @@ -1,4 +1,4 @@ -#ifndef ZM_RTSP_SERVER_D_H +#ifndef ZM_RTSP_SERVER_AUTHENTICATOR_H #define ZM_RTSP_SERVER_AUTHENTICATOR_H #include "zm_config.h" From 08110fe9b15203c57922c68ca5408ac2486cba04 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 10 Mar 2021 14:02:03 -0500 Subject: [PATCH 0147/1277] try again at adding RtspServer submodule --- .gitmodules | 2 +- dep/RtspServer | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) create mode 160000 dep/RtspServer diff --git a/.gitmodules b/.gitmodules index 721bc1da1..be290a26c 100644 --- a/.gitmodules +++ b/.gitmodules @@ -7,4 +7,4 @@ url = https://github.com/ZoneMinder/CakePHP-Enum-Behavior.git [submodule "dep/RtspServer"] path = dep/RtspServer - url = https://github.com/ZoneMinder/RtspServer.git + url = https://github.com/ZoneMinder/RtspServer diff --git a/dep/RtspServer b/dep/RtspServer new file mode 160000 index 000000000..042e6e56e --- /dev/null +++ b/dep/RtspServer @@ -0,0 +1 @@ +Subproject commit 042e6e56e37e2d3667de2daa2d502c680bcdde59 From faddace69d29f2f0892652542e46f683e2e2258c Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 10 Mar 2021 15:59:38 -0500 Subject: [PATCH 0148/1277] Update version --- dep/RtspServer | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dep/RtspServer b/dep/RtspServer index 042e6e56e..75c20b450 160000 --- a/dep/RtspServer +++ b/dep/RtspServer @@ -1 +1 @@ -Subproject commit 042e6e56e37e2d3667de2daa2d502c680bcdde59 +Subproject commit 75c20b450fdf00c120b9a917f48abd96095b665b From 93e4691b085a90fd723716303df631cb20df40ec Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 10 Mar 2021 14:15:32 -0500 Subject: [PATCH 0149/1277] remove liblivemedia as a dependency. We have gone another way --- distros/ubuntu1604/control | 2 -- distros/ubuntu2004/control | 2 -- 2 files changed, 4 deletions(-) diff --git a/distros/ubuntu1604/control b/distros/ubuntu1604/control index 310cbac7c..0def73f60 100644 --- a/distros/ubuntu1604/control +++ b/distros/ubuntu1604/control @@ -33,7 +33,6 @@ Build-Depends: debhelper (>= 9), dh-systemd, python3-sphinx, apache2-dev, dh-lin ,libcrypt-eksblowfish-perl ,libdata-entropy-perl ,libvncserver-dev - ,liblivemedia-dev Standards-Version: 3.9.8 Homepage: http://www.zoneminder.com/ Vcs-Browser: http://anonscm.debian.org/cgit/collab-maint/zoneminder.git @@ -80,7 +79,6 @@ Depends: ${shlibs:Depends}, ${misc:Depends}, ${perl:Depends} ,libcrypt-eksblowfish-perl ,libdata-entropy-perl ,libvncclient1|libvncclient0 - ,liblivemedia50|liblivemedia62 Recommends: ${misc:Recommends} ,libapache2-mod-php5 | libapache2-mod-php | php5-fpm | php-fpm ,mysql-server | mariadb-server | virtual-mysql-server diff --git a/distros/ubuntu2004/control b/distros/ubuntu2004/control index 67554b8ea..28c9374a1 100644 --- a/distros/ubuntu2004/control +++ b/distros/ubuntu2004/control @@ -32,7 +32,6 @@ Build-Depends: debhelper (>= 12), sphinx-doc, python3-sphinx, dh-linktree, dh-ap ,libcrypt-eksblowfish-perl ,libdata-entropy-perl ,libvncserver-dev - ,liblivemedia-dev Standards-Version: 4.5.0 Homepage: https://www.zoneminder.com/ @@ -75,7 +74,6 @@ Depends: ${shlibs:Depends}, ${misc:Depends}, ${perl:Depends} ,libcrypt-eksblowfish-perl ,libdata-entropy-perl ,libvncclient1|libvncclient0 - ,liblivemedia62|liblivemedia64|liblivemedia77 Recommends: ${misc:Recommends} ,libapache2-mod-php | php-fpm ,default-mysql-server | mariadb-server | virtual-mysql-server From 000df68f72553c16f80e21c121c3d61d37f94097 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 10 Mar 2021 16:16:22 -0500 Subject: [PATCH 0150/1277] include memory --- src/zm_utils.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/src/zm_utils.cpp b/src/zm_utils.cpp index 905500b45..702c619c7 100644 --- a/src/zm_utils.cpp +++ b/src/zm_utils.cpp @@ -25,6 +25,7 @@ #include #include #include /* Definition of AT_* constants */ +#include #include #include From 5d968358f7e44cd484bf3f6f62d19d2e0b1f9e5c Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 10 Mar 2021 16:35:03 -0500 Subject: [PATCH 0151/1277] Need to use ZM::make_unique because std::make_unique is c++14 --- src/zm_utils.cpp | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/zm_utils.cpp b/src/zm_utils.cpp index 702c619c7..3fce11d1c 100644 --- a/src/zm_utils.cpp +++ b/src/zm_utils.cpp @@ -25,7 +25,6 @@ #include #include #include /* Definition of AT_* constants */ -#include #include #include @@ -407,7 +406,7 @@ QueryString::QueryString(std::istream &input) { auto foundItr = parameters_.find(name); if (foundItr == parameters_.end()) { - auto newParam = std::make_unique(name); + auto newParam = ZM::make_unique(name); if (value.size() > 0) { newParam->addValue(value); } From 644d6d34a42c2415977d26042cd7969a5c863301 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 10 Mar 2021 17:23:44 -0500 Subject: [PATCH 0152/1277] Must init video_stream_id and audio_stream_id --- src/zm_monitor.cpp | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 5618314d0..88028ca32 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -386,6 +386,8 @@ Monitor::Monitor() video_store_data(nullptr), shared_timestamps(nullptr), shared_images(nullptr), + video_stream_id(-1), + audio_stream_id(-1), video_fifo(nullptr), audio_fifo(nullptr), camera(nullptr), From 0da9ee2e4c0f2ddc3714c819c19961fb0d5f95e3 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 11 Mar 2021 08:07:45 -0500 Subject: [PATCH 0153/1277] Remove live555 detection --- CMakeLists.txt | 14 +----- cmake/Modules/FindLive555.cmake | 75 --------------------------------- 2 files changed, 2 insertions(+), 87 deletions(-) delete mode 100644 cmake/Modules/FindLive555.cmake diff --git a/CMakeLists.txt b/CMakeLists.txt index 9acaedc9b..76c48f530 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -169,7 +169,7 @@ set(ZM_ONVIF "ON" CACHE BOOL "Set to ON to enable basic ONVIF support. This is EXPERIMENTAL and may not work with all cameras claiming to be ONVIF compliant. default: ON") set(ZM_NO_RTSPSERVER "OFF" CACHE BOOL - "Set to ON to skip live555 checks and force building ZM without rtsp server support. default: OFF") + "Set to ON to skip building ZM with rtsp server support. default: OFF") set(ZM_PERL_MM_PARMS INSTALLDIRS=vendor NO_PACKLIST=1 NO_PERLLOCAL=1 CACHE STRING "By default, ZoneMinder's Perl modules are installed into the Vendor folders, as defined by your installation of Perl. You can change that here. Consult Perl's @@ -629,17 +629,7 @@ endif() #endif() if(NOT ZM_NO_RTSPSERVER) - find_package(Live555) - if(Live555_FOUND) - include_directories(${Live555_INCLUDE_DIRS}) - set(CMAKE_REQUIRED_INCLUDES "${Live555_INCLUDE_DIRS}") - list(APPEND ZM_BIN_LIBS "${Live555_LIBRARIES}") - set(HAVE_RTSP_SERVER 1) - set(optlibsfound "${optlibsfound} libLive555") - else() - set(HAVE_RTSP_SERVER 0) - set(optlibsnotfound "${optlibsnotfound} libLive555") - endif() + set(HAVE_RTSP_SERVER 1) else() set(HAVE_RTSP_SERVER 0) endif() diff --git a/cmake/Modules/FindLive555.cmake b/cmake/Modules/FindLive555.cmake deleted file mode 100644 index c0f8550d1..000000000 --- a/cmake/Modules/FindLive555.cmake +++ /dev/null @@ -1,75 +0,0 @@ -# Try to find Live555 libraries -# Once done this will define -# Live555_FOUND -# Live555_INCLUDE_DIRS -# Live555_LIBRARIES - -if (NOT Live555_FOUND) - set(_Live555_FOUND ON) - - foreach (library liveMedia BasicUsageEnvironment Groupsock UsageEnvironment) - - string(TOLOWER ${library} lowercase_library) - - find_path(Live555_${library}_INCLUDE_DIR - NAMES - ${library}.hh - ${lowercase_library}.hh - PATHS - ${Live555_ROOT}/${library}/include - ${Live555_ROOT}/live/${library}/include - /usr/include/${library} - /usr/local/include/${library} - /usr/include/${lowercase_library} - /usr/local/include/${lowercase_library} - ) - - if (Live555_${library}_INCLUDE_DIR) - list(APPEND _Live555_INCLUDE_DIRS ${Live555_${library}_INCLUDE_DIR}) - else() - set(_Live555_FOUND OFF) - endif () - - foreach (mode DEBUG RELEASE) - find_library(Live555_${library}_LIBRARY_${mode} - NAMES - ${library} - ${lowercase_library} - PATHS - ${Live555_ROOT}/lib/${mode} - ${Live555_ROOT}/${library} - ) - if (Live555_${library}_LIBRARY_${mode}) - if (${mode} STREQUAL RELEASE) - list(APPEND _Live555_LIBRARIES optimized ${Live555_${library}_LIBRARY_${mode}}) - elseif (${mode} STREQUAL DEBUG) - list(APPEND _Live555_LIBRARIES debug ${Live555_${library}_LIBRARY_${mode}}) - else () - MESSAGE(STATUS no) - list(APPEND _Live555_LIBRARIES ${Live555_${library}_LIBRARY_${mode}}) - endif() - else() - set(_Live555_FOUND OFF) - endif () - endforeach () - - endforeach () - - if (_Live555_FOUND) - set(Live555_INCLUDE_DIRS ${_Live555_INCLUDE_DIRS} CACHE INTERNAL "") - set(Live555_LIBRARIES ${_Live555_LIBRARIES} CACHE INTERNAL "") - set(Live555_FOUND ${_Live555_FOUND} CACHE BOOL "" FORCE) - endif() - - include(FindPackageHandleStandardArgs) - # handle the QUIETLY and REQUIRED arguments and set LOGGING_FOUND to TRUE - # if all listed variables are TRUE - find_package_handle_standard_args(Live555 DEFAULT_MSG Live555_INCLUDE_DIRS Live555_LIBRARIES Live555_FOUND) - - # Tell cmake GUIs to ignore the "local" variables. - mark_as_advanced(Live555_INCLUDE_DIRS Live555_LIBRARIES Live555_FOUND) -endif (NOT Live555_FOUND) - -if (Live555_FOUND) - message(STATUS "Found live555") -endif() From 874e61d68187702f18da36804d6fc4e9e28cf92b Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 11 Mar 2021 09:26:50 -0500 Subject: [PATCH 0154/1277] remove Base64 include from live555 --- src/zm_rtsp_server_fifo_h264_source.cpp | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/zm_rtsp_server_fifo_h264_source.cpp b/src/zm_rtsp_server_fifo_h264_source.cpp index 6d8a5393e..6d7dda1b1 100644 --- a/src/zm_rtsp_server_fifo_h264_source.cpp +++ b/src/zm_rtsp_server_fifo_h264_source.cpp @@ -14,8 +14,6 @@ #include #if HAVE_RTSP_SERVER -// live555 -#include // --------------------------------- // H264 ZoneMinder FramedSource From f2553220b6e2f53dd127374631edfa7f549c5845 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 11 Mar 2021 11:45:39 -0500 Subject: [PATCH 0155/1277] Implement xop::Authenticator::GetFailedResponse to return a 401 --- dep/RtspServer | 2 +- src/zm_rtsp_server_authenticator.h | 10 +++++++++- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/dep/RtspServer b/dep/RtspServer index 75c20b450..20846b25f 160000 --- a/dep/RtspServer +++ b/dep/RtspServer @@ -1 +1 @@ -Subproject commit 75c20b450fdf00c120b9a917f48abd96095b665b +Subproject commit 20846b25ffc0c9a1de1b6701ca99425ef39e9f3f diff --git a/src/zm_rtsp_server_authenticator.h b/src/zm_rtsp_server_authenticator.h index a1b2a5c45..55a7784e7 100644 --- a/src/zm_rtsp_server_authenticator.h +++ b/src/zm_rtsp_server_authenticator.h @@ -17,7 +17,7 @@ class ZM_RtspServer_Authenticator : public xop::Authenticator { ZM_RtspServer_Authenticator() {}; ~ZM_RtspServer_Authenticator() {}; - virtual bool Authenticate(std::shared_ptr request, std::string &nonce) { + bool Authenticate(std::shared_ptr request, std::string &nonce) { if (!config.opt_use_auth) { Debug(1, "Not doing auth"); @@ -77,6 +77,14 @@ class ZM_RtspServer_Authenticator : public xop::Authenticator { } return false; } + + size_t GetFailedResponse( + std::shared_ptr request, + std::shared_ptr buf, + size_t size) { + return request->BuildUnauthorizedRes(buf.get(), size); + } + }; // end class ZM_RtspServer_Authenticator #endif // HAVE_RTSP_SERVER From 43e7e612c52ab83e2e0f5d1a8ded1f1743c57948 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 11 Mar 2021 13:07:47 -0500 Subject: [PATCH 0156/1277] Have to turn off DB logging when logging from a db query or else we infinite loop --- src/zm_db.cpp | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/src/zm_db.cpp b/src/zm_db.cpp index 77bb20094..55e2c8462 100644 --- a/src/zm_db.cpp +++ b/src/zm_db.cpp @@ -24,7 +24,7 @@ MYSQL dbconn; std::mutex db_mutex; -zmDbQueue dbQueue; +zmDbQueue dbQueue; bool zmDbConnected = false; @@ -173,12 +173,21 @@ int zmDbDo(const char *query) { return 0; int rc; while ((rc = mysql_query(&dbconn, query)) and !zm_terminate) { + Logger *logger = Logger::fetch(); + Logger::Level oldLevel = logger->databaseLevel(); + logger->databaseLevel(Logger::NOLOG); Error("Can't run query %s: %s", query, mysql_error(&dbconn)); + logger->databaseLevel(oldLevel); if ( (mysql_errno(&dbconn) != ER_LOCK_WAIT_TIMEOUT) ) { return rc; } } + Logger *logger = Logger::fetch(); + Logger::Level oldLevel = logger->databaseLevel(); + logger->databaseLevel(Logger::NOLOG); + Debug(1, "Success running sql query %s", query); + logger->databaseLevel(oldLevel); return 1; } From 86a26ef3b1e5d03668c258af29ef41405a2f8b91 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 11 Mar 2021 13:16:32 -0500 Subject: [PATCH 0157/1277] nvsnprintf won't exceed the buffer but it can hit the end of it so adding the ending ]\n can overflow. Test and prevent. --- src/zm_logger.cpp | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/zm_logger.cpp b/src/zm_logger.cpp index 914f9eefc..75d4d4865 100644 --- a/src/zm_logger.cpp +++ b/src/zm_logger.cpp @@ -427,7 +427,7 @@ void Logger::logPrint(bool hex, const char * const filepath, const int line, con log_mutex.lock(); // Can we save some cycles by having these as members and not allocate them on the fly? I think so. char timeString[64]; - char logString[8192]; + char logString[4096]; // SQL TEXT can hold 64k so we could go up to 32k here but why? va_list argPtr; struct timeval timeVal; @@ -500,6 +500,11 @@ void Logger::logPrint(bool hex, const char * const filepath, const int line, con } va_end(argPtr); char *syslogEnd = logPtr; + + if ( static_cast(logPtr - logString) >= sizeof(logString) ) { + // vsnprintf won't exceed the the buffer, but it might hit the end. + logPtr = logString + sizeof(logString)-3; + } strncpy(logPtr, "]\n", sizeof(logString)-(logPtr-logString)); if (level <= mTerminalLevel) { From b5f45b0987729e44c4feabe2c418fed18256d4ab Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 11 Mar 2021 13:16:44 -0500 Subject: [PATCH 0158/1277] fix alignment of help --- src/zmc.cpp | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/zmc.cpp b/src/zmc.cpp index c4e12c7a0..19c8891c7 100644 --- a/src/zmc.cpp +++ b/src/zmc.cpp @@ -73,12 +73,12 @@ void Usage() { fprintf(stderr, "Options:\n"); #if defined(BSD) - fprintf(stderr, " -d, --device : For local cameras, device to access. E.g /dev/bktr0 etc\n"); + fprintf(stderr, " -d, --device : For local cameras, device to access. E.g /dev/bktr0 etc\n"); #else - fprintf(stderr, " -d, --device : For local cameras, device to access. E.g /dev/video0 etc\n"); + fprintf(stderr, " -d, --device : For local cameras, device to access. E.g /dev/video0 etc\n"); #endif - fprintf(stderr, " -f, --file : For local images, jpg file to access.\n"); - fprintf(stderr, " -m, --monitor : For sources associated with a single monitor\n"); + fprintf(stderr, " -f, --file : For local images, jpg file to access.\n"); + fprintf(stderr, " -m, --monitor : For sources associated with a single monitor\n"); fprintf(stderr, " -h, --help : This screen\n"); fprintf(stderr, " -v, --version : Report the installed version of ZoneMinder\n"); exit(0); From cbec5b2800f1aed6c58098c10aa8f6ae4937adaa Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 11 Mar 2021 13:48:16 -0500 Subject: [PATCH 0159/1277] Implement zmDbDoUpdate which returns -mysql_errer or # of rows modified --- src/zm_db.cpp | 16 +++++++++++++++- src/zm_db.h | 1 + 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/src/zm_db.cpp b/src/zm_db.cpp index 55e2c8462..1557b8b72 100644 --- a/src/zm_db.cpp +++ b/src/zm_db.cpp @@ -201,10 +201,24 @@ int zmDbDoInsert(const char *query) { return 0; } int id = mysql_insert_id(&dbconn); - Debug(1, "Success running sql insert %s. Resulting id is %d", query, id); + Debug(2, "Success running sql insert %s. Resulting id is %d", query, id); return id; } +int zmDbDoUpdate(const char *query) { + std::lock_guard lck(db_mutex); + if (!zmDbConnected) return 0; + int rc; + while ( (rc = mysql_query(&dbconn, query)) and !zm_terminate) { + Error("Can't run query %s: %s", query, mysql_error(&dbconn)); + if ( (mysql_errno(&dbconn) != ER_LOCK_WAIT_TIMEOUT) ) + return -rc; + } + int affected = mysql_affected_rows(&dbconn); + Debug(2, "Success running sql update %s. Rows modified %d", query, affected); + return affected; +} + zmDbRow::~zmDbRow() { if (result_set) { mysql_free_result(result_set); diff --git a/src/zm_db.h b/src/zm_db.h index eabbebca4..499b9af20 100644 --- a/src/zm_db.h +++ b/src/zm_db.h @@ -70,6 +70,7 @@ bool zmDbConnect(); void zmDbClose(); int zmDbDo(const char *query); int zmDbDoInsert(const char *query); +int zmDbDoUpdate(const char *query); MYSQL_RES * zmDbFetch(const char *query); zmDbRow *zmDbFetchOne(const char *query); From 28490816dc06a439f4f0129aa0bb11e35c71939d Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 11 Mar 2021 13:48:42 -0500 Subject: [PATCH 0160/1277] Use new zmDbDoUpdate to end the event --- src/zm_event.cpp | 38 ++++++++++++++++---------------------- 1 file changed, 16 insertions(+), 22 deletions(-) diff --git a/src/zm_event.cpp b/src/zm_event.cpp index d6e3985b7..f0b2da694 100644 --- a/src/zm_event.cpp +++ b/src/zm_event.cpp @@ -240,8 +240,7 @@ Event::~Event() { DELTA_TIMEVAL(delta_time, end_time, start_time, DT_PREC_2); Debug(2, "start_time:%d.%d end_time%d.%d", start_time.tv_sec, start_time.tv_usec, end_time.tv_sec, end_time.tv_usec); - if ( frame_data.size() ) - WriteDbFrames(); + if (frame_data.size()) WriteDbFrames(); // Should not be static because we might be multi-threaded char sql[ZM_SQL_LGE_BUFSIZ]; @@ -252,25 +251,17 @@ Event::~Event() { frames, alarm_frames, tot_score, (int)(alarm_frames?(tot_score/alarm_frames):0), max_score, id); - { // scope for lock - std::lock_guard lck(db_mutex); - while ( mysql_query(&dbconn, sql) && !zm_terminate ) { - Error("Can't update event: %s reason: %s", sql, mysql_error(&dbconn)); - } - if ( !mysql_affected_rows(&dbconn) ) { - // Name might have been changed during recording, so just do the update without changing the name. - snprintf(sql, sizeof(sql), - "UPDATE Events SET EndDateTime = from_unixtime(%ld), Length = %s%ld.%02ld, Frames = %d, AlarmFrames = %d, TotScore = %d, AvgScore = %d, MaxScore = %d WHERE Id = %" PRIu64, - end_time.tv_sec, - delta_time.positive?"":"-", delta_time.sec, delta_time.fsec, - frames, alarm_frames, - tot_score, (int)(alarm_frames?(tot_score/alarm_frames):0), max_score, - id); - while ( mysql_query(&dbconn, sql) && !zm_terminate ) { - Error("Can't update event: %s reason: %s", sql, mysql_error(&dbconn)); - } - } // end if no changed rows due to Name change during recording - } + if (!zmDbDoUpdate(sql)) { + // Name might have been changed during recording, so just do the update without changing the name. + snprintf(sql, sizeof(sql), + "UPDATE Events SET EndDateTime = from_unixtime(%ld), Length = %s%ld.%02ld, Frames = %d, AlarmFrames = %d, TotScore = %d, AvgScore = %d, MaxScore = %d WHERE Id = %" PRIu64, + end_time.tv_sec, + delta_time.positive?"":"-", delta_time.sec, delta_time.fsec, + frames, alarm_frames, + tot_score, (int)(alarm_frames?(tot_score/alarm_frames):0), max_score, + id); + zmDbDoUpdate(sql); + } // end if no changed rows due to Name change during recording } // Event::~Event() void Event::createNotes(std::string ¬es) { @@ -535,7 +526,10 @@ void Event::WriteDbFrames() { delete frame; } *(frame_insert_values_ptr-1) = '\0'; // The -1 is for the extra , added for values above - zmDbDo(frame_insert_sql); + std::string sql(frame_insert_sql); + + dbQueue.push(std::move(sql)); + //zmDbDo(frame_insert_sql); } // end void Event::WriteDbFrames() void Event::AddFrame(Image *image, struct timeval timestamp, int score, Image *alarm_image) { From 786adc55115a9d180ddae68b26aa794dfb88ab74 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 11 Mar 2021 13:48:52 -0500 Subject: [PATCH 0161/1277] Spacing --- src/zm_packetqueue.cpp | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/src/zm_packetqueue.cpp b/src/zm_packetqueue.cpp index 40aeff3bd..2ba04e71b 100644 --- a/src/zm_packetqueue.cpp +++ b/src/zm_packetqueue.cpp @@ -322,7 +322,7 @@ void PacketQueue::clear() { std::unique_lock lck(mutex); - while ( !pktQueue.empty() ) { + while (!pktQueue.empty()) { ZMPacket *packet = pktQueue.front(); // Someone might have this packet, but not for very long and since we have locked the queue they won't be able to get another one packet->lock(); @@ -660,10 +660,11 @@ bool PacketQueue::is_there_an_iterator_pointing_to_packet(ZMPacket *zm_packet) { } // end foreach iterator return false; } - void PacketQueue::setMaxVideoPackets(int p) { - max_video_packet_count = p; - Debug(1, "Setting max_video_packet_count to %d", p); - if ( max_video_packet_count < 1 ) - max_video_packet_count = 1 ; - // We can simplify a lot of logic in queuePacket if we can assume at least 1 packet in queue - } + +void PacketQueue::setMaxVideoPackets(int p) { + max_video_packet_count = p; + Debug(1, "Setting max_video_packet_count to %d", p); + if ( max_video_packet_count < 1 ) + max_video_packet_count = 1 ; + // We can simplify a lot of logic in queuePacket if we can assume at least 1 packet in queue +} From 620806a1bf49677afacf303b9cddb8c615a3e7e6 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 12 Mar 2021 09:26:13 -0500 Subject: [PATCH 0162/1277] Add Snapshots and Snapshot_Events Tables. Add HomeView to Users. --- db/zm_create.sql.in | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/db/zm_create.sql.in b/db/zm_create.sql.in index ae8d500ab..dec8814d2 100644 --- a/db/zm_create.sql.in +++ b/db/zm_create.sql.in @@ -677,6 +677,7 @@ CREATE TABLE `Users` ( `MonitorIds` text, `TokenMinExpiry` BIGINT UNSIGNED NOT NULL DEFAULT 0, `APIEnabled` tinyint(3) UNSIGNED NOT NULL default 1, + `HomeView` varchar(64) NOT NULL DEFAULT '', PRIMARY KEY (`Id`), UNIQUE KEY `UC_Username` (`Username`) ) ENGINE=@ZM_MYSQL_ENGINE@; @@ -1046,6 +1047,26 @@ CREATE TABLE Sessions ( PRIMARY KEY(id) ) ENGINE=InnoDB; +CREATE TABLE Snapshots ( + `Id` int(10) unsigned NOT NULL auto_increment, + `Name` VARCHAR(64), + `Description` TEXT, + `CreatedBy` int(10), + `CreatedOn` datetime default NULL, + PRIMARY KEY(Id) +) ENGINE=InnoDB; + +CREATE TABLE Snapshot_Events ( + `Id` int(10) unsigned NOT NULL auto_increment, + `SnapshotId` int(10) unsigned NOT NULL, + FOREIGN KEY (`SnapshotId`) REFERENCES `Snapshots` (`Id`) ON DELETE CASCADE, + `EventId` bigint unsigned NOT NULL, + FOREIGN KEY (`EventId`) REFERENCES `Events` (`Id`) ON DELETE CASCADE, + KEY `Snapshot_Events_SnapshotId_idx` (`SnapshotId`), + PRIMARY KEY(Id) +) ENGINE=InnoDB; + + -- We generally don't alter triggers, we drop and re-create them, so let's keep them in a separate file that we can just source in update scripts. source @PKGDATADIR@/db/triggers.sql -- From 6682ec7da55294d72ecb979b24c187526d76da3c Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 12 Mar 2021 09:26:23 -0500 Subject: [PATCH 0163/1277] Add Snapshots and Snapshot_Events Tables. Add HomeView to Users. --- web/includes/Monitor.php | 15 +++++++++++++++ web/includes/Object.php | 10 ++++++++-- web/includes/User.php | 1 + web/skins/classic/includes/functions.php | 16 +++++++++++++++- web/skins/classic/js/skin.js.php | 7 +++++++ web/skins/classic/views/js/montage.js | 5 +++++ web/skins/classic/views/js/postlogin.js.php | 2 +- web/skins/classic/views/montage.php | 5 +++++ web/skins/classic/views/user.php | 5 +++++ 9 files changed, 62 insertions(+), 4 deletions(-) diff --git a/web/includes/Monitor.php b/web/includes/Monitor.php index 189adb08f..9470c68f3 100644 --- a/web/includes/Monitor.php +++ b/web/includes/Monitor.php @@ -578,5 +578,20 @@ class Monitor extends ZM_Object { global $user; return ( $user && ($user['Monitors'] == 'Edit') && ( !$this->{'Id'} || visibleMonitor($this->{'Id'}) )); } + + function TriggerOn() { + $cmd = getZmuCommand(' -a -m '.$this->{'Id'}); + $output = shell_exec($cmd); + Debug("Running $cmd output: $output"); + if ( $output and preg_match('/Alarmed event id: (\d+)$/', $output, $matches) ) { + return $matches[1]; + } + Warning("No event returned from TriggerOn"); + return 0; + } + function TriggerOff() { + $cmd = getZmuCommand(' -c -m '.$this->{'Id'}); + $output = shell_exec($cmd); + } } // end class Monitor ?> diff --git a/web/includes/Object.php b/web/includes/Object.php index 5820d6541..9c9a3c095 100644 --- a/web/includes/Object.php +++ b/web/includes/Object.php @@ -337,9 +337,12 @@ class ZM_Object { $sql = 'INSERT INTO `'.$table. '` ('.implode(', ', array_map(function($field) {return '`'.$field.'`';}, $fields)). ') VALUES ('. - implode(', ', array_map(function($field){return '?';}, $fields)).')'; + implode(', ', array_map(function($field){return $this->$field() == 'NOW()' ? 'NOW()' : '?';}, $fields)).')'; - $values = array_map(function($field){return $this->$field();}, $fields); + $values = array_values(array_map( + function($field){return $this->$field();}, + array_filter($fields, function($field){ return $this->$field() != 'NOW()';}) + )); if ( dbQuery($sql, $values) ) { $this->{'Id'} = dbInsertId(); return true; @@ -415,5 +418,8 @@ class ZM_Object { Error("Unable to lock $class record for Id=".$this->Id()); } } + public function remove_from_cache() { + return ZM_Object::_remove_from_cache(get_class(), $this); + } } # end class Object ?> diff --git a/web/includes/User.php b/web/includes/User.php index daebd57cf..3a46ea551 100644 --- a/web/includes/User.php +++ b/web/includes/User.php @@ -24,6 +24,7 @@ class User extends ZM_Object { 'MonitorIds' => '', 'TokenMinExpiry' => 0, 'APIEnabled' => 1, + 'HomeView' => '', ); public static function find( $parameters = array(), $options = array() ) { diff --git a/web/skins/classic/includes/functions.php b/web/skins/classic/includes/functions.php index 734eb1b11..aedb6b68c 100644 --- a/web/skins/classic/includes/functions.php +++ b/web/skins/classic/includes/functions.php @@ -233,6 +233,7 @@ function getNormalNavBarHTML($running, $user, $bandwidth_options, $view, $skin) echo getCycleHTML($view); echo getMontageHTML($view); echo getMontageReviewHTML($view); + echo getSnapshotsHTML($view); echo getRprtEvntAuditHTML($view); echo getHeaderFlipHTML(); echo ''; @@ -361,6 +362,7 @@ function getCollapsedNavBarHTML($running, $user, $bandwidth_options, $view, $ski echo getCycleHTML($view); echo getMontageHTML($view); echo getMontageReviewHTML($view); + echo getSnapshotsHTML($view); echo getRprtEvntAuditHTML($view); echo ''; } @@ -672,7 +674,7 @@ function getMontageHTML($view) { $result = ''; if ( canView('Stream') ) { - $class = $view == 'cycle' ? ' selected' : ''; + $class = $view == 'montage' ? ' selected' : ''; $result .= ''.PHP_EOL; } @@ -706,6 +708,18 @@ function getMontageReviewHTML($view) { return $result; } +// Returns the html representing the Montage menu item +function getSnapshotsHTML($view) { + $result = ''; + + if ( canView('Events') ) { + $class = $view == 'snapshots' ? ' selected' : ''; + $result .= ''.PHP_EOL; + } + + return $result; +} + // Returns the html representing the Audit Events Report menu item function getRprtEvntAuditHTML($view) { $result = ''; diff --git a/web/skins/classic/js/skin.js.php b/web/skins/classic/js/skin.js.php index 4ba7efb73..5fe31164a 100644 --- a/web/skins/classic/js/skin.js.php +++ b/web/skins/classic/js/skin.js.php @@ -23,6 +23,7 @@ // Static JavaScript should go in skin.js // +global $user; ?> var AJAX_TIMEOUT = ; var navBarRefresh = ; @@ -82,4 +83,10 @@ var imagePrefix = ""; var auth_hash = ''; var auth_relay = ''; +var user = { + + "Id" : "", + "Username" : "" + +}; var running = ; diff --git a/web/skins/classic/views/js/montage.js b/web/skins/classic/views/js/montage.js index 2b1fa750a..1a3c901d4 100644 --- a/web/skins/classic/views/js/montage.js +++ b/web/skins/classic/views/js/montage.js @@ -270,6 +270,11 @@ function reloadWebSite(ndx) { document.getElementById('imageFeed'+ndx).innerHTML = document.getElementById('imageFeed'+ndx).innerHTML; } +function takeSnapshot() { + monitor_ids = monitorData.map( monitor=> { return 'monitor_ids[]='+monitor.id; }); + window.location = '?view=snapshot&action=create&'+monitor_ids.join('&'); +} + var monitors = new Array(); function initPage() { $j("#hdrbutton").click(function() { diff --git a/web/skins/classic/views/js/postlogin.js.php b/web/skins/classic/views/js/postlogin.js.php index b38b25955..27e1679ef 100644 --- a/web/skins/classic/views/js/postlogin.js.php +++ b/web/skins/classic/views/js/postlogin.js.php @@ -7,7 +7,7 @@ // redirect the user to his original intended destination by appending it to the URL. // -$redirectSuffix = '?view=console'; +$redirectSuffix = $user['HomeView'] != '' ? $user['HomeView'] : '?view=console'; if ( !empty($_SESSION['postLoginQuery']) ) { parse_str($_SESSION['postLoginQuery'], $queryParams); $redirectSuffix = '?' . http_build_query($queryParams); diff --git a/web/skins/classic/views/montage.php b/web/skins/classic/views/montage.php index a66998800..fa7c58ac3 100644 --- a/web/skins/classic/views/montage.php +++ b/web/skins/classic/views/montage.php @@ -197,6 +197,11 @@ if ( $showZones ) { + +
diff --git a/web/skins/classic/views/user.php b/web/skins/classic/views/user.php index 17dd08469..981b205e2 100644 --- a/web/skins/classic/views/user.php +++ b/web/skins/classic/views/user.php @@ -155,6 +155,11 @@ if ( canEdit('System') and ( $newUser->Username() != 'admin' ) ) { } // end if ZM_OPT_USE_API } // end if canEdit(System) ?> + + + + +
From c9170a87b2050ec5bbf9eaeb32b837ec18403206 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 12 Mar 2021 09:26:56 -0500 Subject: [PATCH 0164/1277] Allow users with Monitors::View to generate and cancel events --- src/zmu.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/zmu.cpp b/src/zmu.cpp index e6c328cca..0bfe90b3b 100644 --- a/src/zmu.cpp +++ b/src/zmu.cpp @@ -179,11 +179,11 @@ bool ValidateAccess(User *user, int mon_id, int function) { if ( user->getEvents() < User::PERM_VIEW ) allowed = false; } - if ( function & (ZMU_ZONES|ZMU_QUERY|ZMU_LIST) ) { + if ( function & (ZMU_ZONES|ZMU_QUERY|ZMU_LIST|ZMU_ALARM|ZMU_CANCEL) ) { if ( user->getMonitors() < User::PERM_VIEW ) allowed = false; } - if ( function & (ZMU_ALARM|ZMU_NOALARM|ZMU_CANCEL|ZMU_RELOAD|ZMU_ENABLE|ZMU_DISABLE|ZMU_SUSPEND|ZMU_RESUME|ZMU_BRIGHTNESS|ZMU_CONTRAST|ZMU_HUE|ZMU_COLOUR) ) { + if ( function & (ZMU_NOALARM|ZMU_RELOAD|ZMU_ENABLE|ZMU_DISABLE|ZMU_SUSPEND|ZMU_RESUME|ZMU_BRIGHTNESS|ZMU_CONTRAST|ZMU_HUE|ZMU_COLOUR) ) { if ( user->getMonitors() < User::PERM_EDIT ) allowed = false; } From 11c2318a05706dc14bd13ec70db01ff5970934ba Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 12 Mar 2021 09:28:05 -0500 Subject: [PATCH 0165/1277] Rough in the ui for Snapshots --- db/zm_update-1.35.21.sql | 56 ++++ web/ajax/snapshots.php | 227 +++++++++++++ web/includes/Snapshot.php | 65 ++++ web/includes/actions/snapshot.php | 73 ++++ web/skins/classic/css/base/views/snapshot.css | 17 + web/skins/classic/views/js/snapshot.js | 115 +++++++ web/skins/classic/views/js/snapshot.js.php | 30 ++ web/skins/classic/views/js/snapshots.js | 312 ++++++++++++++++++ web/skins/classic/views/js/snapshots.js.php | 6 + web/skins/classic/views/snapshot.php | 98 ++++++ web/skins/classic/views/snapshots.php | 92 ++++++ 11 files changed, 1091 insertions(+) create mode 100644 db/zm_update-1.35.21.sql create mode 100644 web/ajax/snapshots.php create mode 100644 web/includes/Snapshot.php create mode 100644 web/includes/actions/snapshot.php create mode 100644 web/skins/classic/css/base/views/snapshot.css create mode 100644 web/skins/classic/views/js/snapshot.js create mode 100644 web/skins/classic/views/js/snapshot.js.php create mode 100644 web/skins/classic/views/js/snapshots.js create mode 100644 web/skins/classic/views/js/snapshots.js.php create mode 100644 web/skins/classic/views/snapshot.php create mode 100644 web/skins/classic/views/snapshots.php diff --git a/db/zm_update-1.35.21.sql b/db/zm_update-1.35.21.sql new file mode 100644 index 000000000..d3d80e08c --- /dev/null +++ b/db/zm_update-1.35.21.sql @@ -0,0 +1,56 @@ +-- +-- Add HomeView to Users +-- + +SET @s = (SELECT IF( + (SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS WHERE table_schema = DATABASE() + AND table_name = 'Users' + AND column_name = 'HomeView' + ) > 0, +"SELECT 'Column HomeView already exists in Users'", +"ALTER TABLE `Users` ADD `HomeView` varchar(64) NOT NULL DEFAULT '' AFTER `APIEnabled`" +)); + +PREPARE stmt FROM @s; +EXECUTE stmt; + +SET @s = (SELECT IF( + (SELECT COUNT(*) + FROM INFORMATION_SCHEMA.TABLES + WHERE table_name = 'Snapshots' + AND table_schema = DATABASE() + ) > 0, + "SELECT 'Snapshots table exists'", + "CREATE TABLE Snapshots ( + `Id` int(10) unsigned NOT NULL auto_increment, + `Name` VARCHAR(64), + `Description` TEXT, + `CreatedBy` int(10), + `CreatedOn` datetime default NULL, + PRIMARY KEY(Id) +) ENGINE=InnoDB;" +)); + +PREPARE stmt FROM @s; +EXECUTE stmt; + +SET @s = (SELECT IF( + (SELECT COUNT(*) + FROM INFORMATION_SCHEMA.TABLES + WHERE table_name = 'Snapshot_Events' + AND table_schema = DATABASE() + ) > 0, + "SELECT 'Snapshot_Events table exists'", + "CREATE TABLE Snapshot_Events ( + `Id` int(10) unsigned NOT NULL auto_increment, + `SnapshotId` int(10) unsigned NOT NULL, + FOREIGN KEY (`SnapshotId`) REFERENCES `Snapshots` (`Id`) ON DELETE CASCADE, + `EventId` bigint unsigned NOT NULL, + FOREIGN KEY (`EventId`) REFERENCES `Events` (`Id`) ON DELETE CASCADE, + KEY `Snapshot_Events_SnapshotId_idx` (`SnapshotId`), + PRIMARY KEY(Id) +) ENGINE=InnoDB;" +)); + +PREPARE stmt FROM @s; +EXECUTE stmt; diff --git a/web/ajax/snapshots.php b/web/ajax/snapshots.php new file mode 100644 index 000000000..e3da25ae3 --- /dev/null +++ b/web/ajax/snapshots.php @@ -0,0 +1,227 @@ + "search text" pairs +// Bootstrap table sends json_ecoded array, which we must decode +$advsearch = isset($_REQUEST['advsearch']) ? json_decode($_REQUEST['advsearch'], JSON_OBJECT_AS_ARRAY) : array(); + +// Sort specifies the name of the column to sort on +$sort = 'Id'; +if ( isset($_REQUEST['sort']) ) { + $sort = $_REQUEST['sort']; +} + +// Offset specifies the starting row to return, used for pagination +$offset = 0; +if ( isset($_REQUEST['offset']) ) { + if ( ( !is_int($_REQUEST['offset']) and !ctype_digit($_REQUEST['offset']) ) ) { + ZM\Error('Invalid value for offset: ' . $_REQUEST['offset']); + } else { + $offset = $_REQUEST['offset']; + } +} + +// Order specifies the sort direction, either asc or desc +$order = (isset($_REQUEST['order']) and (strtolower($_REQUEST['order']) == 'asc')) ? 'ASC' : 'DESC'; + +// Limit specifies the number of rows to return +// Set the default to 0 for events view, to prevent an issue with ALL pagination +$limit = 0; +if ( isset($_REQUEST['limit']) ) { + if ( ( !is_int($_REQUEST['limit']) and !ctype_digit($_REQUEST['limit']) ) ) { + ZM\Error('Invalid value for limit: ' . $_REQUEST['limit']); + } else { + $limit = $_REQUEST['limit']; + } +} + +// +// MAIN LOOP +// + +switch ( $task ) { + case 'delete' : + if ( !canEdit('Events') ) { + ajaxError('Insufficient permissions for user '.$user['Username']); + return; + } + + foreach ( $ids as $id ) $data[] = deleteRequest($id); + break; + case 'query' : + $data = queryRequest($search, $advsearch, $sort, $offset, $order, $limit); + break; + default : + ZM\Fatal("Unrecognised task '$task'"); +} // end switch task + +ajaxResponse($data); + +// +// FUNCTION DEFINITIONS +// + +function deleteRequest($id) { + $message = array(); + $snapshot = new ZM\Snapshot($id); + if ( !$snapshot->Id() ) { + $message[] = array($id=>'Snapshot not found.'); + //} else if ( $snapshot->Archived() ) { + //$message[] = array($id=>'Event is archived, cannot delete it.'); + } else { + $snapshot->delete(); + } + + return $message; +} + +function queryRequest($search, $advsearch, $sort, $offset, $order, $limit) { + + $data = array( + 'total' => 0, + 'totalNotFiltered' => 0, + 'rows' => array(), + 'updated' => preg_match('/%/', DATE_FMT_CONSOLE_LONG) ? strftime(DATE_FMT_CONSOLE_LONG) : date(DATE_FMT_CONSOLE_LONG) + ); + + // Put server pagination code here + // The table we want our data from + $table = 'Snapshots'; + + // The names of the dB columns in the events table we are interested in + $columns = array('Id', 'Name', 'Description', 'CreatedDateTime'); + + if ( !in_array($sort, array_merge($columns)) ) { + ZM\Error('Invalid sort field: ' . $sort); + $sort = 'Id'; + } + + $values = array(); + $likes = array(); + $where = ''; + + $col_str = '*'; + $sql = 'SELECT ' .$col_str. ' FROM `Snapshots`'.$where.' ORDER BY '.$sort.' '.$order; + + $unfiltered_rows = array(); + $event_ids = array(); + + ZM\Debug('Calling the following sql query: ' .$sql); + $query = dbQuery($sql, $values); + if ( $query ) { + while ( $row = dbFetchNext($query) ) { + $snapshot = new ZM\Snapshot($row); + $snapshot->remove_from_cache(); + $snapshot_ids[] = $snapshot->Id(); + $unfiltered_rows[] = $row; + } # end foreach row + } + + ZM\Debug('Have ' . count($unfiltered_rows) . ' snapshots matching base filter.'); + + $filtered_rows = null; + + if ( count($advsearch) or $search != '' ) { + $search_filter = new ZM\Filter(); + + $search_filter = $search_filter->addTerm(array('cnj'=>'and', 'attr'=>'Id', 'op'=>'IN', 'val'=>$event_ids)); + + // There are two search bars in the log view, normal and advanced + // Making an exuctive decision to ignore the normal search, when advanced search is in use + // Alternatively we could try to do both + if ( count($advsearch) ) { + $terms = array(); + foreach ( $advsearch as $col=>$text ) { + $terms[] = array('cnj'=>'and', 'attr'=>$col, 'op'=>'LIKE', 'val'=>$text); + } # end foreach col in advsearch + $terms[0]['obr'] = 1; + $terms[count($terms)-1]['cbr'] = 1; + $search_filter->addTerms($terms); + } else if ( $search != '' ) { + $search = '%' .$search. '%'; + $terms = array(); + foreach ( $columns as $col ) { + $terms[] = array('cnj'=>'or', 'attr'=>$col, 'op'=>'LIKE', 'val'=>$search); + } + $terms[0]['obr'] = 1; + $terms[0]['cnj'] = 'and'; + $terms[count($terms)-1]['cbr'] = 1; + $search_filter = $search_filter->addTerms($terms, array('obr'=>1, 'cbr'=>1, 'op'=>'OR')); + } # end if search + + $sql = 'SELECT ' .$col_str. ' FROM `Snapshots` WHERE '.$search_filter->sql().' ORDER BY ' .$sort. ' ' .$order; + ZM\Debug('Calling the following sql query: ' .$sql); + $filtered_rows = dbFetchAll($sql); + ZM\Debug('Have ' . count($filtered_rows) . ' snapshots matching search filter.'); + } else { + $filtered_rows = $unfiltered_rows; + } # end if search_filter->terms() > 1 + + $returned_rows = array(); + foreach ( array_slice($filtered_rows, $offset, $limit) as $row ) { + $snapshot = new ZM\Snapshot($row); + + //$scale = intval(5*100*ZM_WEB_LIST_THUMB_WIDTH / $event->Width()); + //$imgSrc = $event->getThumbnailSrc(array(), '&'); + //$streamSrc = $event->getStreamSrc(array( + //'mode'=>'jpeg', 'scale'=>$scale, 'maxfps'=>ZM_WEB_VIDEO_MAXFPS, 'replay'=>'single', 'rate'=>'400'), '&'); + + // Modify the row data as needed + //$row['imgHtml'] = 'Event '.$event->Id().''; + $row['Name'] = validHtmlStr($row['Name']); + $row['Description'] = validHtmlStr($row['Description']); + //$row['Archived'] = $row['Archived'] ? translate('Yes') : translate('No'); + //$row['Emailed'] = $row['Emailed'] ? translate('Yes') : translate('No'); + //$row['Cause'] = validHtmlStr($row['Cause']); + $row['CreatedOn'] = strftime(STRF_FMT_DATETIME_SHORTER, strtotime($row['CreatedOn'])); + //$row['StartDateTime'] = strftime(STRF_FMT_DATETIME_SHORTER, strtotime($row['StartDateTime'])); + //$row['EndDateTime'] = $row['EndDateTime'] ? strftime(STRF_FMT_DATETIME_SHORTER, strtotime($row['EndDateTime'])) : null; + //$row['Length'] = gmdate('H:i:s', $row['Length'] ); + //$row['Storage'] = ( $row['StorageId'] and isset($StorageById[$row['StorageId']]) ) ? $StorageById[$row['StorageId']]->Name() : 'Default'; + //$row['Notes'] = nl2br(htmlspecialchars($row['Notes'])); + //$row['DiskSpace'] = human_filesize($event->DiskSpace()); + $returned_rows[] = $row; + } # end foreach row matching search + + $data['rows'] = $returned_rows; + + # totalNotFiltered must equal total, except when either search bar has been used + $data['totalNotFiltered'] = count($unfiltered_rows); + if ( $search != '' || count($advsearch) ) { + $data['total'] = count($filtered_rows); + } else { + $data['total'] = $data['totalNotFiltered']; + } + + return $data; +} +?> diff --git a/web/includes/Snapshot.php b/web/includes/Snapshot.php new file mode 100644 index 000000000..2c36306f5 --- /dev/null +++ b/web/includes/Snapshot.php @@ -0,0 +1,65 @@ + null, + 'CreatedBy' => null, + 'CreatedOn' => 'NOW()', + 'Name' => '', + 'Description' => '', + ); + + public static function find( $parameters = array(), $options = array() ) { + return ZM_Object::_find(get_class(), $parameters, $options); + } + + public static function find_one( $parameters = array(), $options = array() ) { + return ZM_Object::_find_one(get_class(), $parameters, $options); + } + + public function Event() { + return new Event( $this->{'EventId'} ); + } + + public function delete() { + if ( property_exists($this, 'Id') ) { + dbQuery('DELETE FROM `Snapshot_Events` WHERE `SnapshotId`=?', array($this->{'Id'})); + } + } + + public function EventIds( ) { + if ( ! property_exists($this, 'EventIds') ) { + $this->{'EventIds'} = dbFetchAll('SELECT `EventId` FROM `Snapshot_Events` WHERE `SnapshotId`=?', 'EventId', array($this->{'Id'})); + } + return $this->{'EventIds'}; + } + public function Events() { + if ( ! property_exists($this, 'Events') ) { + $this->{'Events'} = Event::find(array('Id'=>$this->EventIds())); + } + return $this->{'Events'}; + } + +} # end class Snapshot + +class Snapshot_Event extends ZM_Object { + protected static $table = 'Snapshot_Events'; + protected $defaults = array( + 'Id' => null, + 'EventId' => 0, + 'SnapshotId' => 0, + ); + public static function find( $parameters = array(), $options = array() ) { + return ZM_Object::_find(get_class(), $parameters, $options); + } + + public static function find_one( $parameters = array(), $options = array() ) { + return ZM_Object::_find_one(get_class(), $parameters, $options); + } +} # end class Snapshot_Event +?> diff --git a/web/includes/actions/snapshot.php b/web/includes/actions/snapshot.php new file mode 100644 index 000000000..11edcb5a6 --- /dev/null +++ b/web/includes/actions/snapshot.php @@ -0,0 +1,73 @@ + 0 ) ) { + ZM\Error('No monitor ids given in snapshot creation request'); + return; + } + $snapshot = new ZM\Snapshot(); + $snapshot->save(array('CreatedBy'=>$user['Id'])); + + foreach ( $_REQUEST['monitor_ids'] as $monitor_id ) { + $snapshot_event = new ZM\Snapshot_Event(); + + $monitor = new ZM\Monitor($monitor_id); + $event_id = $monitor->TriggerOn(); + ZM\Debug("Have event $event_id for monitor $monitor_id"); + if ( $event_id ) { + $snapshot_event->save(array( + 'SnapshotId'=>$snapshot->Id(), + 'EventId'=>$event_id + )); + } + } # end foreach monitor + foreach ( $_REQUEST['monitor_ids'] as $monitor_id ) { + $monitor = new ZM\Monitor($monitor_id); + $monitor->TriggerOff(); + } + $redirect = '?view=snapshot&id='.$snapshot->Id(); + return; +} + +// Event scope actions, view permissions only required +if ( isset($_REQUEST['id']) ) { + $snapshot = new ZM\Snapshot($_REQUEST['id']); + if ( ($action == 'save') ) { + if ( canEdit('Events') or $snapshot->CreatedBy() == $user['Id'] ) { + + $changes = $snapshot->changes($_REQUEST['snapshot']); + if ( count($changes) ) { + $snapshot->save($changes); + } + $redirect = '?view=snapshots'; + } + } else if ( $action == 'delete' ) { + if ( canEdit('Events') ) { + $snapshot->delete(); + $redirect = '?view=snapshots'; + } + } +} // end if canEdit(Events) +?> diff --git a/web/skins/classic/css/base/views/snapshot.css b/web/skins/classic/css/base/views/snapshot.css new file mode 100644 index 000000000..b54036859 --- /dev/null +++ b/web/skins/classic/css/base/views/snapshot.css @@ -0,0 +1,17 @@ +#content { + margin: 0 15px; +} + +.Name { + width: 600px; +} +.Name input { + width: 100%; +} + +.Description { + width: 600px; +} +.Description textarea { + width: 100%; +} diff --git a/web/skins/classic/views/js/snapshot.js b/web/skins/classic/views/js/snapshot.js new file mode 100644 index 000000000..9db096277 --- /dev/null +++ b/web/skins/classic/views/js/snapshot.js @@ -0,0 +1,115 @@ +var backBtn = $j('#backBtn'); +var saveBtn = $j('#saveBtn'); +var deleteBtn = $j('#deleteBtn'); + +// Manage the DELETE CONFIRMATION modal button +function manageDelConfirmModalBtns() { + document.getElementById('delConfirmBtn').addEventListener('click', function onDelConfirmClick(evt) { + if ( !canEdit.Events ) { + enoperm(); + return; + } + + evt.preventDefault(); + /* + $j.getJSON(thisUrl + '?request=events&task=delete&eids[]='+eventData.Id) + .done(function(data) { + streamNext(true); + }) + .fail(logAjaxFail); + */ + }); + + // Manage the CANCEL modal button + document.getElementById("delCancelBtn").addEventListener("click", function onDelCancelClick(evt) { + $j('#deleteConfirm').modal('hide'); + }); +} + +function initPage() { + + // enable or disable buttons based on current selection and user rights + /* + renameBtn.prop('disabled', !canEdit.Events); + archiveBtn.prop('disabled', !(!eventData.Archived && canEdit.Events)); + unarchiveBtn.prop('disabled', !(eventData.Archived && canEdit.Events)); + */ + saveBtn.prop('disabled', !(canEdit.Events || (snapshot.CreatedBy == user.Id) )); + /* + exportBtn.prop('disabled', !canView.Events); + downloadBtn.prop('disabled', !canView.Events); + */ + deleteBtn.prop('disabled', !canEdit.Events); + + // Don't enable the back button if there is no previous zm page to go back to + backBtn.prop('disabled', !document.referrer.length); + + // Manage the BACK button + bindButton('#backBtn', 'click', null, function onBackClick(evt) { + evt.preventDefault(); + window.history.back(); + }); + + // Manage the REFRESH Button + bindButton('#refreshBtn', 'click', null, function onRefreshClick(evt) { + evt.preventDefault(); + window.location.reload(true); + }); + + // Manage the EDIT button + bindButton('#saveBtn', 'click', null, function onSaveClick(evt) { + /* + if ( ! canEdit.Events ) { + enoperm(); + return; + } + */ + console.log(evt); + evt.target.form.submit(); + }); + + /* + // Manage the EXPORT button + bindButton('#exportBtn', 'click', null, function onExportClick(evt) { + evt.preventDefault(); + window.location.assign('?view=export&eids[]='+eventData.Id); + }); + + // Manage the DOWNLOAD VIDEO button + bindButton('#downloadBtn', 'click', null, function onDownloadClick(evt) { + evt.preventDefault(); + $j.getJSON(thisUrl + '?request=modal&modal=download&eids[]='+eventData.Id) + .done(function(data) { + insertModalHtml('downloadModal', data.html); + $j('#downloadModal').modal('show'); + // Manage the GENERATE DOWNLOAD button + $j('#exportButton').click(exportEvent); + }) + .fail(logAjaxFail); + }); +*/ + // Manage the DELETE button + bindButton('#deleteBtn', 'click', null, function onDeleteClick(evt) { + if ( !canEdit.Events ) { + enoperm(); + return; + } + + evt.preventDefault(); + if ( ! $j('#deleteConfirm').length ) { + // Load the delete confirmation modal into the DOM + $j.getJSON(thisUrl + '?request=modal&modal=delconfirm') + .done(function(data) { + insertModalHtml('deleteConfirm', data.html); + manageDelConfirmModalBtns(); + $j('#deleteConfirm').modal('show'); + }) + .fail(logAjaxFail); + return; + } + $j('#deleteConfirm').modal('show'); + }); +} // end initPage + +// Kick everything off +$j(document).ready(initPage); diff --git a/web/skins/classic/views/js/snapshot.js.php b/web/skins/classic/views/js/snapshot.js.php new file mode 100644 index 000000000..fd97be727 --- /dev/null +++ b/web/skins/classic/views/js/snapshot.js.php @@ -0,0 +1,30 @@ + +var snapshot = ; + +var eventDataStrings = { + Id: '', + Name: '', + MonitorId: '', + MonitorName: '', + Cause: '', + StartDateTimeFmt: '', + Length: '', + Frames: '', + AlarmFrames: '', + TotScore: '', + AvgScore: '', + MaxScore: '', + DiskSpace: '', + Storage: '', + ArchivedStr: '', + EmailedStr: '' +}; + +// Strings +// +var deleteString = ""; +var causeString = ""; +var WEB_LIST_THUMB_WIDTH = ''; +var WEB_LIST_THUMB_HEIGHT = ''; diff --git a/web/skins/classic/views/js/snapshots.js b/web/skins/classic/views/js/snapshots.js new file mode 100644 index 000000000..235fd709f --- /dev/null +++ b/web/skins/classic/views/js/snapshots.js @@ -0,0 +1,312 @@ +var backBtn = $j('#backBtn'); +var exportBtn = $j('#exportBtn'); +var deleteBtn = $j('#deleteBtn'); +var table = $j('#snapshotTable'); + +/* +This is the format of the json object sent by bootstrap-table + +var params = +{ +"type":"get", +"data": + { + "search":"some search text", + "sort":"StartDateTime", + "order":"asc", + "offset":0, + "limit":25 + "filter": + { + "Name":"some advanced search text" + "StartDateTime":"some more advanced search text" + } + }, +"cache":true, +"contentType":"application/json", +"dataType":"json" +}; +*/ + +// Called by bootstrap-table to retrieve zm event data +function ajaxRequest(params) { + if ( params.data && params.data.filter ) { + params.data.advsearch = params.data.filter; + delete params.data.filter; + } + $j.getJSON(thisUrl + '?view=request&request=snapshots&task=query', params.data) + .done(function(data) { + var rows = processRows(data.rows); + // rearrange the result into what bootstrap-table expects + params.success({total: data.total, totalNotFiltered: data.totalNotFiltered, rows: rows}); + }) + .fail(function(jqXHR) { + logAjaxFail(jqXHR); + table.bootstrapTable('refresh'); + }); +} + +function processRows(rows) { + $j.each(rows, function(ndx, row) { + + var id = row.Id; + row.Id = '' + id + ''; + row.Name = '' + row.Name + ''; + row.Description = '' + row.Description + ''; + + //if ( WEB_LIST_THUMBS ) row.Thumbnail = '' + row.imgHtml + ''; + }); + + return rows; +} + +// Returns the event id's of the selected rows +function getIdSelections() { + var table = $j('#snapshotTable'); + + return $j.map(table.bootstrapTable('getSelections'), function(row) { + return row.Id.replace(/(<([^>]+)>)/gi, ''); // strip the html from the element before sending + }); +} + +// Returns a boolen to indicate at least one selected row is archived +function getArchivedSelections() { + var table = $j('#snapshotTable'); + var selection = $j.map(table.bootstrapTable('getSelections'), function(row) { + return row.Archived; + }); + return selection.includes("Yes"); +} + +// Load the Delete Confirmation Modal HTML via Ajax call +function getDelConfirmModal() { + $j.getJSON(thisUrl + '?request=modal&modal=delconfirm') + .done(function(data) { + insertModalHtml('deleteConfirm', data.html); + manageDelConfirmModalBtns(); + }) + .fail(logAjaxFail); +} + +// Manage the DELETE CONFIRMATION modal button +function manageDelConfirmModalBtns() { + document.getElementById("delConfirmBtn").addEventListener("click", function onDelConfirmClick(evt) { + if ( ! canEdit.Events ) { + enoperm(); + return; + } + + var selections = getIdSelections(); + + evt.preventDefault(); + $j.getJSON(thisUrl + '?request=events&task=delete&eids[]='+selections.join('&eids[]=')) + .done( function(data) { + $j('#snapshotTable').bootstrapTable('refresh'); + $j('#deleteConfirm').modal('hide'); + }) + .fail( function(jqxhr) { + logAjaxFail(jqxhr); + $j('#snapshotTable').bootstrapTable('refresh'); + $j('#deleteConfirm').modal('hide'); + }); + }); + + // Manage the CANCEL modal button + document.getElementById("delCancelBtn").addEventListener("click", function onDelCancelClick(evt) { + $j('#deleteConfirm').modal('hide'); + }); +} + +function getEventDetailModal(eid) { + $j.getJSON(thisUrl + '?request=modal&modal=eventdetail&eids[]=' + eid) + .done(function(data) { + insertModalHtml('eventDetailModal', data.html); + $j('#eventDetailModal').modal('show'); + // Manage the Save button + $j('#eventDetailSaveBtn').click(function(evt) { + evt.preventDefault(); + $j('#eventDetailForm').submit(); + }); + }) + .fail(logAjaxFail); +} + +function getObjdetectModal(eid) { + $j.getJSON(thisUrl + '?request=modal&modal=objdetect&eid=' + eid) + .done(function(data) { + insertModalHtml('objdetectModal', data.html); + $j('#objdetectModal').modal('show'); + }) + .fail(logAjaxFail); +} + +function initPage() { + // Remove the thumbnail column from the DOM if thumbnails are off globally + if ( !WEB_LIST_THUMBS ) $j('th[data-field="Thumbnail"]').remove(); + + // Load the delete confirmation modal into the DOM + getDelConfirmModal(); + + // Init the bootstrap-table + table.bootstrapTable({icons: icons}); + + // Hide these columns on first run when no cookie is saved + if ( !getCookie("zmEventsTable.bs.table.columns") ) { + table.bootstrapTable('hideColumn', 'Archived'); + table.bootstrapTable('hideColumn', 'Emailed'); + } + + // enable or disable buttons based on current selection and user rights + table.on('check.bs.table uncheck.bs.table ' + + 'check-all.bs.table uncheck-all.bs.table', + function() { + selections = table.bootstrapTable('getSelections'); + + viewBtn.prop('disabled', !(selections.length && canView.Events)); + archiveBtn.prop('disabled', !(selections.length && canEdit.Events)); + unarchiveBtn.prop('disabled', !(getArchivedSelections()) && canEdit.Events); + editBtn.prop('disabled', !(selections.length && canEdit.Events)); + exportBtn.prop('disabled', !(selections.length && canView.Events)); + downloadBtn.prop('disabled', !(selections.length && canView.Events)); + deleteBtn.prop('disabled', !(selections.length && canEdit.Events)); + }); + + // Don't enable the back button if there is no previous zm page to go back to + backBtn.prop('disabled', !document.referrer.length); + + // Setup the thumbnail video animation + initThumbAnimation(); + + // Some toolbar events break the thumbnail animation, so re-init eventlistener + table.on('all.bs.table', initThumbAnimation); + + // Manage the BACK button + document.getElementById("backBtn").addEventListener("click", function onBackClick(evt) { + evt.preventDefault(); + window.history.back(); + }); + + // Manage the REFRESH Button + document.getElementById("refreshBtn").addEventListener("click", function onRefreshClick(evt) { + evt.preventDefault(); + window.location.reload(true); + }); + +/* + // Manage the ARCHIVE button + document.getElementById("archiveBtn").addEventListener("click", function onArchiveClick(evt) { + var selections = getIdSelections(); + + evt.preventDefault(); + $j.getJSON(thisUrl + '?request=events&task=archive&eids[]='+selections.join('&eids[]=')) + .done( function(data) { + $j('#snapshotTable').bootstrapTable('refresh'); + }) + .fail(logAjaxFail); + }); + + // Manage the UNARCHIVE button + document.getElementById("unarchiveBtn").addEventListener("click", function onUnarchiveClick(evt) { + if ( ! canEdit.Events ) { + enoperm(); + return; + } + + var selections = getIdSelections(); + //console.log(selections); + + evt.preventDefault(); + $j.getJSON(thisUrl + '?request=events&task=unarchive&eids[]='+selections.join('&eids[]=')) + .done( function(data) { + $j('#snapshotTable').bootstrapTable('refresh'); + }) + .fail(logAjaxFail); + }); + + // Manage the EDIT button + document.getElementById("editBtn").addEventListener("click", function onEditClick(evt) { + if ( ! canEdit.Events ) { + enoperm(); + return; + } + + var selections = getIdSelections(); + + evt.preventDefault(); + $j.getJSON(thisUrl + '?request=modal&modal=eventdetail&eids[]='+selections.join('&eids[]=')) + .done(function(data) { + insertModalHtml('eventDetailModal', data.html); + $j('#eventDetailModal').modal('show'); + // Manage the Save button + $j('#eventDetailSaveBtn').click(function(evt) { + evt.preventDefault(); + $j('#eventDetailForm').submit(); + }); + }) + .fail(logAjaxFail); + }); + + // Manage the EXPORT button + document.getElementById("exportBtn").addEventListener("click", function onExportClick(evt) { + var selections = getIdSelections(); + + evt.preventDefault(); + window.location.assign('?view=export&eids[]='+selections.join('&eids[]=')); + }); + + // Manage the DOWNLOAD VIDEO button + document.getElementById("downloadBtn").addEventListener("click", function onDownloadClick(evt) { + var selections = getIdSelections(); + + evt.preventDefault(); + $j.getJSON(thisUrl + '?request=modal&modal=download&eids[]='+selections.join('&eids[]=')) + .done(function(data) { + insertModalHtml('downloadModal', data.html); + $j('#downloadModal').modal('show'); + // Manage the GENERATE DOWNLOAD button + $j('#exportButton').click(exportEvent); + }) + .fail(logAjaxFail); + }); +*/ + // Manage the DELETE button + document.getElementById("deleteBtn").addEventListener("click", function onDeleteClick(evt) { + if ( ! canEdit.Events ) { + enoperm(); + return; + } + + evt.preventDefault(); + $j('#deleteConfirm').modal('show'); + }); + + // Update table links each time after new data is loaded + table.on('post-body.bs.table', function(data) { + // Manage the Object Detection links in the events list + $j(".objDetectLink").click(function(evt) { + evt.preventDefault(); + var eid = $j(this).data('eid'); + getObjdetectModal(eid); + }); + + // Manage the eventdetail links in the events list + $j(".eDetailLink").click(function(evt) { + evt.preventDefault(); + var eid = $j(this).data('eid'); + getEventDetailModal(eid); + }); + + var thumb_ndx = $j('#snapshotTable tr th').filter(function() { + return $j(this).text().trim() == 'Thumbnail'; + }).index(); + table.find("tr td:nth-child(" + (thumb_ndx+1) + ")").addClass('colThumbnail'); + }); + + table.bootstrapTable('resetSearch'); + // The table is initially given a hidden style, so now that we are done rendering, show it + table.show(); +} + +$j(document).ready(function() { + initPage(); +}); diff --git a/web/skins/classic/views/js/snapshots.js.php b/web/skins/classic/views/js/snapshots.js.php new file mode 100644 index 000000000..487d9cd5d --- /dev/null +++ b/web/skins/classic/views/js/snapshots.js.php @@ -0,0 +1,6 @@ +var confirmDeleteEventsString = ""; +var archivedString = ""; +var emailedString = ""; +var yesString = ""; +var noString = ""; +var WEB_LIST_THUMBS = ; diff --git a/web/skins/classic/views/snapshot.php b/web/skins/classic/views/snapshot.php new file mode 100644 index 000000000..042d4fe41 --- /dev/null +++ b/web/skins/classic/views/snapshot.php @@ -0,0 +1,98 @@ +Id()); +?> + +
+ +Id() ) { + echo '
Snapshot was not found.
'; +} +?> + +
+ +
+
+ + +Id() ) { ?> + + + + +Id ?> +
+ +

Id() ?>

+
+
+ +
+ CreatedOn() ?> +
+
+ +
+
+ +
+
+Id() ) { ?> + +
+Events(); + $width = 100 / ( ( count($events) < 4 ) ? count($events) : 4 ) -1; + foreach ( $snapshot->Events() as $event ) { + $imgSrc = $event->getThumbnailSrc(array(), '&'); + echo ''; + } +?> +
+Id() ?> +
+
+ diff --git a/web/skins/classic/views/snapshots.php b/web/skins/classic/views/snapshots.php new file mode 100644 index 000000000..4fefc2b98 --- /dev/null +++ b/web/skins/classic/views/snapshots.php @@ -0,0 +1,92 @@ + + +
+ +
+ + + + + +
+ + +
+ + + + + + + + + + + + + + + + + + +
+
+ From 527defc0c8dcb789e5aed2268022e5bd2d0c0f84 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 12 Mar 2021 09:28:47 -0500 Subject: [PATCH 0166/1277] Bump version to 1.35.21 for Snapshots --- distros/redhat/zoneminder.spec | 2 +- version | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/distros/redhat/zoneminder.spec b/distros/redhat/zoneminder.spec index 9211793e9..dc6120dc1 100644 --- a/distros/redhat/zoneminder.spec +++ b/distros/redhat/zoneminder.spec @@ -28,7 +28,7 @@ %global _hardened_build 1 Name: zoneminder -Version: 1.35.20 +Version: 1.35.21 Release: 1%{?dist} Summary: A camera monitoring and analysis tool Group: System Environment/Daemons diff --git a/version b/version index a3dc39092..f4b991b78 100644 --- a/version +++ b/version @@ -1 +1 @@ -1.35.20 +1.35.21 From 70073ea017969b343a439abdaf43816c7fcc57b4 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 12 Mar 2021 10:07:17 -0500 Subject: [PATCH 0167/1277] Add thumbnails to snapshots list --- web/ajax/snapshots.php | 14 ++++++++------ web/skins/classic/views/js/snapshots.js | 2 +- web/skins/classic/views/snapshots.php | 2 +- 3 files changed, 10 insertions(+), 8 deletions(-) diff --git a/web/ajax/snapshots.php b/web/ajax/snapshots.php index e3da25ae3..bfe6b536d 100644 --- a/web/ajax/snapshots.php +++ b/web/ajax/snapshots.php @@ -190,13 +190,14 @@ function queryRequest($search, $advsearch, $sort, $offset, $order, $limit) { foreach ( array_slice($filtered_rows, $offset, $limit) as $row ) { $snapshot = new ZM\Snapshot($row); - //$scale = intval(5*100*ZM_WEB_LIST_THUMB_WIDTH / $event->Width()); - //$imgSrc = $event->getThumbnailSrc(array(), '&'); - //$streamSrc = $event->getStreamSrc(array( - //'mode'=>'jpeg', 'scale'=>$scale, 'maxfps'=>ZM_WEB_VIDEO_MAXFPS, 'replay'=>'single', 'rate'=>'400'), '&'); - + $row['imgHtml'] = ''; // Modify the row data as needed - //$row['imgHtml'] = 'Event '.$event->Id().''; + foreach ( $snapshot->Events() as $event ) { + $scale = intval(5*100*ZM_WEB_LIST_THUMB_WIDTH / $event->Width()); + $imgSrc = $event->getThumbnailSrc(array(), '&'); + + $row['imgHtml'] .= 'Event '.$event->Id().''; + } $row['Name'] = validHtmlStr($row['Name']); $row['Description'] = validHtmlStr($row['Description']); //$row['Archived'] = $row['Archived'] ? translate('Yes') : translate('No'); @@ -209,6 +210,7 @@ function queryRequest($search, $advsearch, $sort, $offset, $order, $limit) { //$row['Storage'] = ( $row['StorageId'] and isset($StorageById[$row['StorageId']]) ) ? $StorageById[$row['StorageId']]->Name() : 'Default'; //$row['Notes'] = nl2br(htmlspecialchars($row['Notes'])); //$row['DiskSpace'] = human_filesize($event->DiskSpace()); + // $returned_rows[] = $row; } # end foreach row matching search diff --git a/web/skins/classic/views/js/snapshots.js b/web/skins/classic/views/js/snapshots.js index 235fd709f..dff843374 100644 --- a/web/skins/classic/views/js/snapshots.js +++ b/web/skins/classic/views/js/snapshots.js @@ -54,7 +54,7 @@ function processRows(rows) { row.Name = '' + row.Name + ''; row.Description = '' + row.Description + ''; - //if ( WEB_LIST_THUMBS ) row.Thumbnail = '' + row.imgHtml + ''; + if (WEB_LIST_THUMBS) row.Thumbnail = '' + row.imgHtml + ''; }); return rows; diff --git a/web/skins/classic/views/snapshots.php b/web/skins/classic/views/snapshots.php index 4fefc2b98..781aff967 100644 --- a/web/skins/classic/views/snapshots.php +++ b/web/skins/classic/views/snapshots.php @@ -77,7 +77,7 @@ getBodyTopHTML(); - + From ba00bee4b898310679f166e7315120be3a1fa08b Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 12 Mar 2021 10:07:33 -0500 Subject: [PATCH 0168/1277] fix header alignment --- web/skins/classic/css/base/views/snapshots.css | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 web/skins/classic/css/base/views/snapshots.css diff --git a/web/skins/classic/css/base/views/snapshots.css b/web/skins/classic/css/base/views/snapshots.css new file mode 100644 index 000000000..c264a5e9c --- /dev/null +++ b/web/skins/classic/css/base/views/snapshots.css @@ -0,0 +1,6 @@ +th[data-field="Id"], +th[data-field="Description"], +th[data-field="CreatedOn"], +th[data-field="Name"] { + text-align: left; +} From 1ae1a89abfdea2ef4d37c0e0448b2461f438a329 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 12 Mar 2021 10:29:13 -0500 Subject: [PATCH 0169/1277] Only allow show zones for System::View permission --- web/skins/classic/views/montage.php | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/web/skins/classic/views/montage.php b/web/skins/classic/views/montage.php index fa7c58ac3..3f492dc1c 100644 --- a/web/skins/classic/views/montage.php +++ b/web/skins/classic/views/montage.php @@ -154,14 +154,16 @@ echo getNavBarHTML(); if ( $showControl ) { echo makeLink('?view=control', translate('Control')); } -if ( $showZones ) { -?> - - - - + + + +
From 9504d8be612fc07f0d4475a6445bc980b2a6c770 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 12 Mar 2021 10:40:23 -0500 Subject: [PATCH 0170/1277] event_ids => snapshot_ids --- web/ajax/snapshots.php | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/web/ajax/snapshots.php b/web/ajax/snapshots.php index bfe6b536d..d2fb2e327 100644 --- a/web/ajax/snapshots.php +++ b/web/ajax/snapshots.php @@ -133,7 +133,7 @@ function queryRequest($search, $advsearch, $sort, $offset, $order, $limit) { $sql = 'SELECT ' .$col_str. ' FROM `Snapshots`'.$where.' ORDER BY '.$sort.' '.$order; $unfiltered_rows = array(); - $event_ids = array(); + $snapshot_ids = array(); ZM\Debug('Calling the following sql query: ' .$sql); $query = dbQuery($sql, $values); @@ -153,7 +153,8 @@ function queryRequest($search, $advsearch, $sort, $offset, $order, $limit) { if ( count($advsearch) or $search != '' ) { $search_filter = new ZM\Filter(); - $search_filter = $search_filter->addTerm(array('cnj'=>'and', 'attr'=>'Id', 'op'=>'IN', 'val'=>$event_ids)); + if (count($snapshot_ids)) + $search_filter = $search_filter->addTerm(array('cnj'=>'and', 'attr'=>'Id', 'op'=>'IN', 'val'=>$snapshot_ids)); // There are two search bars in the log view, normal and advanced // Making an exuctive decision to ignore the normal search, when advanced search is in use From b50916e02a4caf4a0f37e305d6116146fb38b5e8 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 12 Mar 2021 10:52:10 -0500 Subject: [PATCH 0171/1277] Allow unknown columns in filters. Allow specifying the table name in FilterTerm. --- web/includes/Filter.php | 2 +- web/includes/FilterTerm.php | 10 +++++++++- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/web/includes/Filter.php b/web/includes/Filter.php index b8e2d4088..3578a1d62 100644 --- a/web/includes/Filter.php +++ b/web/includes/Filter.php @@ -635,7 +635,7 @@ class Filter extends ZM_Object { if ( !FilterTerm::is_valid_attr($term['attr']) ) { Error('Unsupported filter attribute ' . $term['attr']); - return $this; + //return $this; } $terms = $this->terms(); diff --git a/web/includes/FilterTerm.php b/web/includes/FilterTerm.php index c6c3cf312..40937b943 100644 --- a/web/includes/FilterTerm.php +++ b/web/includes/FilterTerm.php @@ -41,6 +41,11 @@ class FilterTerm { Warning('Invalid cnj ' . $term['cnj'].' in '.print_r($term, true)); } } + if ( isset($term['tablename']) ) { + $this->tablename = $term['tablename']; + } else { + $this->tablename = 'E'; + } if ( isset($term['obr']) ) { if ( (string)(int)$term['obr'] == $term['obr'] ) { @@ -288,7 +293,10 @@ class FilterTerm { case 'Notes': case 'StateId': case 'Archived': - $sql .= 'E.'.$this->attr; + $sql .= $this->tablename.'.'.$this->attr; + break; + default : + $sql .= $this->tablename.'.'.$this->attr; } $sql .= $this->sql_operator(); $values = $this->sql_values(); From 62cfdd8d61e156022ce6676f09f52d2952a00c39 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 12 Mar 2021 10:52:23 -0500 Subject: [PATCH 0172/1277] fix searching in snapshots --- web/ajax/snapshots.php | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/web/ajax/snapshots.php b/web/ajax/snapshots.php index d2fb2e327..6669ad0f9 100644 --- a/web/ajax/snapshots.php +++ b/web/ajax/snapshots.php @@ -118,7 +118,7 @@ function queryRequest($search, $advsearch, $sort, $offset, $order, $limit) { $table = 'Snapshots'; // The names of the dB columns in the events table we are interested in - $columns = array('Id', 'Name', 'Description', 'CreatedDateTime'); + $columns = array('Id', 'Name', 'Description', 'CreatedOn'); if ( !in_array($sort, array_merge($columns)) ) { ZM\Error('Invalid sort field: ' . $sort); @@ -154,7 +154,7 @@ function queryRequest($search, $advsearch, $sort, $offset, $order, $limit) { $search_filter = new ZM\Filter(); if (count($snapshot_ids)) - $search_filter = $search_filter->addTerm(array('cnj'=>'and', 'attr'=>'Id', 'op'=>'IN', 'val'=>$snapshot_ids)); + $search_filter = $search_filter->addTerm(array('cnj'=>'and', 'attr'=>'Id', 'op'=>'IN', 'val'=>$snapshot_ids, 'tablename'=>'Snapshots')); // There are two search bars in the log view, normal and advanced // Making an exuctive decision to ignore the normal search, when advanced search is in use @@ -162,7 +162,7 @@ function queryRequest($search, $advsearch, $sort, $offset, $order, $limit) { if ( count($advsearch) ) { $terms = array(); foreach ( $advsearch as $col=>$text ) { - $terms[] = array('cnj'=>'and', 'attr'=>$col, 'op'=>'LIKE', 'val'=>$text); + $terms[] = array('cnj'=>'and', 'attr'=>$col, 'op'=>'LIKE', 'val'=>$text, 'tablename'=>'Snapshots'); } # end foreach col in advsearch $terms[0]['obr'] = 1; $terms[count($terms)-1]['cbr'] = 1; @@ -171,7 +171,7 @@ function queryRequest($search, $advsearch, $sort, $offset, $order, $limit) { $search = '%' .$search. '%'; $terms = array(); foreach ( $columns as $col ) { - $terms[] = array('cnj'=>'or', 'attr'=>$col, 'op'=>'LIKE', 'val'=>$search); + $terms[] = array('cnj'=>'or', 'attr'=>$col, 'op'=>'LIKE', 'val'=>$search, 'tablename'=>'Snapshots'); } $terms[0]['obr'] = 1; $terms[0]['cnj'] = 'and'; From e5f9654e504028cf3eaeca6cbb300c3ce603c3fa Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 12 Mar 2021 11:33:16 -0500 Subject: [PATCH 0173/1277] honour Groups:View permission in monitor filters --- web/skins/classic/views/_monitor_filters.php | 28 +++++++++++--------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/web/skins/classic/views/_monitor_filters.php b/web/skins/classic/views/_monitor_filters.php index 2acc26fc0..3c7165ebe 100644 --- a/web/skins/classic/views/_monitor_filters.php +++ b/web/skins/classic/views/_monitor_filters.php @@ -53,20 +53,22 @@ $html = '; -$GroupsById = array(); -foreach ( ZM\Group::find() as $G ) { - $GroupsById[$G->Id()] = $G; -} - $groupSql = ''; -if ( count($GroupsById) ) { - $html .= ''; - # This will end up with the group_id of the deepest selection - $group_id = isset($_SESSION['GroupId']) ? $_SESSION['GroupId'] : null; - $html .= ZM\Group::get_group_dropdown(); - $groupSql = ZM\Group::get_group_sql($group_id); - $html .= ' -'; +if ( canView('Groups') ) { + $GroupsById = array(); + foreach ( ZM\Group::find() as $G ) { + $GroupsById[$G->Id()] = $G; + } + + if ( count($GroupsById) ) { + $html .= ''; + # This will end up with the group_id of the deepest selection + $group_id = isset($_SESSION['GroupId']) ? $_SESSION['GroupId'] : null; + $html .= ZM\Group::get_group_dropdown(); + $groupSql = ZM\Group::get_group_sql($group_id); + $html .= ' + '; + } } $selected_monitor_ids = isset($_SESSION['MonitorId']) ? $_SESSION['MonitorId'] : array(); From 9d33688f89508eb323e4db73ed0e247a966a3fdd Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Fri, 12 Mar 2021 11:40:46 -0500 Subject: [PATCH 0174/1277] Honour various permissions on what's in the navbar. --- web/skins/classic/includes/functions.php | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/web/skins/classic/includes/functions.php b/web/skins/classic/includes/functions.php index aedb6b68c..1e490fd46 100644 --- a/web/skins/classic/includes/functions.php +++ b/web/skins/classic/includes/functions.php @@ -382,6 +382,7 @@ function getCollapsedNavBarHTML($running, $user, $bandwidth_options, $view, $ski // Returns the html representing the current unix style system load function getSysLoadHTML() { $result = ''; + if ( !canView('System') ) return $result; $result .= ''.PHP_EOL; @@ -650,6 +655,7 @@ function getGroupsHTML($view) { // Returns the html representing the Filter menu item function getFilterHTML($view) { $result = ''; + if ( !canView('Events') ) return $result; $class = $view == 'filter' ? ' selected' : ''; $result .= ''.PHP_EOL; From 587cebecbb11c38cc793373a9ca55b209d12f10b Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sat, 13 Mar 2021 12:10:35 -0500 Subject: [PATCH 0175/1277] navbar requests don't pass an auth token so we never send an auth update. Just always send it. --- web/ajax/status.php | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/web/ajax/status.php b/web/ajax/status.php index da711b1f8..9d14dd24d 100644 --- a/web/ajax/status.php +++ b/web/ajax/status.php @@ -4,10 +4,8 @@ if ( $_REQUEST['entity'] == 'navBar' ) { $data = array(); if ( ZM_OPT_USE_AUTH && (ZM_AUTH_RELAY == 'hashed') ) { $auth_hash = generateAuthHash(ZM_AUTH_HASH_IPS); - if ( isset($_REQUEST['auth']) and ($_REQUEST['auth'] != $auth_hash) ) { - $data['auth'] = $auth_hash; - $data['auth_relay'] = get_auth_relay(); - } + $data['auth'] = $auth_hash; + $data['auth_relay'] = get_auth_relay(); } // Each widget on the navbar has its own function // Call the functions we want to dynamically update From 5eae1c0a9f4f4a86f087f46bc8ccaa0db25ca2e6 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sat, 13 Mar 2021 12:10:55 -0500 Subject: [PATCH 0176/1277] spacing, add code comment --- web/includes/actions/monitor.php | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/web/includes/actions/monitor.php b/web/includes/actions/monitor.php index 0a04fdbeb..762a98729 100644 --- a/web/includes/actions/monitor.php +++ b/web/includes/actions/monitor.php @@ -19,7 +19,7 @@ // // Monitor edit actions, monitor id derived, require edit permissions for that monitor -if ( ! canEdit('Monitors') ) { +if ( !canEdit('Monitors') ) { ZM\Warning('Monitor actions require Monitors Permissions'); return; } @@ -34,8 +34,7 @@ if ( $action == 'save' ) { } if ( ZM_OPT_X10 ) { $x10Monitor = dbFetchOne('SELECT * FROM TriggersX10 WHERE MonitorId=?', NULL, array($mid)); - if ( !$x10Monitor ) - $x10Monitor = array(); + if ( !$x10Monitor ) $x10Monitor = array(); } } else { if ( $user['MonitorIds'] ) { @@ -88,6 +87,7 @@ if ( $action == 'save' ) { $restart = false; if ( count($changes) ) { + // monitor->Id() has a value when the db record exists if ( $monitor->Id() ) { # If we change anything that changes the shared mem size, zma can complain. So let's stop first. @@ -212,9 +212,7 @@ if ( $action == 'save' ) { # FIXME This is actually a race condition. Should lock the table. $maxSeq = dbFetchOne('SELECT MAX(Sequence) AS MaxSequence FROM Monitors', 'MaxSequence'); $changes['Sequence'] = $maxSeq+1; - if ( $mid and !$monitor->Id() ) { - $changes['Id'] = $mid; - } + if ( $mid ) $changes['Id'] = $mid; # mid specified in request, doesn't exist in db, will re-use slot if ( $monitor->insert($changes) ) { $mid = $monitor->Id(); From 0bb4afa0bcad91f3583858ee0400eee95651bf7c Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sat, 13 Mar 2021 12:11:55 -0500 Subject: [PATCH 0177/1277] The test for xmlhttprequest is bogus. chrome jquery JSON requests don't send it. Replace with a test for instead. So now only redirect on proper html views. --- web/index.php | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) diff --git a/web/index.php b/web/index.php index c4a44afbf..bf71b059c 100644 --- a/web/index.php +++ b/web/index.php @@ -249,21 +249,16 @@ if ( $action and !$request ) { # If I put this here, it protects all views and popups, but it has to go after actions.php because actions.php does the actual logging in. if ( ZM_OPT_USE_AUTH and (!isset($user)) and ($view != 'login') and ($view != 'none') ) { - /* AJAX check */ - if ( !empty($_SERVER['HTTP_X_REQUESTED_WITH']) - && strtolower($_SERVER['HTTP_X_REQUESTED_WITH']) == 'xmlhttprequest' ) { + if ($request) { + # requests only return json header('HTTP/1.1 401 Unauthorized'); exit; } - ZM\Debug('Redirecting to login'); $view = 'none'; $redirect = ZM_BASE_URL.$_SERVER['PHP_SELF'].'?view=login'; - if ( ! $request ) { - zm_session_start(); - $_SESSION['postLoginQuery'] = $_SERVER['QUERY_STRING']; - session_write_close(); - } - $request = null; + zm_session_start(); + $_SESSION['postLoginQuery'] = $_SERVER['QUERY_STRING']; + session_write_close(); } else if ( ZM_SHOW_PRIVACY && ($view != 'privacy') && ($view != 'options') && (!$request) && canEdit('System') ) { $view = 'none'; $redirect = ZM_BASE_URL.$_SERVER['PHP_SELF'].'?view=privacy'; From 478d11e6f2785d19b2012dc633c20e1c7d8faea1 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sat, 13 Mar 2021 12:12:20 -0500 Subject: [PATCH 0178/1277] Update the auth in streamCmdParams on every request. It wasn't picking up updates --- web/js/MonitorStream.js | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/web/js/MonitorStream.js b/web/js/MonitorStream.js index 0dc19f2d6..bcf9321b2 100644 --- a/web/js/MonitorStream.js +++ b/web/js/MonitorStream.js @@ -13,11 +13,6 @@ function MonitorStream(monitorData) { request: 'stream', connkey: this.connKey }; - if ( auth_hash ) { - this.streamCmdParms.auth = auth_hash; - } else if ( auth_relay ) { - this.streamCmdParms.auth_relay = ''; - } this.type = monitorData.type; this.refresh = monitorData.refresh; this.start = function(delay) { @@ -208,6 +203,11 @@ function MonitorStream(monitorData) { if ( this.type != 'WebSite' ) { this.streamCmdReq = function(streamCmdParms) { + if ( auth_hash ) { + this.streamCmdParms.auth = auth_hash; + } else if ( auth_relay ) { + this.streamCmdParms.auth_relay = ''; + } $j.ajaxSetup({timeout: AJAX_TIMEOUT}); $j.getJSON(this.url, streamCmdParms) .done(this.getStreamCmdResponse.bind(this)) From 2a6621959c3f6ce5d78cd3687eaa46a4187e93e4 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sat, 13 Mar 2021 12:47:05 -0500 Subject: [PATCH 0179/1277] We CANNOT specify nextId for the next mid to use. If we do that then we can overwrite existing db entries --- web/skins/classic/views/monitor.php | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/web/skins/classic/views/monitor.php b/web/skins/classic/views/monitor.php index 1806a1745..a2402b8bc 100644 --- a/web/skins/classic/views/monitor.php +++ b/web/skins/classic/views/monitor.php @@ -33,13 +33,14 @@ if ( defined('ZM_SERVER_ID') ) { if ( !$Server ) { $Server = array('Id' => ''); } - +$mid = null; $monitor = null; if ( !empty($_REQUEST['mid']) ) { - $nextId = validInt($_REQUEST['mid']); - $monitor = new ZM\Monitor($_REQUEST['mid']); + $mid = validInt($_REQUEST['mid']); + + $monitor = new ZM\Monitor($mid); if ( $monitor and ZM_OPT_X10 ) - $x10Monitor = dbFetchOne('SELECT * FROM TriggersX10 WHERE MonitorId = ?', NULL, array($_REQUEST['mid'])); + $x10Monitor = dbFetchOne('SELECT * FROM TriggersX10 WHERE MonitorId = ?', NULL, array($mid)); } if ( !$monitor ) { @@ -56,7 +57,7 @@ if ( isset($_REQUEST['dupId']) ) { $x10Monitor = dbFetchOne('SELECT * FROM TriggersX10 WHERE MonitorId = ?', NULL, array($_REQUEST['dupId'])); $clonedName = $monitor->Name(); $monitor->Name('Clone of '.$monitor->Name()); - $monitor->Id(!empty($_REQUEST['mid'])?validInt($_REQUEST['mid']) : 0); + $monitor->Id($mid); } if ( ZM_OPT_X10 && empty($x10Monitor) ) { @@ -467,7 +468,7 @@ if ( canEdit('Monitors') ) {
- +
Date: Sat, 13 Mar 2021 16:30:41 -0500 Subject: [PATCH 0180/1277] remove deprecated reset and tidy up a bit --- src/zm_packet.cpp | 71 ++++------------------------------------------- src/zm_packet.h | 1 - 2 files changed, 6 insertions(+), 66 deletions(-) diff --git a/src/zm_packet.cpp b/src/zm_packet.cpp index d6174cee8..58c2bda8c 100644 --- a/src/zm_packet.cpp +++ b/src/zm_packet.cpp @@ -68,71 +68,12 @@ ZMPacket::ZMPacket(ZMPacket &p) : ZMPacket::~ZMPacket() { zm_av_packet_unref(&packet); - if ( in_frame ) { - av_frame_free(&in_frame); - } - if ( out_frame ) { - av_frame_free(&out_frame); - } - if ( buffer ) { - av_freep(&buffer); - } - if ( analysis_image ) { - delete analysis_image; - analysis_image = nullptr; - } - if ( image ) { - delete image; - image = nullptr; - } - if ( timestamp ) { - delete timestamp; - timestamp = nullptr; - } - -#if 0 - if ( image ) { - if ( image->IsBufferHeld() ) { - // Don't free the mmap'd image - } else { - delete image; - image = nullptr; - delete timestamp; - timestamp = nullptr; - } - } else { - if ( timestamp ) { - delete timestamp; - timestamp = nullptr; - } - } -#endif -} - -// deprecated -void ZMPacket::reset() { - zm_av_packet_unref(&packet); - if ( in_frame ) { - av_frame_free(&in_frame); - } - if ( out_frame ) { - av_frame_free(&out_frame); - } - if ( buffer ) { - av_freep(&buffer); - } - if ( analysis_image ) { - delete analysis_image; - analysis_image = nullptr; - } -#if 0 - if ( (! image) && timestamp ) { - delete timestamp; - timestamp = NULL; - } -#endif - score = -1; - keyframe = 0; + if ( in_frame ) av_frame_free(&in_frame); + if ( out_frame ) av_frame_free(&out_frame); + if ( buffer ) av_freep(&buffer); + if ( analysis_image ) delete analysis_image; + if ( image ) delete image; + if ( timestamp ) delete timestamp; } /* returns < 0 on error, 0 on not ready, int bytes consumed on success diff --git a/src/zm_packet.h b/src/zm_packet.h index abd0f3486..118239860 100644 --- a/src/zm_packet.h +++ b/src/zm_packet.h @@ -61,7 +61,6 @@ class ZMPacket { int is_keyframe() { return keyframe; }; int decode( AVCodecContext *ctx ); - void reset(); explicit ZMPacket(Image *image); explicit ZMPacket(ZMPacket &packet); ZMPacket(); From 7db38954584d3766eea4082f7c19b4dd3a3653b4 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sun, 14 Mar 2021 20:11:13 -0400 Subject: [PATCH 0181/1277] Fix default user creation after we added HomeView column --- db/zm_create.sql.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/db/zm_create.sql.in b/db/zm_create.sql.in index dec8814d2..0e8ac7f17 100644 --- a/db/zm_create.sql.in +++ b/db/zm_create.sql.in @@ -780,7 +780,7 @@ insert into Storage VALUES (NULL, '@ZM_DIR_EVENTS@', 'Default', 'local', NULL, N -- -- Create a default admin user. -- -insert into Users VALUES (NULL,'admin','$2b$12$NHZsm6AM2f2LQVROriz79ul3D6DnmFiZC.ZK5eqbF.ZWfwH9bqUJ6','',1,'View','Edit','Edit','Edit','Edit','Edit','Edit','','',0,1); +insert into Users VALUES (NULL,'admin','$2b$12$NHZsm6AM2f2LQVROriz79ul3D6DnmFiZC.ZK5eqbF.ZWfwH9bqUJ6','',1,'View','Edit','Edit','Edit','Edit','Edit','Edit','','',0,1,''); -- -- Add a sample filter to purge the oldest 100 events when the disk is 95% full From f4e60be444acedb490ae92e4ba86e1412487f2c1 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sun, 14 Mar 2021 22:08:48 -0400 Subject: [PATCH 0182/1277] If we failed to bind libvnc then fail Prime to prevent crash --- src/zm_libvnc_camera.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/zm_libvnc_camera.cpp b/src/zm_libvnc_camera.cpp index 7e1d260ab..8ca144913 100644 --- a/src/zm_libvnc_camera.cpp +++ b/src/zm_libvnc_camera.cpp @@ -144,6 +144,10 @@ VncCamera::~VncCamera() { } int VncCamera::PrimeCapture() { + if (libvnc_lib != nullptr) { + Error("No libvnc shared lib bound."); + return -1; + } Debug(1, "Priming capture from %s", mHost.c_str()); if (!mRfb) { From feec631ca50d42829be6c0d8bbba5bc7f2e21ea7 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 15 Mar 2021 15:02:43 -0400 Subject: [PATCH 0183/1277] Only save updated DiskSpace if event is finished --- web/views/image.php | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/web/views/image.php b/web/views/image.php index 53d91f3b6..2852b3f7b 100644 --- a/web/views/image.php +++ b/web/views/image.php @@ -196,8 +196,10 @@ if ( empty($_REQUEST['path']) ) { Output was: '.implode(PHP_EOL,$output) ); } # Generating an image file will use up more disk space, so update the Event record. - $Event->DiskSpace(null); - $Event->save(); + if ( $Event->EndDateTime() ) { + $Event->DiskSpace(null); + $Event->save(); + } } else { header('HTTP/1.0 404 Not Found'); ZM\Fatal('No snapshot jpg found for event '.$_REQUEST['eid']); @@ -281,8 +283,10 @@ Command was: '.$command.' Output was: '.implode(PHP_EOL,$output) ); } # Generating an image file will use up more disk space, so update the Event record. - $Event->DiskSpace(null); - $Event->save(); + if ( $Event->EndDateTime() ) { + $Event->DiskSpace(null); + $Event->save(); + } } else { header('HTTP/1.0 404 Not Found'); ZM\Fatal("Can't create frame $show images from video because there is no video file for this event at ". From b238a2bc25389fd826bbd295e0a817cda2047e10 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 15 Mar 2021 15:08:43 -0400 Subject: [PATCH 0184/1277] Implement decoder thread, locking is broken --- src/CMakeLists.txt | 1 + src/zm_monitor.cpp | 177 +++++++++++++++++++++++++++++++---------- src/zm_monitor.h | 4 + src/zm_packet.cpp | 4 +- src/zm_packet.h | 49 +++++++++--- src/zm_packetqueue.cpp | 12 +-- 6 files changed, 185 insertions(+), 62 deletions(-) diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index c6a9c04c7..37b08ad58 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -16,6 +16,7 @@ set(ZM_BIN_SRC_FILES zm_crypt.cpp zm.cpp zm_db.cpp + zm_decoder_thread.cpp zm_logger.cpp zm_event.cpp zm_eventstream.cpp diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 88028ca32..73822502e 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -395,6 +395,8 @@ Monitor::Monitor() storage(nullptr), videoStore(nullptr), analysis_it(nullptr), + decoder_it(nullptr), + decoder(nullptr), n_zones(0), zones(nullptr), privacy_bitmask(nullptr), @@ -1802,21 +1804,20 @@ void Monitor::UpdateAnalysisFPS() { // If there isn't then we keep pre-event + alarm frames. = pre_event_count bool Monitor::Analyse() { - if ( !Enabled() ) { + if (!Enabled()) { Warning("Shouldn't be doing Analyse when not Enabled"); return false; } - if ( !analysis_it ) - analysis_it = packetqueue.get_video_it(true); + if (!analysis_it) analysis_it = packetqueue.get_video_it(true); // if have event, send frames until we find a video packet, at which point do analysis. Adaptive skip should only affect which frames we do analysis on. // get_analysis_packet will lock the packet and may wait if analysis_it is at the end ZMPacket *snap = packetqueue.get_packet(analysis_it); - if ( !snap ) return false; + if (!snap) return false; // Is it possible for snap->score to be ! -1 ? Not if everything is working correctly - if ( snap->score != -1 ) { + if (snap->score != -1) { snap->unlock(); packetqueue.increment_it(analysis_it); Error("skipping because score was %d", snap->score); @@ -1837,25 +1838,24 @@ bool Monitor::Analyse() { std::lock_guard lck(event_mutex); // if we have been told to be OFF, then we are off and don't do any processing. - if ( trigger_data->trigger_state != TriggerState::TRIGGER_OFF ) { + if (trigger_data->trigger_state != TriggerState::TRIGGER_OFF) { Debug(4, "Trigger not OFF state is (%d)", int(trigger_data->trigger_state)); int score = 0; // Ready means that we have captured the warmup # of frames - if ( !Ready() ) { + if (!Ready()) { Debug(3, "Not ready?"); snap->unlock(); return false; } - Debug(4, "Ready"); std::string cause; Event::StringSetMap noteSetMap; // Specifically told to be on. Setting the score here will trigger the alarm. - if ( trigger_data->trigger_state == TriggerState::TRIGGER_ON ) { + if (trigger_data->trigger_state == TriggerState::TRIGGER_ON) { score += trigger_data->trigger_score; Debug(1, "Triggered on score += %d => %d", trigger_data->trigger_score, score); - if ( !event ) { + if (!event) { cause += trigger_data->trigger_cause; } Event::StringSet noteSet; @@ -1863,12 +1863,13 @@ bool Monitor::Analyse() { noteSetMap[trigger_data->trigger_cause] = noteSet; } // end if trigger_on - if ( signal_change ) { + // FIXME this snap might not be the one that caused the signal change. Need to store that in the packet. + if (signal_change) { Debug(2, "Signal change, new signal is %d", signal); const char *signalText = "Unknown"; - if ( !signal ) { + if (!signal) { signalText = "Lost"; - if ( event ) { + if (event) { Info("%s: %03d - Closing event %" PRIu64 ", signal loss", name, analysis_image_count, event->Id()); closeEvent(); last_section_mod = 0; @@ -1877,9 +1878,8 @@ bool Monitor::Analyse() { signalText = "Reacquired"; score += 100; } - if ( !event ) { - if ( cause.length() ) - cause += ", "; + if (!event) { + if (cause.length()) cause += ", "; cause += SIGNAL_CAUSE; } Event::StringSet noteSet; @@ -1887,15 +1887,26 @@ bool Monitor::Analyse() { noteSetMap[SIGNAL_CAUSE] = noteSet; shared_data->state = state = IDLE; shared_data->active = signal; - if ( (function == MODECT or function == MOCORD) and snap->image ) + if ((function == MODECT or function == MOCORD) and snap->image) ref_image.Assign(*(snap->image)); }// else if (signal) { - if (snap->image or (snap->codec_type == AVMEDIA_TYPE_VIDEO)) { + if (snap->codec_type == AVMEDIA_TYPE_VIDEO) { + while (!snap->image and !snap->decoded) { + // Need to wait for the decoder thread. + Debug(1, "Waiting for decode"); + snap->wait(); + if (!snap->image and snap->decoded) { + Debug(1, "No image but was decoded, giving up"); + snap->unlock(); + return false; + } + } // end while ! decoded + struct timeval *timestamp = snap->timestamp; - if ( Active() and (function == MODECT or function == MOCORD) and snap->image ) { + if (Active() and (function == MODECT or function == MOCORD)) { Debug(3, "signal and active and modect"); Event::StringSet zoneSet; @@ -1999,7 +2010,7 @@ bool Monitor::Analyse() { ); // This gets a lock on the starting packet - ZMPacket *starting_packet = packetqueue.get_packet(start_it); + ZMPacket *starting_packet = ( *start_it == snap_it ) ? snap : packetqueue.get_packet(start_it); event = new Event(this, *(starting_packet->timestamp), "Continuous", noteSetMap); // Write out starting packets, do not modify packetqueue it will garbage collect itself @@ -2030,9 +2041,7 @@ bool Monitor::Analyse() { for ( int i=0; i < n_zones; i++ ) { if ( zones[i]->Alarmed() ) { alarm_cause += std::string(zones[i]->Label()); - if ( i < n_zones-1 ) { - alarm_cause += ","; - } + if (i < n_zones-1) alarm_cause += ","; } } alarm_cause = cause+" "+alarm_cause; @@ -2085,7 +2094,7 @@ bool Monitor::Analyse() { snap_it, (pre_event_count > alarm_frame_count ? pre_event_count : alarm_frame_count) ); - ZMPacket *starting_packet = *(*start_it); + ZMPacket *starting_packet = ( *start_it == snap_it ) ? snap : packetqueue.get_packet(start_it); event = new Event(this, *(starting_packet->timestamp), cause, noteSetMap); shared_data->last_event_id = event->Id(); @@ -2255,24 +2264,6 @@ bool Monitor::Analyse() { } // end if ( trigger_data->trigger_state != TRIGGER_OFF ) if (event) event->AddPacket(snap); -#if 0 - if (snap->packet.stream_index == video_stream_id) { - if (video_fifo) { - if ( snap->keyframe ) { - // avcodec strips out important nals that describe the stream and - // stick them in extradata. Need to send them along with keyframes - AVStream *stream = camera->getVideoStream(); - video_fifo->write( - static_cast(stream->codecpar->extradata), - stream->codecpar->extradata_size); - } - video_fifo->writePacket(*snap); - } - } else if (snap->packet.stream_index == audio_stream_id) { - if (audio_fifo) - audio_fifo->writePacket(*snap); - } -#endif // popPacket will have placed a second lock on snap, so release it here. snap->unlock(); @@ -2507,7 +2498,6 @@ int Monitor::Capture() { gettimeofday(packet->timestamp, nullptr); shared_data->zmc_heartbeat_time = packet->timestamp->tv_sec; - Image* capture_image = image_buffer[index].image; int captureResult = 0; if ( deinterlacing_value == 4 ) { @@ -2536,7 +2526,7 @@ int Monitor::Capture() { Rgb signalcolor; /* HTML colour code is actually BGR in memory, we want RGB */ signalcolor = rgb_convert(signal_check_colour, ZM_SUBPIX_ORDER_BGR); - capture_image = new Image(width, height, camera->Colours(), camera->SubpixelOrder()); + Image *capture_image = new Image(width, height, camera->Colours(), camera->SubpixelOrder()); capture_image->Fill(signalcolor); shared_data->signal = false; shared_data->last_write_index = index; @@ -2592,6 +2582,7 @@ int Monitor::Capture() { return 1; } // end if audio +#if 0 if ( !packet->image ) { if ( packet->packet.size and !packet->in_frame ) { if ( !decoding_enabled ) { @@ -2678,6 +2669,7 @@ int Monitor::Capture() { shared_data->signal = ( capture_image and signal_check_points ) ? CheckSignal(capture_image) : true; shared_data->last_write_index = index; shared_data->last_write_time = packet->timestamp->tv_sec; +#endif image_count++; // Will only be queued if there are iterators allocated in the queue. @@ -2710,6 +2702,100 @@ int Monitor::Capture() { return captureResult; } // end Monitor::Capture +bool Monitor::Decode() { + if (!decoder_it) decoder_it = packetqueue.get_video_it(true); + + ZMPacket *packet = packetqueue.get_packet(decoder_it); + if (!packet) return false; + packetqueue.increment_it(decoder_it); + if (packet->image or (packet->codec_type != AVMEDIA_TYPE_VIDEO)) { + packet->unlock(); + return true; // Don't need decode + } + + int ret = 0; + if (packet->packet.size and !packet->in_frame) { + // Allocate the image first so that it can be used by hwaccel + // We don't actually care about camera colours, pixel order etc. We care about the desired settings + // + //capture_image = packet->image = new Image(width, height, camera->Colours(), camera->SubpixelOrder()); + ret = packet->decode(camera->getVideoCodecContext()); + } else { + Debug(1, "No packet.size(%d) or packet->in_frame(%p). Not decoding", packet->packet.size, packet->in_frame); + } + Image* capture_image = nullptr; + unsigned int index = image_count % image_buffer_count; + if (ret > 0) { + if (packet->in_frame and !packet->image) { + packet->image = new Image(camera_width, camera_height, camera->Colours(), camera->SubpixelOrder()); + packet->get_image(); + } + + if (packet->image) { + capture_image = packet->image; + + /* Deinterlacing */ + if ( deinterlacing_value ) { + if ( deinterlacing_value == 1 ) { + capture_image->Deinterlace_Discard(); + } else if ( deinterlacing_value == 2 ) { + capture_image->Deinterlace_Linear(); + } else if ( deinterlacing_value == 3 ) { + capture_image->Deinterlace_Blend(); + } else if ( deinterlacing_value == 4 ) { + capture_image->Deinterlace_4Field(next_buffer.image, (deinterlacing>>8)&0xff); + } else if ( deinterlacing_value == 5 ) { + capture_image->Deinterlace_Blend_CustomRatio((deinterlacing>>8)&0xff); + } + } + + if ( orientation != ROTATE_0 ) { + Debug(2, "Doing rotation"); + switch ( orientation ) { + case ROTATE_0 : + // No action required + break; + case ROTATE_90 : + case ROTATE_180 : + case ROTATE_270 : + capture_image->Rotate((orientation-1)*90); + break; + case FLIP_HORI : + case FLIP_VERT : + capture_image->Flip(orientation==FLIP_HORI); + break; + } + } // end if have rotation + + if (privacy_bitmask) { + Debug(1, "Applying privacy"); + capture_image->MaskPrivacy(privacy_bitmask); + } + + if (config.timestamp_on_capture) { + Debug(1, "Timestampprivacy"); + TimestampImage(packet->image, packet->timestamp); + } + + if (!ref_image.Buffer()) { + // First image, so assign it to ref image + Debug(1, "Assigning ref image %dx%d size: %d", width, height, camera->ImageSize()); + ref_image.Assign(width, height, camera->Colours(), camera->SubpixelOrder(), + packet->image->Buffer(), camera->ImageSize()); + } + image_buffer[index].image->Assign(*(packet->image)); + *(image_buffer[index].timestamp) = *(packet->timestamp); + } // end if have image + } // end if did decoding + packet->decoded = true; + packet->unlock(); // unlock will also signal + + shared_data->signal = ( capture_image and signal_check_points ) ? CheckSignal(capture_image) : true; + shared_data->last_write_index = index; + shared_data->last_write_time = packet->timestamp->tv_sec; + return true; +} // end bool Monitor::Decode() + void Monitor::TimestampImage(Image *ts_image, const struct timeval *ts_time) const { if ( !label_format[0] ) return; @@ -3026,6 +3112,7 @@ int Monitor::PrimeCapture() { audio_fifo = new Fifo(shared_data->audio_fifo_path, true); } } // end if rtsp_server + decoder = new DecoderThread(this); } else { Debug(2, "Failed to prime %d", ret); } @@ -3035,6 +3122,8 @@ int Monitor::PrimeCapture() { int Monitor::PreCapture() const { return camera->PreCapture(); } int Monitor::PostCapture() const { return camera->PostCapture(); } int Monitor::Close() { + decoder->Stop(); + delete decoder; std::lock_guard lck(event_mutex); if (event) { Info("%s: image_count:%d - Closing event %" PRIu64 ", shutting down", name, image_count, event->Id()); diff --git a/src/zm_monitor.h b/src/zm_monitor.h index 20a59d79c..91d12dd24 100644 --- a/src/zm_monitor.h +++ b/src/zm_monitor.h @@ -22,6 +22,7 @@ #include "zm_define.h" #include "zm_camera.h" +#include "zm_decoder_thread.h" #include "zm_event.h" #include "zm_fifo.h" #include "zm_image.h" @@ -375,6 +376,8 @@ protected: VideoStore *videoStore; PacketQueue packetqueue; packetqueue_iterator *analysis_it; + packetqueue_iterator *decoder_it; + DecoderThread *decoder; int n_zones; @@ -549,6 +552,7 @@ public: //unsigned int DetectBlack( const Image &comp_image, Event::StringSet &zoneSet ); bool CheckSignal( const Image *image ); bool Analyse(); + bool Decode(); void DumpImage( Image *dump_image ) const; void TimestampImage( Image *ts_image, const struct timeval *ts_time ) const; bool closeEvent(); diff --git a/src/zm_packet.cpp b/src/zm_packet.cpp index 58c2bda8c..3a865fcc3 100644 --- a/src/zm_packet.cpp +++ b/src/zm_packet.cpp @@ -27,6 +27,7 @@ using namespace std; AVPixelFormat target_format = AV_PIX_FMT_NONE; ZMPacket::ZMPacket() : + lck(mutex,std::defer_lock), keyframe(0), in_frame(nullptr), out_frame(nullptr), @@ -37,7 +38,8 @@ ZMPacket::ZMPacket() : score(-1), codec_type(AVMEDIA_TYPE_UNKNOWN), image_index(-1), - codec_imgsize(0) + codec_imgsize(0), + decoded(0) { av_init_packet(&packet); packet.size = 0; // So we can detect whether it has been filled. diff --git a/src/zm_packet.h b/src/zm_packet.h index 118239860..f4032bf80 100644 --- a/src/zm_packet.h +++ b/src/zm_packet.h @@ -21,6 +21,7 @@ #define ZM_PACKET_H #include "zm_logger.h" +#include #include extern "C" { @@ -36,7 +37,7 @@ class Image; class ZMPacket { public: - std::recursive_mutex mutex; + int keyframe; AVStream *stream; // Input stream AVPacket packet; // Input packet, undecoded @@ -51,6 +52,7 @@ class ZMPacket { int image_index; int codec_imgsize; int64_t pts; // pts in the packet can be in another time base. This MUST be in AV_TIME_BASE_Q + bool decoded; public: AVPacket *av_packet() { return &packet; } @@ -65,21 +67,44 @@ class ZMPacket { explicit ZMPacket(ZMPacket &packet); ZMPacket(); ~ZMPacket(); - void lock() { - Debug(4,"Locking packet %d", this->image_index); - mutex.lock(); - Debug(4,"packet %d locked", this->image_index); + + std::unique_lock * lock() { + std::unique_lock *lck = new std::unique_lock(mutex); + Debug(4, "packet %d locked", this->image_index); + return lck; }; - bool trylock() { - Debug(4,"TryLocking packet %d", this->image_index); - return mutex.try_lock(); + std::unique_lock * trylock() { + std::unique_lock *lck = new std::unique_lock(mutex, std::defer_lock); + Debug(4, "TryLocking packet %d", this->image_index); + if ( lck.try_lock() ) + return lck; + delete lck; + return nullptr; }; - void unlock() { - Debug(4,"packet %d unlocked", this->image_index); - mutex.unlock(); + void unlock(std::unique_lock *lck) { + Debug(4, "packet %d unlocked", this->image_index); + lck->unlock(); + condition.notify_all(); }; - AVFrame *get_out_frame( const AVCodecContext *ctx ); + void wait(std::unique_lock *lck) { + Debug(4, "packet %d waiting", this->image_index); + // We already have a lock, but it's a recursive mutex.. so this may be ok + condition.wait(*lck); + } + AVFrame *get_out_frame(const AVCodecContext *ctx); int get_codec_imgsize() { return codec_imgsize; }; }; +class ZMLockedPacket : public ZMPacket { + public: + std::mutex mutex_; + std::condition_variable condition_; + std::unique_lock lck_; + ZMPacket *packet_; + + ZMLockedPacket(ZMPacket *p) : packet_(packet), lck_(mutex_) { + } + +} + #endif /* ZM_PACKET_H */ diff --git a/src/zm_packetqueue.cpp b/src/zm_packetqueue.cpp index 2ba04e71b..5f34b1f0d 100644 --- a/src/zm_packetqueue.cpp +++ b/src/zm_packetqueue.cpp @@ -478,8 +478,10 @@ ZMPacket *PacketQueue::get_packet(packetqueue_iterator *it) { return nullptr; } Debug(3, "get_packet %p image_index: %d, about to lock packet", p, p->image_index); - while ( !(zm_terminate or deleting) and !p->trylock() ) { - Debug(3, "waiting. Queue size %d it == end? %d", pktQueue.size(), ( *it == pktQueue.end() ) ); + while (!(zm_terminate or deleting) and !p->trylock()) { + Debug(3, "waiting on index %d. Queue size %d it == end? %d", + p->image_index, pktQueue.size(), ( *it == pktQueue.end() ) ); + ZM_DUMP_PACKET(p->packet, ""); condition.wait(lck); } Debug(2, "Locked packet, unlocking packetqueue mutex"); @@ -612,14 +614,14 @@ packetqueue_iterator * PacketQueue::get_video_it(bool wait) { while ( *it != pktQueue.end() ) { ZMPacket *zm_packet = *(*it); - if ( !zm_packet ) { + if (!zm_packet) { Error("Null zmpacket in queue!?"); free_it(it); return nullptr; } Debug(1, "Packet keyframe %d for stream %d, so returning the it to it", zm_packet->keyframe, zm_packet->packet.stream_index); - if ( zm_packet->keyframe and ( zm_packet->packet.stream_index == video_stream_id ) ) { + if (zm_packet->keyframe and ( zm_packet->packet.stream_index == video_stream_id )) { Debug(1, "Found a keyframe for stream %d, so returning the it to it", video_stream_id); return it; } @@ -627,7 +629,7 @@ packetqueue_iterator * PacketQueue::get_video_it(bool wait) { } Debug(1, "DIdn't Found a keyframe for stream %d, so returning the it to it", video_stream_id); return it; -} +} // get video_it void PacketQueue::free_it(packetqueue_iterator *it) { for ( From 6a11b23aafc401fb336257599ab99ccccec5a61a Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 15 Mar 2021 15:08:59 -0400 Subject: [PATCH 0185/1277] Add decoder thread --- src/zm_decoder_thread.cpp | 54 +++++++++++++++++++++++++++++++++++++++ src/zm_decoder_thread.h | 29 +++++++++++++++++++++ 2 files changed, 83 insertions(+) create mode 100644 src/zm_decoder_thread.cpp create mode 100644 src/zm_decoder_thread.h diff --git a/src/zm_decoder_thread.cpp b/src/zm_decoder_thread.cpp new file mode 100644 index 000000000..e8023e905 --- /dev/null +++ b/src/zm_decoder_thread.cpp @@ -0,0 +1,54 @@ +#include "zm_decoder_thread.h" + +#include "zm_monitor.h" +#include "zm_signal.h" +#include "zm_utils.h" + +//DecoderThread::DecoderThread(std::shared_ptr monitor) : +DecoderThread::DecoderThread(Monitor * monitor) : + monitor_(monitor), terminate_(false) { + //monitor_(std::move(monitor)), terminate_(false) { + thread_ = std::thread(&DecoderThread::Run, this); +} + +DecoderThread::~DecoderThread() { + Stop(); + if (thread_.joinable()) + thread_.join(); +} + +void DecoderThread::Run() { + Debug(2, "DecoderThread::Run() for %d", monitor_->Id()); + + //Microseconds decoder_rate = Microseconds(monitor_->GetDecoderRate()); + //Seconds decoder_update_delay = Seconds(monitor_->GetDecoderUpdateDelay()); + //Debug(2, "DecoderThread::Run() have update delay %d", decoder_update_delay); + + //TimePoint last_decoder_update_time = std::chrono::steady_clock::now(); + //TimePoint cur_time; + + while (!(terminate_ or zm_terminate)) { + // Some periodic updates are required for variable capturing framerate + //if (decoder_update_delay != Seconds::zero()) { + //cur_time = std::chrono::steady_clock::now(); + //Debug(2, "Updating adaptive skip"); + //if ((cur_time - last_decoder_update_time) > decoder_update_delay) { + //decoder_rate = Microseconds(monitor_->GetDecoderRate()); + //last_decoder_update_time = cur_time; + //} + //} + + if (!monitor_->Decode()) { + //if ( !(terminate_ or zm_terminate) ) { + //Microseconds sleep_for = monitor_->Active() ? Microseconds(ZM_SAMPLE_RATE) : Microseconds(ZM_SUSPENDED_RATE); + //Debug(2, "Sleeping for %" PRId64 "us", int64(sleep_for.count())); + //std::this_thread::sleep_for(sleep_for); + //} + //} else if (decoder_rate != Microseconds::zero()) { + //Debug(2, "Sleeping for %" PRId64 " us", int64(decoder_rate.count())); + //std::this_thread::sleep_for(decoder_rate); + //} else { + //Debug(2, "Not sleeping"); + } + } +} diff --git a/src/zm_decoder_thread.h b/src/zm_decoder_thread.h new file mode 100644 index 000000000..703a7e1e1 --- /dev/null +++ b/src/zm_decoder_thread.h @@ -0,0 +1,29 @@ +#ifndef ZM_DECODER_THREAD_H +#define ZM_DECODER_THREAD_H + +#include +#include +#include + +class Monitor; + +class DecoderThread { + public: + explicit DecoderThread(Monitor* monitor); + //explicit DecoderThread(std::shared_ptr monitor); + ~DecoderThread(); + DecoderThread(DecoderThread &rhs) = delete; + DecoderThread(DecoderThread &&rhs) = delete; + + void Stop() { terminate_ = true; } + + private: + void Run(); + + Monitor* monitor_; + //std::shared_ptr monitor_; + std::atomic terminate_; + std::thread thread_; +}; + +#endif From baf73fea7be90a307925ee479e253d51d47b40f1 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 15 Mar 2021 15:11:12 -0400 Subject: [PATCH 0186/1277] Ensure that we disconnect when ShmValid fails --- src/zm_rtsp_server.cpp | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/src/zm_rtsp_server.cpp b/src/zm_rtsp_server.cpp index 0fb5a8d15..7fe8e916a 100644 --- a/src/zm_rtsp_server.cpp +++ b/src/zm_rtsp_server.cpp @@ -187,14 +187,16 @@ int main(int argc, char *argv[]) { for (size_t i = 0; i < monitors.size(); i++) { std::shared_ptr monitor = monitors[i]; - if (!(monitor->ShmValid() or monitor->connect())) { - Warning("Couldn't connect to monitor %d", monitor->Id()); - if (sessions[i]) { - rtspServer->RemoveSession(sessions[i]->GetMediaSessionId()); - sessions[i] = nullptr; - } - monitor->disconnect(); - continue; + if (!monitor->ShmValid()) { + monitor->disconnect(); + if (!monitor->connect()) { + Warning("Couldn't connect to monitor %d", monitor->Id()); + if (sessions[i]) { + rtspServer->RemoveSession(sessions[i]->GetMediaSessionId()); + sessions[i] = nullptr; + } + continue; + } } if (!sessions[i]) { From 9903e909af93b628c558c8ec34de5fb0e9790236 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 15 Mar 2021 17:05:30 -0400 Subject: [PATCH 0187/1277] Rework locking in ZMPacket by using a new class called ZMLockedPacket. --- src/zm_monitor.cpp | 83 +++++++++++++++++++++++++++--------------- src/zm_packet.cpp | 1 - src/zm_packet.h | 61 ++++++++++++++++--------------- src/zm_packetqueue.cpp | 30 ++++++++------- src/zm_packetqueue.h | 5 ++- src/zm_videostore.cpp | 23 ------------ 6 files changed, 105 insertions(+), 98 deletions(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 73822502e..0ea6b5fb6 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -1813,12 +1813,13 @@ bool Monitor::Analyse() { // if have event, send frames until we find a video packet, at which point do analysis. Adaptive skip should only affect which frames we do analysis on. // get_analysis_packet will lock the packet and may wait if analysis_it is at the end - ZMPacket *snap = packetqueue.get_packet(analysis_it); - if (!snap) return false; + ZMLockedPacket *packet_lock = packetqueue.get_packet(analysis_it); + if (!packet_lock) return false; + ZMPacket *snap = packet_lock->packet_; // Is it possible for snap->score to be ! -1 ? Not if everything is working correctly if (snap->score != -1) { - snap->unlock(); + delete packet_lock; packetqueue.increment_it(analysis_it); Error("skipping because score was %d", snap->score); return false; @@ -1844,7 +1845,7 @@ bool Monitor::Analyse() { // Ready means that we have captured the warmup # of frames if (!Ready()) { Debug(3, "Not ready?"); - snap->unlock(); + delete packet_lock; return false; } @@ -1896,10 +1897,10 @@ bool Monitor::Analyse() { while (!snap->image and !snap->decoded) { // Need to wait for the decoder thread. Debug(1, "Waiting for decode"); - snap->wait(); + packet_lock->wait(); if (!snap->image and snap->decoded) { Debug(1, "No image but was decoded, giving up"); - snap->unlock(); + delete packet_lock; return false; } } // end while ! decoded @@ -2010,22 +2011,32 @@ bool Monitor::Analyse() { ); // This gets a lock on the starting packet - ZMPacket *starting_packet = ( *start_it == snap_it ) ? snap : packetqueue.get_packet(start_it); + + ZMLockedPacket *starting_packet_lock = nullptr; + ZMPacket *starting_packet = nullptr; + if ( *start_it != snap_it ) { + starting_packet_lock = packetqueue.get_packet(start_it); + if (!starting_packet_lock) return false; + starting_packet = starting_packet_lock->packet_; + } else { + starting_packet = snap; + } event = new Event(this, *(starting_packet->timestamp), "Continuous", noteSetMap); // Write out starting packets, do not modify packetqueue it will garbage collect itself - while ( starting_packet and (*start_it) != snap_it ) { + while ( starting_packet and ((*start_it) != snap_it) ) { event->AddPacket(starting_packet); // Have added the packet, don't want to unlock it until we have locked the next packetqueue.increment_it(start_it); if ( (*start_it) == snap_it ) { - starting_packet->unlock(); + if (starting_packet_lock) delete starting_packet_lock; break; } - ZMPacket *p = packetqueue.get_packet(start_it); - starting_packet->unlock(); - starting_packet = p; + ZMLockedPacket *lp = packetqueue.get_packet(start_it); + delete starting_packet_lock; + starting_packet_lock = lp; + starting_packet = lp->packet_; } packetqueue.free_it(start_it); delete start_it; @@ -2094,7 +2105,16 @@ bool Monitor::Analyse() { snap_it, (pre_event_count > alarm_frame_count ? pre_event_count : alarm_frame_count) ); - ZMPacket *starting_packet = ( *start_it == snap_it ) ? snap : packetqueue.get_packet(start_it); + + ZMLockedPacket *starting_packet_lock = nullptr; + ZMPacket *starting_packet = nullptr; + if ( *start_it != snap_it ) { + starting_packet_lock = packetqueue.get_packet(start_it); + if (!starting_packet_lock) return false; + starting_packet = starting_packet_lock->packet_; + } else { + starting_packet = snap; + } event = new Event(this, *(starting_packet->timestamp), cause, noteSetMap); shared_data->last_event_id = event->Id(); @@ -2108,12 +2128,13 @@ bool Monitor::Analyse() { packetqueue.increment_it(start_it); if ( (*start_it) == snap_it ) { - starting_packet->unlock(); + if (starting_packet_lock) delete starting_packet_lock; break; } - ZMPacket *p = packetqueue.get_packet(start_it); - starting_packet->unlock(); - starting_packet = p; + ZMLockedPacket *lp = packetqueue.get_packet(start_it); + delete starting_packet_lock; + starting_packet_lock = lp; + starting_packet = lp->packet_; } packetqueue.free_it(start_it); delete start_it; @@ -2266,7 +2287,7 @@ bool Monitor::Analyse() { if (event) event->AddPacket(snap); // popPacket will have placed a second lock on snap, so release it here. - snap->unlock(); + delete packet_lock; if ( snap->image_index > 0 ) { // Only do these if it's a video packet. @@ -2705,11 +2726,12 @@ int Monitor::Capture() { bool Monitor::Decode() { if (!decoder_it) decoder_it = packetqueue.get_video_it(true); - ZMPacket *packet = packetqueue.get_packet(decoder_it); - if (!packet) return false; + ZMLockedPacket *packet_lock = packetqueue.get_packet(decoder_it); + if (!packet_lock) return false; + ZMPacket *packet = packet_lock->packet_; packetqueue.increment_it(decoder_it); if (packet->image or (packet->codec_type != AVMEDIA_TYPE_VIDEO)) { - packet->unlock(); + delete packet_lock; return true; // Don't need decode } @@ -2788,7 +2810,7 @@ bool Monitor::Decode() { } // end if have image } // end if did decoding packet->decoded = true; - packet->unlock(); // unlock will also signal + delete packet_lock; shared_data->signal = ( capture_image and signal_check_points ) ? CheckSignal(capture_image) : true; shared_data->last_write_index = index; @@ -3112,7 +3134,7 @@ int Monitor::PrimeCapture() { audio_fifo = new Fifo(shared_data->audio_fifo_path, true); } } // end if rtsp_server - decoder = new DecoderThread(this); + if (decoding_enabled) decoder = new DecoderThread(this); } else { Debug(2, "Failed to prime %d", ret); } @@ -3140,25 +3162,25 @@ Monitor::Orientation Monitor::getOrientation() const { return orientation; } // So this should be done as the first task in the analysis thread startup. // This function is deprecated. void Monitor::get_ref_image() { - ZMPacket *snap = nullptr; + ZMLockedPacket *snap_lock = nullptr; if ( !analysis_it ) analysis_it = packetqueue.get_video_it(true); while ( ( - !( snap = packetqueue.get_packet(analysis_it)) + !( snap_lock = packetqueue.get_packet(analysis_it)) or - ( snap->codec_type != AVMEDIA_TYPE_VIDEO ) + ( snap_lock->packet_->codec_type != AVMEDIA_TYPE_VIDEO ) or - ! snap->image + ! snap_lock->packet_->image ) and !zm_terminate) { Debug(1, "Waiting for capture daemon lastwriteindex(%d) lastwritetime(%d)", shared_data->last_write_index, shared_data->last_write_time); - if ( snap and ! snap->image ) { - snap->unlock(); + if ( snap_lock and ! snap_lock->packet_->image ) { + delete snap_lock; // can't analyse it anyways, incremement packetqueue.increment_it(analysis_it); } @@ -3167,6 +3189,7 @@ void Monitor::get_ref_image() { if ( zm_terminate ) return; + ZMPacket *snap = snap_lock->packet_; Debug(1, "get_ref_image: packet.stream %d ?= video_stream %d, packet image id %d packet image %p", snap->packet.stream_index, video_stream_id, snap->image_index, snap->image ); // Might not have been decoded yet FIXME @@ -3177,7 +3200,7 @@ void Monitor::get_ref_image() { } else { Debug(2, "Have no ref image about to unlock"); } - snap->unlock(); + delete snap_lock; } std::vector Monitor::Groups() { diff --git a/src/zm_packet.cpp b/src/zm_packet.cpp index 3a865fcc3..cb37558b3 100644 --- a/src/zm_packet.cpp +++ b/src/zm_packet.cpp @@ -27,7 +27,6 @@ using namespace std; AVPixelFormat target_format = AV_PIX_FMT_NONE; ZMPacket::ZMPacket() : - lck(mutex,std::defer_lock), keyframe(0), in_frame(nullptr), out_frame(nullptr), diff --git a/src/zm_packet.h b/src/zm_packet.h index f4032bf80..507547f99 100644 --- a/src/zm_packet.h +++ b/src/zm_packet.h @@ -37,6 +37,8 @@ class Image; class ZMPacket { public: + std::mutex mutex_; + std::condition_variable condition_; int keyframe; AVStream *stream; // Input stream @@ -68,43 +70,44 @@ class ZMPacket { ZMPacket(); ~ZMPacket(); - std::unique_lock * lock() { - std::unique_lock *lck = new std::unique_lock(mutex); - Debug(4, "packet %d locked", this->image_index); - return lck; - }; - std::unique_lock * trylock() { - std::unique_lock *lck = new std::unique_lock(mutex, std::defer_lock); - Debug(4, "TryLocking packet %d", this->image_index); - if ( lck.try_lock() ) - return lck; - delete lck; - return nullptr; - }; - void unlock(std::unique_lock *lck) { - Debug(4, "packet %d unlocked", this->image_index); - lck->unlock(); - condition.notify_all(); - }; - void wait(std::unique_lock *lck) { - Debug(4, "packet %d waiting", this->image_index); - // We already have a lock, but it's a recursive mutex.. so this may be ok - condition.wait(*lck); - } AVFrame *get_out_frame(const AVCodecContext *ctx); int get_codec_imgsize() { return codec_imgsize; }; }; -class ZMLockedPacket : public ZMPacket { +class ZMLockedPacket { public: - std::mutex mutex_; - std::condition_variable condition_; - std::unique_lock lck_; ZMPacket *packet_; + std::unique_lock lck_; - ZMLockedPacket(ZMPacket *p) : packet_(packet), lck_(mutex_) { + ZMLockedPacket(ZMPacket *p) : + packet_(p), + lck_(packet_->mutex_, std::defer_lock) { + } + ~ZMLockedPacket() { + unlock(); } -} + void lock() { + Debug(4, "locking packet %d", packet_->image_index); + lck_.lock(); + Debug(4, "packet %d locked", packet_->image_index); + }; + bool trylock() { + Debug(4, "TryLocking packet %d", packet_->image_index); + return lck_.try_lock(); + }; + void unlock() { + Debug(4, "packet %d unlocked", packet_->image_index); + lck_.unlock(); + packet_->condition_.notify_all(); + }; + + void wait() { + Debug(4, "packet %d waiting", packet_->image_index); + // We already have a lock, but it's a recursive mutex.. so this may be ok + packet_->condition_.wait(lck_); + } + +}; #endif /* ZM_PACKET_H */ diff --git a/src/zm_packetqueue.cpp b/src/zm_packetqueue.cpp index 5f34b1f0d..fe1c6f5b2 100644 --- a/src/zm_packetqueue.cpp +++ b/src/zm_packetqueue.cpp @@ -144,17 +144,19 @@ void PacketQueue::clearPackets(ZMPacket *add_packet) { // First packet is special because we know it is a video keyframe and only need to check for lock ZMPacket *zm_packet = *it; - if ( zm_packet->trylock() ) { + ZMLockedPacket *lp = new ZMLockedPacket(zm_packet); + if ( lp->trylock() ) { ++it; - zm_packet->unlock(); + delete lp; // Since we have many packets in the queue, we should NOT be pointing at end so don't need to test for that while ( *it != add_packet ) { zm_packet = *it; - if ( !zm_packet->trylock() ) { + lp = new ZMLockedPacket(zm_packet); + if ( !lp->trylock() ) { break; } - zm_packet->unlock(); + delete lp; if ( is_there_an_iterator_pointing_to_packet(zm_packet) ) { Warning("Found iterator at beginning of queue. Some thread isn't keeping up"); @@ -200,7 +202,7 @@ void PacketQueue::clearPackets(ZMPacket *add_packet) { return; } // end voidPacketQueue::clearPackets(ZMPacket* zm_packet) -ZMPacket* PacketQueue::popPacket( ) { +ZMLockedPacket* PacketQueue::popPacket( ) { Debug(4, "pktQueue size %d", pktQueue.size()); if ( pktQueue.empty() ) { return nullptr; @@ -222,14 +224,15 @@ ZMPacket* PacketQueue::popPacket( ) { } } // end foreach iterator - zm_packet->lock(); + ZMLockedPacket *lp = new ZMLockedPacket (zm_packet); + lp->lock(); pktQueue.pop_front(); packet_counts[zm_packet->packet.stream_index] -= 1; mutex.unlock(); - return zm_packet; + return lp; } // popPacket @@ -325,9 +328,9 @@ void PacketQueue::clear() { while (!pktQueue.empty()) { ZMPacket *packet = pktQueue.front(); // Someone might have this packet, but not for very long and since we have locked the queue they won't be able to get another one - packet->lock(); + ZMLockedPacket lp(packet); + lp.lock(); pktQueue.pop_front(); - packet->unlock(); delete packet; } @@ -452,7 +455,7 @@ int PacketQueue::packet_count(int stream_id) { // Returns a packet. Packet will be locked -ZMPacket *PacketQueue::get_packet(packetqueue_iterator *it) { +ZMLockedPacket *PacketQueue::get_packet(packetqueue_iterator *it) { if ( deleting or zm_terminate ) return nullptr; @@ -477,16 +480,17 @@ ZMPacket *PacketQueue::get_packet(packetqueue_iterator *it) { Error("Null p?!"); return nullptr; } + ZMLockedPacket *lp = new ZMLockedPacket(p); Debug(3, "get_packet %p image_index: %d, about to lock packet", p, p->image_index); - while (!(zm_terminate or deleting) and !p->trylock()) { + while (!(zm_terminate or deleting) and !lp->trylock()) { Debug(3, "waiting on index %d. Queue size %d it == end? %d", p->image_index, pktQueue.size(), ( *it == pktQueue.end() ) ); ZM_DUMP_PACKET(p->packet, ""); condition.wait(lck); } Debug(2, "Locked packet, unlocking packetqueue mutex"); - return p; -} // end ZMPacket *PacketQueue::get_packet(it) + return lp; +} // end ZMLockedPacket *PacketQueue::get_packet(it) bool PacketQueue::increment_it(packetqueue_iterator *it) { Debug(2, "Incrementing %p, queue size %d, end? %d", it, pktQueue.size(), ((*it) == pktQueue.end())); diff --git a/src/zm_packetqueue.h b/src/zm_packetqueue.h index c0ba045b8..e759dc777 100644 --- a/src/zm_packetqueue.h +++ b/src/zm_packetqueue.h @@ -24,6 +24,7 @@ #include class ZMPacket; +class ZMLockedPacket; typedef std::list::iterator packetqueue_iterator; @@ -52,7 +53,7 @@ class PacketQueue { void setMaxVideoPackets(int p); bool queuePacket(ZMPacket* packet); - ZMPacket * popPacket(); + ZMLockedPacket * popPacket(); bool popVideoPacket(ZMPacket* packet); bool popAudioPacket(ZMPacket* packet); unsigned int clear(unsigned int video_frames_to_keep, int stream_id); @@ -68,7 +69,7 @@ class PacketQueue { bool increment_it(packetqueue_iterator *it); bool increment_it(packetqueue_iterator *it, int stream_id); - ZMPacket *get_packet(packetqueue_iterator *); + ZMLockedPacket *get_packet(packetqueue_iterator *); packetqueue_iterator *get_video_it(bool wait); packetqueue_iterator *get_stream_it(int stream_id); void free_it(packetqueue_iterator *); diff --git a/src/zm_videostore.cpp b/src/zm_videostore.cpp index 1ffc69e90..ba19f6f57 100644 --- a/src/zm_videostore.cpp +++ b/src/zm_videostore.cpp @@ -1211,29 +1211,6 @@ int VideoStore::writeAudioFramePacket(ZMPacket *zm_packet) { return 0; } // end int VideoStore::writeAudioFramePacket(AVPacket *ipkt) -int VideoStore::write_packets(PacketQueue &queue) { - // Need to write out all the frames from the last keyframe? - // No... need to write out all frames from when the event began. Due to PreEventFrames, this could be more than since the last keyframe. - unsigned int packet_count = 0; - ZMPacket *queued_packet; - - while ( ( queued_packet = queue.popPacket() ) ) { - AVPacket *avp = queued_packet->av_packet(); - - packet_count += 1; - //Write the packet to our video store - Debug(2, "Writing queued packet stream: %d KEY %d, remaining (%d)", - avp->stream_index, avp->flags & AV_PKT_FLAG_KEY, queue.size() ); - int ret = this->writePacket( queued_packet ); - if ( ret < 0 ) { - //Less than zero and we skipped a frame - } - delete queued_packet; - } // end while packets in the packetqueue - Debug(2, "Wrote %d queued packets", packet_count ); - return packet_count; -} // end int VideoStore::write_packets( PacketQueue &queue ) { - int VideoStore::write_packet(AVPacket *pkt, AVStream *stream) { pkt->pos = -1; pkt->stream_index = stream->index; From 76267bc57f48ec4ab4fda7cf506aeff25db6d35b Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 16 Mar 2021 12:08:11 -0400 Subject: [PATCH 0188/1277] put back deleting the raw image when not saving jpegs. We only need it for the snapshot and that should be the alarmed image anyways. --- src/zm_monitor.cpp | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 0ea6b5fb6..4097036ae 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -2286,6 +2286,15 @@ bool Monitor::Analyse() { if (event) event->AddPacket(snap); + // In the case where people have pre-alarm frames, the web ui will generate the frame images + // from the mp4. So no one will notice anyways. + if ((videowriter == PASSTHROUGH) and !savejpegs) { + Debug(1, "Deleting image data for %d", snap->image_index); + // Don't need raw images anymore + delete snap->image; + snap->image = nullptr; + } + // popPacket will have placed a second lock on snap, so release it here. delete packet_lock; From 9d239219de5c6a0bf06030ff8fc513c64ac2dd81 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 16 Mar 2021 12:08:45 -0400 Subject: [PATCH 0189/1277] Break out early if no more buffer. Saves a couple cycles --- src/zm_rtsp_server_fifo_h264_source.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/src/zm_rtsp_server_fifo_h264_source.cpp b/src/zm_rtsp_server_fifo_h264_source.cpp index 6d7dda1b1..3d32f1809 100644 --- a/src/zm_rtsp_server_fifo_h264_source.cpp +++ b/src/zm_rtsp_server_fifo_h264_source.cpp @@ -83,6 +83,7 @@ std::list< std::pair > H264_ZoneMinderFifoSource::splitF } #endif frameList.push_back(std::pair(buffer, size)); + if (!bufSize) break; buffer = this->extractFrame(&buffer[size], bufSize, size); } // end while buffer From 28700fd56b94345b2c9651ca3a8bfab2cc68133c Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 16 Mar 2021 12:09:14 -0400 Subject: [PATCH 0190/1277] Implement saving DecodingEnabled from function view --- web/includes/actions/function.php | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/web/includes/actions/function.php b/web/includes/actions/function.php index 338361883..fc7f7703b 100644 --- a/web/includes/actions/function.php +++ b/web/includes/actions/function.php @@ -40,10 +40,12 @@ if ( $action == 'function' ) { $newFunction = validStr($_REQUEST['newFunction']); # Because we use a checkbox, it won't get passed in the request. So not being in _REQUEST means 0 $newEnabled = ( !isset($_REQUEST['newEnabled']) or $_REQUEST['newEnabled'] != '1' ) ? '0' : '1'; + $newDecodingEnabled = ( !isset($_REQUEST['newDecodingEnabled']) or $_REQUEST['newDecodingEnabled'] != '1' ) ? '0' : '1'; $oldFunction = $monitor->Function(); $oldEnabled = $monitor->Enabled(); - if ( $newFunction != $oldFunction || $newEnabled != $oldEnabled ) { - $monitor->save(array('Function'=>$newFunction, 'Enabled'=>$newEnabled)); + $oldDecodingEnabled = $monitor->DecodingEnabled(); + if ( $newFunction != $oldFunction || $newEnabled != $oldEnabled || $newDecodingEnabled != $oldDecodingEnabled ) { + $monitor->save(array('Function'=>$newFunction, 'Enabled'=>$newEnabled, 'DecodingEnabled'=>$newDecodingEnabled)); if ( daemonCheck() && ($monitor->Type() != 'WebSite') ) { $monitor->zmcControl(($newFunction != 'None') ? 'restart' : 'stop'); From ebf1b7cbdc05a5b952c6aebebfcc40b01a27ff3f Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 16 Mar 2021 13:26:06 -0400 Subject: [PATCH 0191/1277] Only output to stdout if mTerminalLevel is something. zms for example SHOULD not output to stdout, ever except maybe when running from terminal to debug --- src/zm_logger.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/zm_logger.cpp b/src/zm_logger.cpp index 75d4d4865..3d0c0c2c5 100644 --- a/src/zm_logger.cpp +++ b/src/zm_logger.cpp @@ -524,8 +524,8 @@ void Logger::logPrint(bool hex, const char * const filepath, const int line, con if (mLogFileFP) { fputs(logString, mLogFileFP); if (mFlush) fflush(mLogFileFP); - } else { - puts("Logging to file, but failed to open it\n"); + } else if (mTerminalLevel != NOLOG) { + puts("Logging to file but failed to open it\n"); } } // end if level <= mFileLevel From 5e54a63bd5f4a83a879f7559c28ed146f60dedbc Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 16 Mar 2021 13:26:40 -0400 Subject: [PATCH 0192/1277] Only load zones if doing something other than QUERY. Only delete decoder if there is one. --- src/zm_monitor.cpp | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 4097036ae..bc91de2bf 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -446,8 +446,7 @@ void Monitor::Load(MYSQL_ROW dbrow, bool load_zones=true, Purpose p = QUERY) { server_id = dbrow[col] ? atoi(dbrow[col]) : 0; col++; storage_id = atoi(dbrow[col]); col++; - if ( storage ) - delete storage; + if (storage) delete storage; storage = new Storage(storage_id); if ( ! strcmp(dbrow[col], "Local") ) { @@ -636,16 +635,16 @@ void Monitor::Load(MYSQL_ROW dbrow, bool load_zones=true, Purpose p = QUERY) { image_buffer_count, image_size, (image_buffer_count*image_size), mem_size); - Zone **zones = 0; - int n_zones = Zone::Load(this, zones); - this->AddZones(n_zones, zones); - this->AddPrivacyBitmask(zones); - // Should maybe store this for later use std::string monitor_dir = stringtf("%s/%d", storage->Path(), id); LoadCamera(); if ( purpose != QUERY ) { + Zone **zones = 0; + int n_zones = Zone::Load(this, zones); + this->AddZones(n_zones, zones); + this->AddPrivacyBitmask(zones); + if ( mkdir(monitor_dir.c_str(), 0755) && ( errno != EEXIST ) ) { Error("Can't mkdir %s: %s", monitor_dir.c_str(), strerror(errno)); } @@ -2733,7 +2732,6 @@ int Monitor::Capture() { } // end Monitor::Capture bool Monitor::Decode() { - if (!decoder_it) decoder_it = packetqueue.get_video_it(true); ZMLockedPacket *packet_lock = packetqueue.get_packet(decoder_it); if (!packet_lock) return false; @@ -3143,7 +3141,10 @@ int Monitor::PrimeCapture() { audio_fifo = new Fifo(shared_data->audio_fifo_path, true); } } // end if rtsp_server - if (decoding_enabled) decoder = new DecoderThread(this); + if (decoding_enabled) { + if (!decoder_it) decoder_it = packetqueue.get_video_it(true); + decoder = new DecoderThread(this); + } } else { Debug(2, "Failed to prime %d", ret); } @@ -3153,8 +3154,10 @@ int Monitor::PrimeCapture() { int Monitor::PreCapture() const { return camera->PreCapture(); } int Monitor::PostCapture() const { return camera->PostCapture(); } int Monitor::Close() { - decoder->Stop(); - delete decoder; + if ( decoder ) { + decoder->Stop(); + delete decoder; + } std::lock_guard lck(event_mutex); if (event) { Info("%s: image_count:%d - Closing event %" PRIu64 ", shutting down", name, image_count, event->Id()); From 028f2dd626eecb62c5f4371aa80e58a931ff556b Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 16 Mar 2021 13:27:27 -0400 Subject: [PATCH 0193/1277] Debug extra error log and code style --- src/zm_monitorstream.cpp | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/src/zm_monitorstream.cpp b/src/zm_monitorstream.cpp index fd618800e..b31c8e642 100644 --- a/src/zm_monitorstream.cpp +++ b/src/zm_monitorstream.cpp @@ -467,7 +467,7 @@ bool MonitorStream::sendFrame(Image *image, struct timeval *timestamp) { } // end bool MonitorStream::sendFrame( Image *image, struct timeval *timestamp ) void MonitorStream::runStream() { - if ( type == STREAM_SINGLE ) { + if (type == STREAM_SINGLE) { // Not yet migrated over to stream class SingleImage(scale); return; @@ -479,15 +479,13 @@ void MonitorStream::runStream() { fputs("Content-Type: multipart/x-mixed-replace; boundary=" BOUNDARY "\r\n\r\n", stdout); if ( !checkInitialised() ) { - Error("Not initialized"); - while ( !(loadMonitor(monitor_id) || zm_terminate) ) { + while (!(loadMonitor(monitor_id) || zm_terminate)) { sendTextFrame("Not connected"); - if ( connkey ) + if (connkey) checkCommandQueue(); sleep(1); } - if ( zm_terminate ) - return; + if (zm_terminate) return; } updateFrameRate(monitor->GetFPS()); @@ -562,7 +560,7 @@ void MonitorStream::runStream() { capture_fps = capture_max_fps; } - while ( !zm_terminate ) { + while (!zm_terminate) { bool got_command = false; if ( feof(stdout) ) { Debug(2, "feof stdout"); @@ -570,7 +568,7 @@ void MonitorStream::runStream() { } else if ( ferror(stdout) ) { Debug(2, "ferror stdout"); break; - } else if ( !monitor->ShmValid() ) { + } else if (!monitor->ShmValid()) { Debug(2, "monitor not valid.... maybe we should wait until it comes back."); break; } From c0242e736970a6d5d534a9f7618d5fc807acc76c Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 16 Mar 2021 13:28:00 -0400 Subject: [PATCH 0194/1277] Fix memleak when connect fails --- src/zm_stream.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/zm_stream.cpp b/src/zm_stream.cpp index 1f141ec65..e309a14bc 100644 --- a/src/zm_stream.cpp +++ b/src/zm_stream.cpp @@ -40,7 +40,7 @@ StreamBase::~StreamBase() { bool StreamBase::loadMonitor(int p_monitor_id) { monitor_id = p_monitor_id; - if ( !(monitor = Monitor::Load(monitor_id, false, Monitor::QUERY)) ) { + if ( !(monitor or (monitor = Monitor::Load(monitor_id, false, Monitor::QUERY))) ) { Error("Unable to load monitor id %d for streaming", monitor_id); return false; } @@ -52,6 +52,7 @@ bool StreamBase::loadMonitor(int p_monitor_id) { if ( !monitor->connect() ) { Error("Unable to connect to monitor id %d for streaming", monitor_id); + monitor->disconnect(); return false; } From 8fa989f8e9452c14d436b6c907d5bdfc15192699 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 16 Mar 2021 20:07:06 -0400 Subject: [PATCH 0195/1277] Increase debug level of input selection --- src/zm_ffmpeg_camera.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/zm_ffmpeg_camera.cpp b/src/zm_ffmpeg_camera.cpp index a1d8fb418..2f89197cb 100644 --- a/src/zm_ffmpeg_camera.cpp +++ b/src/zm_ffmpeg_camera.cpp @@ -200,10 +200,10 @@ int FfmpegCamera::Capture(ZMPacket &zm_packet) { ) ) { // if audio stream is behind video stream, then read from audio, otherwise video mFormatContextPtr = mSecondFormatContext; - Debug(1, "Using audio input"); + Debug(2, "Using audio input"); } else { mFormatContextPtr = mFormatContext; - Debug(1, "Using video input because %" PRId64 " >= %" PRId64, + Debug(2, "Using video input because %" PRId64 " >= %" PRId64, (mAudioStream?av_rescale_q(mLastAudioPTS, mAudioStream->time_base, AV_TIME_BASE_Q):0), av_rescale_q(mLastVideoPTS, mVideoStream->time_base, AV_TIME_BASE_Q) ); From 12ed02a5b092ef7aaaf5d514b82593ca793a472f Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 16 Mar 2021 20:07:59 -0400 Subject: [PATCH 0196/1277] Move trigger detection before motion detection. Only wait for decoding if decoding is enabled --- src/zm_monitor.cpp | 99 ++++++++++++++++++++++++---------------------- 1 file changed, 52 insertions(+), 47 deletions(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index bc91de2bf..091d2278e 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -1893,16 +1893,55 @@ bool Monitor::Analyse() { if (signal) { if (snap->codec_type == AVMEDIA_TYPE_VIDEO) { - while (!snap->image and !snap->decoded) { - // Need to wait for the decoder thread. + // Check to see if linked monitors are triggering. + if (n_linked_monitors > 0) { + Debug(1, "Checking linked monitors"); + // FIXME improve logic here + bool first_link = true; + Event::StringSet noteSet; + for ( int i = 0; i < n_linked_monitors; i++ ) { + // TODO: Shouldn't we try to connect? + if ( linked_monitors[i]->isConnected() ) { + Debug(1, "Linked monitor %d %s is connected", + linked_monitors[i]->Id(), linked_monitors[i]->Name()); + if ( linked_monitors[i]->hasAlarmed() ) { + Debug(1, "Linked monitor %d %s is alarmed", + linked_monitors[i]->Id(), linked_monitors[i]->Name()); + if ( !event ) { + if ( first_link ) { + if ( cause.length() ) + cause += ", "; + cause += LINKED_CAUSE; + first_link = false; + } + } + noteSet.insert(linked_monitors[i]->Name()); + score += linked_monitors[i]->lastFrameScore(); // 50; + } else { + Debug(1, "Linked monitor %d %s is not alarmed", + linked_monitors[i]->Id(), linked_monitors[i]->Name()); + } + } else { + Debug(1, "Linked monitor %d %d is not connected. Connecting.", i, linked_monitors[i]->Id()); + linked_monitors[i]->connect(); + } + } // end foreach linked_monitor + if ( noteSet.size() > 0 ) + noteSetMap[LINKED_CAUSE] = noteSet; + } // end if linked_monitors + + if ( decoding_enabled ) { + while (!snap->image and !snap->decoded and !zm_terminate) { + // Need to wait for the decoder thread. Debug(1, "Waiting for decode"); - packet_lock->wait(); - if (!snap->image and snap->decoded) { - Debug(1, "No image but was decoded, giving up"); - delete packet_lock; - return false; - } - } // end while ! decoded + packet_lock->wait(); + if (!snap->image and snap->decoded) { + Debug(1, "No image but was decoded, giving up"); + delete packet_lock; + return false; + } + } // end while ! decoded + } struct timeval *timestamp = snap->timestamp; @@ -1943,42 +1982,6 @@ bool Monitor::Analyse() { } // end if motion_score } // end if active and doing motion detection - // Check to see if linked monitors are triggering. - if (n_linked_monitors > 0) { - Debug(4, "Checking linked monitors"); - // FIXME improve logic here - bool first_link = true; - Event::StringSet noteSet; - for ( int i = 0; i < n_linked_monitors; i++ ) { - // TODO: Shouldn't we try to connect? - if ( linked_monitors[i]->isConnected() ) { - Debug(4, "Linked monitor %d %s is connected", - linked_monitors[i]->Id(), linked_monitors[i]->Name()); - if ( linked_monitors[i]->hasAlarmed() ) { - Debug(4, "Linked monitor %d %s is alarmed", - linked_monitors[i]->Id(), linked_monitors[i]->Name()); - if ( !event ) { - if ( first_link ) { - if ( cause.length() ) - cause += ", "; - cause += LINKED_CAUSE; - first_link = false; - } - } - noteSet.insert(linked_monitors[i]->Name()); - score += linked_monitors[i]->lastFrameScore(); // 50; - } else { - Debug(4, "Linked monitor %d %s is not alarmed", - linked_monitors[i]->Id(), linked_monitors[i]->Name()); - } - } else { - Debug(1, "Linked monitor %d %d is not connected. Connecting.", i, linked_monitors[i]->Id()); - linked_monitors[i]->connect(); - } - } // end foreach linked_monitor - if ( noteSet.size() > 0 ) - noteSetMap[LINKED_CAUSE] = noteSet; - } // end if linked_monitors if (function == RECORD or function == MOCORD) { // If doing record, check to see if we need to close the event or not. @@ -2287,7 +2290,7 @@ bool Monitor::Analyse() { // In the case where people have pre-alarm frames, the web ui will generate the frame images // from the mp4. So no one will notice anyways. - if ((videowriter == PASSTHROUGH) and !savejpegs) { + if (snap->image and (videowriter == PASSTHROUGH) and !savejpegs) { Debug(1, "Deleting image data for %d", snap->image_index); // Don't need raw images anymore delete snap->image; @@ -2569,6 +2572,8 @@ int Monitor::Capture() { // Don't want to do analysis on it, but we won't due to signal return -1; } else if ( captureResult > 0 ) { + // If we captured, let's assume signal, ::Decode will detect further + shared_data->signal = true; Debug(2, "Have packet stream_index:%d ?= videostream_id:(%d) q.vpktcount(%d) event?(%d) ", packet->packet.stream_index, video_stream_id, packetqueue.packet_count(video_stream_id), ( event ? 1 : 0 ) ); @@ -2732,6 +2737,7 @@ int Monitor::Capture() { } // end Monitor::Capture bool Monitor::Decode() { + if (!decoder_it) decoder_it = packetqueue.get_video_it(true); ZMLockedPacket *packet_lock = packetqueue.get_packet(decoder_it); if (!packet_lock) return false; @@ -3142,7 +3148,6 @@ int Monitor::PrimeCapture() { } } // end if rtsp_server if (decoding_enabled) { - if (!decoder_it) decoder_it = packetqueue.get_video_it(true); decoder = new DecoderThread(this); } } else { From 1daafd7f851ff6b336a973160036b98805d01dc3 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 16 Mar 2021 20:08:08 -0400 Subject: [PATCH 0197/1277] add GetType --- src/zm_monitor.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/zm_monitor.h b/src/zm_monitor.h index 91d12dd24..dbd54d6cd 100644 --- a/src/zm_monitor.h +++ b/src/zm_monitor.h @@ -414,6 +414,8 @@ public: if ( shared_data && shared_data->valid ) { struct timeval now; gettimeofday(&now, nullptr); + Debug(3, "Shared data is valid, checking heartbeat %u - %u = %d < %f", + now.tv_sec, shared_data->zmc_heartbeat_time, (now.tv_sec - shared_data->zmc_heartbeat_time), config.watch_max_delay); if ((now.tv_sec - shared_data->zmc_heartbeat_time) < config.watch_max_delay) return true; } @@ -429,6 +431,7 @@ public: } return storage; } + inline CameraType GetType() const { return type; } inline Function GetFunction() const { return function; } inline PacketQueue * GetPacketQueue() { return &packetqueue; } inline bool Enabled() const { From fe17d7bb2320e233e1c1ed94842c6d7599e4b16a Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 16 Mar 2021 20:09:14 -0400 Subject: [PATCH 0198/1277] Add checks for aliveness of monitor in streaming. If decoding disabled can't view stream. --- src/zm_monitorstream.cpp | 31 ++++++++++++++++++++----------- 1 file changed, 20 insertions(+), 11 deletions(-) diff --git a/src/zm_monitorstream.cpp b/src/zm_monitorstream.cpp index b31c8e642..491019a3a 100644 --- a/src/zm_monitorstream.cpp +++ b/src/zm_monitorstream.cpp @@ -469,24 +469,33 @@ bool MonitorStream::sendFrame(Image *image, struct timeval *timestamp) { void MonitorStream::runStream() { if (type == STREAM_SINGLE) { // Not yet migrated over to stream class - SingleImage(scale); + if (checkInitialised()) + SingleImage(scale); + else + sendTextFrame("Unable to send image"); + return; } openComms(); - if ( type == STREAM_JPEG ) + if (type == STREAM_JPEG) fputs("Content-Type: multipart/x-mixed-replace; boundary=" BOUNDARY "\r\n\r\n", stdout); - if ( !checkInitialised() ) { - while (!(loadMonitor(monitor_id) || zm_terminate)) { - sendTextFrame("Not connected"); - if (connkey) - checkCommandQueue(); - sleep(1); - } - if (zm_terminate) return; + while (!(loadMonitor(monitor_id) || zm_terminate)) { + sendTextFrame("Not connected"); + if (connkey) + checkCommandQueue(); + sleep(1); } + if (zm_terminate) return; + while (!checkInitialised() and !zm_terminate) { + sendTextFrame("Unable to stream"); + if (connkey) + checkCommandQueue(); + sleep(1); + } + if (zm_terminate) return; updateFrameRate(monitor->GetFPS()); @@ -854,7 +863,7 @@ void MonitorStream::SingleImage(int scale) { int img_buffer_size = 0; static JOCTET img_buffer[ZM_MAX_IMAGE_SIZE]; Image scaled_image; - while ( monitor->shared_data->last_write_index >= monitor->image_buffer_count ) { + while ((monitor->shared_data->last_write_index >= monitor->image_buffer_count) and !zm_terminate) { Debug(1, "Waiting for capture to begin"); usleep(100000); } From 14ef8336b9c4d5380e1c355b79997b2799f6cfa5 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 16 Mar 2021 20:10:00 -0400 Subject: [PATCH 0199/1277] If ffmpeg and decoding Disabled don't stream --- src/zm_stream.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/zm_stream.cpp b/src/zm_stream.cpp index e309a14bc..869341114 100644 --- a/src/zm_stream.cpp +++ b/src/zm_stream.cpp @@ -72,6 +72,10 @@ bool StreamBase::checkInitialised() { Error("Monitor shm is not connected"); return false; } + if ((monitor->GetType() == Monitor::FFMPEG) and !monitor->DecodingEnabled() ) { + Error("Monitor is not decoding."); + return false; + } return true; } From 74616d106192a7579a7b3272bf4083cda5f30cc3 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 17 Mar 2021 10:06:45 -0400 Subject: [PATCH 0200/1277] Update CaptureFPS SQL to just do an update and don't use static sql. TThis may fix a signal 6 crash that we have been seeing --- src/zm_monitor.cpp | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 091d2278e..2835146ff 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -1734,16 +1734,10 @@ void Monitor::UpdateCaptureFPS() { last_fps_time = now_double; last_capture_image_count = image_count; - static char sql[ZM_SQL_SML_BUFSIZ]; - // The reason we update the Status as well is because if mysql restarts, the Monitor_Status table is lost, - // and nothing else will update the status until zmc restarts. Since we are successfully capturing we can - // assume that we are connected - snprintf(sql, sizeof(sql), - "INSERT INTO Monitor_Status (MonitorId,CaptureFPS,CaptureBandwidth,Status) " - "VALUES (%d, %.2lf, %u, 'Connected') ON DUPLICATE KEY UPDATE " - "CaptureFPS = %.2lf, CaptureBandwidth=%u, Status='Connected'", - id, new_capture_fps, new_capture_bandwidth, new_capture_fps, new_capture_bandwidth); - dbQueue.push(sql); + std::string sql = stringtf( + "UPDATE Monitor_Status SET CaptureFPS = %.2lf, CaptureBandwidth=%u WHERE MonitorId=%u", + new_capture_fps, new_capture_bandwidth, id); + dbQueue.push(sql.c_str()); } // now != last_fps_time } // end if report fps } // void Monitor::UpdateCaptureFPS() From 6a2e237902a57306875ab0aeac18eb0b737f3045 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 17 Mar 2021 10:07:03 -0400 Subject: [PATCH 0201/1277] Fix delete packet before deleting lock on packet --- src/zm_packetqueue.cpp | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/src/zm_packetqueue.cpp b/src/zm_packetqueue.cpp index fe1c6f5b2..4cbf23910 100644 --- a/src/zm_packetqueue.cpp +++ b/src/zm_packetqueue.cpp @@ -87,7 +87,7 @@ bool PacketQueue::queuePacket(ZMPacket* add_packet) { pktQueue.push_back(add_packet); packet_counts[add_packet->packet.stream_index] += 1; - Debug(1, "packet counts for %d is %d", + Debug(2, "packet counts for %d is %d", add_packet->packet.stream_index, packet_counts[add_packet->packet.stream_index]); @@ -328,9 +328,10 @@ void PacketQueue::clear() { while (!pktQueue.empty()) { ZMPacket *packet = pktQueue.front(); // Someone might have this packet, but not for very long and since we have locked the queue they won't be able to get another one - ZMLockedPacket lp(packet); - lp.lock(); + ZMLockedPacket *lp = new ZMLockedPacket(packet); + lp->lock(); pktQueue.pop_front(); + delete lp; delete packet; } @@ -605,7 +606,7 @@ packetqueue_iterator * PacketQueue::get_video_it(bool wait) { if ( wait ) { while ( ((! pktQueue.size()) or (*it == pktQueue.end())) and !zm_terminate and !deleting ) { - Debug(2, "waiting. Queue size %d it == end? %d", pktQueue.size(), ( *it == pktQueue.end() ) ); + Debug(2, "waiting for packets in queue. Queue size %d it == end? %d", pktQueue.size(), ( *it == pktQueue.end() ) ); condition.wait(lck); *it = pktQueue.begin(); } From 657a5edda4ce0135672a2f0a124e024f06b02c0b Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 17 Mar 2021 10:11:06 -0400 Subject: [PATCH 0202/1277] If decoding disabled, set signal and last_write_time in the Capture thread. So that zm_watch knows we are alive --- src/zm_monitor.cpp | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 2835146ff..86d1774f4 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -2567,7 +2567,11 @@ int Monitor::Capture() { return -1; } else if ( captureResult > 0 ) { // If we captured, let's assume signal, ::Decode will detect further - shared_data->signal = true; + if (!decoding_enabled) { + shared_data->last_write_index = index; + shared_data->last_write_time = packet->timestamp->tv_sec; + shared_data->signal = true; + } Debug(2, "Have packet stream_index:%d ?= videostream_id:(%d) q.vpktcount(%d) event?(%d) ", packet->packet.stream_index, video_stream_id, packetqueue.packet_count(video_stream_id), ( event ? 1 : 0 ) ); From a8d31ca6862405dcfb34a961e7424ceaf665d062 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 17 Mar 2021 12:46:55 -0400 Subject: [PATCH 0203/1277] After moving analysis thread into monitor, I don't know how to handle the shared_ptr stuff --- src/zm_analysis_thread.cpp | 7 +++++-- src/zm_analysis_thread.h | 8 +++++--- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/src/zm_analysis_thread.cpp b/src/zm_analysis_thread.cpp index 3e66940a0..7a6b01bbb 100644 --- a/src/zm_analysis_thread.cpp +++ b/src/zm_analysis_thread.cpp @@ -1,10 +1,13 @@ #include "zm_analysis_thread.h" +#include "zm_monitor.h" #include "zm_signal.h" #include "zm_utils.h" -AnalysisThread::AnalysisThread(std::shared_ptr monitor) : - monitor_(std::move(monitor)), terminate_(false) { +//AnalysisThread::AnalysisThread(std::shared_ptr monitor) : +AnalysisThread::AnalysisThread(Monitor* monitor) : + monitor_(monitor), terminate_(false) { + //monitor_(std::move(monitor)), terminate_(false) { thread_ = std::thread(&AnalysisThread::Run, this); } diff --git a/src/zm_analysis_thread.h b/src/zm_analysis_thread.h index 8cf389e93..7fc0fd8c9 100644 --- a/src/zm_analysis_thread.h +++ b/src/zm_analysis_thread.h @@ -1,14 +1,15 @@ #ifndef ZM_ANALYSIS_THREAD_H #define ZM_ANALYSIS_THREAD_H -#include "zm_monitor.h" +class Monitor; #include #include #include class AnalysisThread { public: - explicit AnalysisThread(std::shared_ptr monitor); + explicit AnalysisThread(Monitor* monitor); + //explicit AnalysisThread(std::shared_ptr monitor); ~AnalysisThread(); AnalysisThread(AnalysisThread &rhs) = delete; AnalysisThread(AnalysisThread &&rhs) = delete; @@ -18,7 +19,8 @@ class AnalysisThread { private: void Run(); - std::shared_ptr monitor_; + Monitor* monitor_; + //std::shared_ptr monitor_; std::atomic terminate_; std::thread thread_; }; From dca34544ec5d9bac52ecfe7e6942ea0bd80b98b1 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 17 Mar 2021 12:47:52 -0400 Subject: [PATCH 0204/1277] move analysis thread into monitor. populate analysis_it and decoder_it in Prime instead of constantly checking for them. Handle cases where LockedPacket are null due to shutdown --- src/zm_monitor.cpp | 120 +++++++++++++++++++++++++++------------------ 1 file changed, 71 insertions(+), 49 deletions(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 86d1774f4..38a2814a3 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -395,6 +395,7 @@ Monitor::Monitor() storage(nullptr), videoStore(nullptr), analysis_it(nullptr), + analysis_thread(nullptr), decoder_it(nullptr), decoder(nullptr), n_zones(0), @@ -1126,12 +1127,11 @@ Monitor::~Monitor() { disconnect(); } // end if mem_ptr - if (analysis_it) { - packetqueue.free_it(analysis_it); - analysis_it = nullptr; - } + // Will be free by packetqueue destructor + analysis_it = nullptr; + decoder_it = nullptr; - for ( int i = 0; i < n_zones; i++ ) { + for (int i = 0; i < n_zones; i++) { delete zones[i]; } delete[] zones; @@ -1801,7 +1801,6 @@ bool Monitor::Analyse() { Warning("Shouldn't be doing Analyse when not Enabled"); return false; } - if (!analysis_it) analysis_it = packetqueue.get_video_it(true); // if have event, send frames until we find a video packet, at which point do analysis. Adaptive skip should only affect which frames we do analysis on. @@ -1959,12 +1958,12 @@ bool Monitor::Analyse() { Debug(3, "After motion detection, score:%d last_motion_score(%d), new motion score(%d)", score, last_motion_score, motion_score); + motion_frame_count += 1; } else { - Warning("No image in snap"); + Debug(1, "No image in snap, codec likely not ready"); } // Why are we updating the last_motion_score too? last_motion_score = motion_score; - motion_frame_count += 1; } else { Debug(1, "Skipped motion detection"); } @@ -2031,6 +2030,7 @@ bool Monitor::Analyse() { } ZMLockedPacket *lp = packetqueue.get_packet(start_it); delete starting_packet_lock; + if (!lp) return false; starting_packet_lock = lp; starting_packet = lp->packet_; } @@ -2129,6 +2129,11 @@ bool Monitor::Analyse() { } ZMLockedPacket *lp = packetqueue.get_packet(start_it); delete starting_packet_lock; + if (!lp) { + // Shutting down event will be closed by ~Monitor() + // Perhaps we shouldn't do this. + return false; + } starting_packet_lock = lp; starting_packet = lp->packet_; } @@ -2164,7 +2169,7 @@ bool Monitor::Analyse() { Debug(1, "Staying in %s", State_Strings[state].c_str()); } - if ( state == ALARM ) { + if (state == ALARM) { last_alarm_count = analysis_image_count; } // This is needed so post_event_count counts after last alarmed frames while in ALARM not single alarmed frames while ALERT } else { // no score? @@ -2172,7 +2177,7 @@ bool Monitor::Analyse() { if (state == ALARM) { Info("%s: %03d - Gone into alert state", name, analysis_image_count); shared_data->state = state = ALERT; - } else if ( state == ALERT ) { + } else if (state == ALERT) { if ( ( analysis_image_count-last_alarm_count > post_event_count ) && @@ -2190,7 +2195,7 @@ bool Monitor::Analyse() { shared_data->state = state = TAPE; } } - } else if ( state == PREALARM ) { + } else if (state == PREALARM) { // Back to IDLE shared_data->state = state = ((function != MOCORD) ? IDLE : TAPE); } else { @@ -2198,19 +2203,19 @@ bool Monitor::Analyse() { State_Strings[state].c_str(), analysis_image_count, last_alarm_count, post_event_count, timestamp->tv_sec, video_store_data->recording.tv_sec, min_section_length); } - if ( Event::PreAlarmCount() ) + if (Event::PreAlarmCount()) Event::EmptyPreAlarmFrames(); } // end if score or not snap->score = score; - if ( state == PREALARM ) { + if (state == PREALARM) { // Generate analysis images if necessary - if ( (savejpegs > 1) and snap->image ) { - for ( int i = 0; i < n_zones; i++ ) { - if ( zones[i]->Alarmed() ) { - if ( zones[i]->AlarmImage() ) { - if ( ! snap->analysis_image ) + if ((savejpegs > 1) and snap->image) { + for (int i = 0; i < n_zones; i++) { + if (zones[i]->Alarmed()) { + if (zones[i]->AlarmImage()) { + if (!snap->analysis_image) snap->analysis_image = new Image(*(snap->image)); snap->analysis_image->Overlay(*(zones[i]->AlarmImage())); } @@ -2221,38 +2226,42 @@ bool Monitor::Analyse() { // incremement pre alarm image count //have_pre_alarmed_frames ++; Event::AddPreAlarmFrame(snap->image, *timestamp, score, nullptr); - } else if ( state == ALARM ) { - if ( ( savejpegs > 1 ) and snap->image ) { - for ( int i = 0; i < n_zones; i++ ) { - if ( zones[i]->Alarmed() ) { - if ( zones[i]->AlarmImage() ) { - if ( ! snap->analysis_image ) + } else if (state == ALARM) { + if ((savejpegs > 1 ) and snap->image) { + for (int i = 0; i < n_zones; i++) { + if (zones[i]->Alarmed()) { + if (zones[i]->AlarmImage()) { + if (!snap->analysis_image) snap->analysis_image = new Image(*(snap->image)); snap->analysis_image->Overlay(*(zones[i]->AlarmImage())); } - if ( config.record_event_stats ) + if (config.record_event_stats) zones[i]->RecordStats(event); } // end if zone is alarmed } // end foreach zone - } - if ( noteSetMap.size() > 0 ) - event->updateNotes(noteSetMap); - if ( section_length - && ( ( timestamp->tv_sec - video_store_data->recording.tv_sec ) >= section_length ) - && ! (image_count % fps_report_interval) - ) { - Warning("%s: %03d - event %" PRIu64 ", has exceeded desired section length. %d - %d = %d >= %d", - name, image_count, event->Id(), - timestamp->tv_sec, video_store_data->recording.tv_sec, - timestamp->tv_sec - video_store_data->recording.tv_sec, - section_length - ); - closeEvent(); - event = new Event(this, *timestamp, cause, noteSetMap); - shared_data->last_event_id = event->Id(); - //set up video store data - snprintf(video_store_data->event_file, sizeof(video_store_data->event_file), "%s", event->getEventFile()); - video_store_data->recording = event->StartTime(); + } + if (event) { + if (noteSetMap.size() > 0) + event->updateNotes(noteSetMap); + if ( section_length + && ( ( timestamp->tv_sec - video_store_data->recording.tv_sec ) >= section_length ) + && ! (image_count % fps_report_interval) + ) { + Warning("%s: %03d - event %" PRIu64 ", has exceeded desired section length. %d - %d = %d >= %d", + name, image_count, event->Id(), + timestamp->tv_sec, video_store_data->recording.tv_sec, + timestamp->tv_sec - video_store_data->recording.tv_sec, + section_length + ); + closeEvent(); + event = new Event(this, *timestamp, cause, noteSetMap); + shared_data->last_event_id = event->Id(); + //set up video store data + snprintf(video_store_data->event_file, sizeof(video_store_data->event_file), "%s", event->getEventFile()); + video_store_data->recording = event->StartTime(); + } + } else { + Error("ALARM but no event"); } } else if ( state == ALERT ) { @@ -2735,7 +2744,6 @@ int Monitor::Capture() { } // end Monitor::Capture bool Monitor::Decode() { - if (!decoder_it) decoder_it = packetqueue.get_video_it(true); ZMLockedPacket *packet_lock = packetqueue.get_packet(decoder_it); if (!packet_lock) return false; @@ -3146,8 +3154,17 @@ int Monitor::PrimeCapture() { } } // end if rtsp_server if (decoding_enabled) { - decoder = new DecoderThread(this); + if (!decoder_it) decoder_it = packetqueue.get_video_it(false); + if (!decoder) decoder = new DecoderThread(this); } + if (function != MONITOR) { + if (!analysis_it) analysis_it = packetqueue.get_video_it(false); + if (!analysis_thread) { + Debug(1, "Starting an analysis thread for monitor (%d)", id); + analysis_thread = new AnalysisThread(this); + } + } + } else { Debug(2, "Failed to prime %d", ret); } @@ -3157,17 +3174,22 @@ int Monitor::PrimeCapture() { int Monitor::PreCapture() const { return camera->PreCapture(); } int Monitor::PostCapture() const { return camera->PostCapture(); } int Monitor::Close() { - if ( decoder ) { - decoder->Stop(); + // Because the stream indexes may change we have to clear out the packetqueue + if (decoder) decoder->Stop(); + if (analysis_thread) analysis_thread->Stop(); + packetqueue.clear(); + if (decoder) { delete decoder; } + if (analysis_thread) { + delete analysis_thread; + } std::lock_guard lck(event_mutex); if (event) { Info("%s: image_count:%d - Closing event %" PRIu64 ", shutting down", name, image_count, event->Id()); closeEvent(); } if (camera) camera->Close(); - packetqueue.clear(); return 1; } From 9ca5f49d82072e54e0666ce5daa3aca0d684843e Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 17 Mar 2021 12:48:08 -0400 Subject: [PATCH 0205/1277] Move analysis_thread into Monitor --- src/zm_monitor.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/zm_monitor.h b/src/zm_monitor.h index dbd54d6cd..7969d2eef 100644 --- a/src/zm_monitor.h +++ b/src/zm_monitor.h @@ -22,6 +22,7 @@ #include "zm_define.h" #include "zm_camera.h" +#include "zm_analysis_thread.h" #include "zm_decoder_thread.h" #include "zm_event.h" #include "zm_fifo.h" @@ -376,9 +377,9 @@ protected: VideoStore *videoStore; PacketQueue packetqueue; packetqueue_iterator *analysis_it; + AnalysisThread *analysis_thread; packetqueue_iterator *decoder_it; - DecoderThread *decoder; - + DecoderThread *decoder; int n_zones; Zone **zones; From 0b4f04c4d5bb776725e77531e54d8b380154d82d Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 17 Mar 2021 12:48:42 -0400 Subject: [PATCH 0206/1277] notify in clear before taking lock to increase chance of other threads exiting. Handle terminate case in get_packet --- src/zm_packetqueue.cpp | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/src/zm_packetqueue.cpp b/src/zm_packetqueue.cpp index 4cbf23910..5dbc4ea00 100644 --- a/src/zm_packetqueue.cpp +++ b/src/zm_packetqueue.cpp @@ -56,11 +56,11 @@ int PacketQueue::addStream() { PacketQueue::~PacketQueue() { clear(); - if ( packet_counts ) { + if (packet_counts) { delete[] packet_counts; packet_counts = nullptr; } - while ( !iterators.empty() ) { + while (!iterators.empty()) { packetqueue_iterator *it = iterators.front(); iterators.pop_front(); delete it; @@ -322,6 +322,7 @@ unsigned int PacketQueue::clear(unsigned int frames_to_keep, int stream_id) { void PacketQueue::clear() { deleting = true; + condition.notify_all(); std::unique_lock lck(mutex); @@ -457,7 +458,7 @@ int PacketQueue::packet_count(int stream_id) { // Returns a packet. Packet will be locked ZMLockedPacket *PacketQueue::get_packet(packetqueue_iterator *it) { - if ( deleting or zm_terminate ) + if (deleting or zm_terminate) return nullptr; Debug(4, "Locking in get_packet using it %p queue end? %d, packet %p", @@ -471,7 +472,7 @@ ZMLockedPacket *PacketQueue::get_packet(packetqueue_iterator *it) { Debug(2, "waiting. Queue size %d it == end? %d", pktQueue.size(), (*it == pktQueue.end())); condition.wait(lck); } - if ( deleting or zm_terminate ) + if (deleting or zm_terminate) return nullptr; Debug(4, "get_packet using it %p queue end? %d, packet %p", @@ -489,6 +490,10 @@ ZMLockedPacket *PacketQueue::get_packet(packetqueue_iterator *it) { ZM_DUMP_PACKET(p->packet, ""); condition.wait(lck); } + if (deleting or zm_terminate) { + // packet may have been deleted so we can't delete the lp FIXME + return nullptr; + } Debug(2, "Locked packet, unlocking packetqueue mutex"); return lp; } // end ZMLockedPacket *PacketQueue::get_packet(it) From 6ab8bee581908e4f36c989109125f69fe441f146 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 17 Mar 2021 12:48:59 -0400 Subject: [PATCH 0207/1277] Increase debug level for fifo writing --- src/zm_fifo.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/zm_fifo.cpp b/src/zm_fifo.cpp index e7ff145af..35867cfc9 100644 --- a/src/zm_fifo.cpp +++ b/src/zm_fifo.cpp @@ -104,7 +104,7 @@ bool Fifo::close() { bool Fifo::writePacket(ZMPacket &packet) { if (!(outfile or open())) return false; - Debug(1, "Writing header ZM %u %" PRId64, packet.packet.size, packet.pts); + Debug(2, "Writing header ZM %u %" PRId64, packet.packet.size, packet.pts); // Going to write a brief header if ( fprintf(outfile, "ZM %u %" PRId64 "\n", packet.packet.size, packet.pts) < 0 ) { if (errno != EAGAIN) { From feafaa29bfc08094f4a4c9b065599b0f594007c8 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 17 Mar 2021 12:49:12 -0400 Subject: [PATCH 0208/1277] improve debug logging --- src/zm_rtsp_server_fifo_source.cpp | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/src/zm_rtsp_server_fifo_source.cpp b/src/zm_rtsp_server_fifo_source.cpp index cc3d443b1..1b3b4f73d 100644 --- a/src/zm_rtsp_server_fifo_source.cpp +++ b/src/zm_rtsp_server_fifo_source.cpp @@ -84,7 +84,7 @@ int ZoneMinderFifoSource::getNextFrame() { return -1; } - Debug(4, "%s bytes read %d bytes, buffer size %u", m_fifo.c_str(), bytes_read, m_buffer.size()); + Debug(1, "%s bytes read %d bytes, buffer size %u", m_fifo.c_str(), bytes_read, m_buffer.size()); while (m_buffer.size()) { unsigned int data_size = 0; int64_t pts; @@ -96,14 +96,14 @@ int ZoneMinderFifoSource::getNextFrame() { header_end = (unsigned char *)memchr(header_start, '\n', m_buffer.tail()-header_start); if (!header_end) { // Must not have enough data. So... keep all. - Debug(1, "Didn't find newline"); + Debug(1, "Didn't find newline buffer size is %d", m_buffer.size()); return 0; } unsigned int header_size = header_end-header_start; char *header = new char[header_size+1]; + strncpy(header, reinterpret_cast(header_start), header_size); header[header_size] = '\0'; - strncpy(header, reinterpret_cast(header_start), header_end-header_start); char *content_length_ptr = strchr(header, ' '); if (!content_length_ptr) { @@ -125,12 +125,12 @@ int ZoneMinderFifoSource::getNextFrame() { pts_ptr ++; data_size = atoi(content_length_ptr); pts = strtoll(pts_ptr, nullptr, 10); + Debug(4, "ZM Packet %s header_size %d packet size %u pts %" PRId64, header, header_size, data_size, pts); delete[] header; } else { - Debug(1, "ZM header not found %s.",m_buffer.head()); + Debug(1, "ZM header not found in %d of buffer:%s.", m_buffer.size(), m_buffer.head()); return 0; } - Debug(4, "ZM Packet size %u pts %" PRId64, data_size, pts); if (header_start != m_buffer) { Debug(4, "ZM Packet didn't start at beginning of buffer %u. %c%c", header_start-m_buffer.head(), m_buffer[0], m_buffer[1]); @@ -142,21 +142,21 @@ int ZoneMinderFifoSource::getNextFrame() { int bytes_needed = data_size - (m_buffer.size() - header_size); if (bytes_needed > 0) { Debug(4, "Need another %d bytes. Trying to read them", bytes_needed); - int bytes_read = m_buffer.read_into(m_fd, bytes_needed, {1,0}); + int bytes_read = m_buffer.read_into(m_fd, bytes_needed); if ( bytes_read != bytes_needed ) { - Debug(4, "Failed to read another %d bytes.", bytes_needed); + Debug(1, "Failed to read another %d bytes, got %d.", bytes_needed, bytes_read); return -1; } } - unsigned char *packet_start = m_buffer.head() + header_size; + m_buffer.consume(header_size); + unsigned char *packet_start = m_buffer.head(); size_t bytes_remaining = data_size; std::list< std::pair > framesList = this->splitFrames(packet_start, bytes_remaining); Debug(3, "Got %d frames, consuming %d bytes, remaining %d", framesList.size(), header_size + data_size, bytes_remaining); - m_buffer.consume(header_size + data_size); + m_buffer.consume(data_size); while (framesList.size()) { std::pair nal = framesList.front(); framesList.pop_front(); - PushFrame(nal.first, nal.second, pts); } } // end while m_buffer.size() From 2b34d09b846cfb3db16d3a34ad3db8961452acde Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 17 Mar 2021 12:49:50 -0400 Subject: [PATCH 0209/1277] Move analysis_thread into Monitor. Don't do extra gettimeofday if no delays are set. Fix status update on terminate --- src/zmc.cpp | 59 +++++++++++++++++++++-------------------------------- 1 file changed, 23 insertions(+), 36 deletions(-) diff --git a/src/zmc.cpp b/src/zmc.cpp index 19c8891c7..6a9a37c14 100644 --- a/src/zmc.cpp +++ b/src/zmc.cpp @@ -54,7 +54,6 @@ possible, this should run at more or less constant speed. */ #include "zm.h" -#include "zm_analysis_thread.h" #include "zm_camera.h" #include "zm_db.h" #include "zm_define.h" @@ -272,8 +271,6 @@ int main(int argc, char *argv[]) { } // end foreach monitor if (zm_terminate) break; - std::vector> analysis_threads = std::vector>(); - int *capture_delays = new int[monitors.size()]; int *alarm_capture_delays = new int[monitors.size()]; struct timeval * last_capture_times = new struct timeval[monitors.size()]; @@ -284,12 +281,6 @@ int main(int argc, char *argv[]) { alarm_capture_delays[i] = monitors[i]->GetAlarmCaptureDelay(); Debug(2, "capture delay(%u mSecs 1000/capture_fps) alarm delay(%u)", capture_delays[i], alarm_capture_delays[i]); - - Monitor::Function function = monitors[0]->GetFunction(); - if (function != Monitor::MONITOR) { - Debug(1, "Starting an analysis thread for monitor (%d)", monitors[i]->Id()); - analysis_threads.emplace_back(ZM::make_unique(monitors[i])); - } } struct timeval now; @@ -320,29 +311,31 @@ int main(int argc, char *argv[]) { break; } - gettimeofday(&now, nullptr); // capture_delay is the amount of time we should sleep in useconds to achieve the desired framerate. int delay = (monitors[i]->GetState() == Monitor::ALARM) ? alarm_capture_delays[i] : capture_delays[i]; - if (delay && last_capture_times[i].tv_sec) { - // DT_PREC_3 means that the value will be in thousands of a second - DELTA_TIMEVAL(delta_time, now, last_capture_times[i], DT_PREC_6); + if (delay) { + gettimeofday(&now, nullptr); + if (last_capture_times[i].tv_sec) { + // DT_PREC_3 means that the value will be in thousands of a second + DELTA_TIMEVAL(delta_time, now, last_capture_times[i], DT_PREC_6); - // You have to add back in the previous sleep time - sleep_time = delay - (delta_time.delta - sleep_time); - Debug(4, "Sleep time is %d from now:%d.%d last:%d.%d delta %d delay: %d", - sleep_time, - now.tv_sec, now.tv_usec, - last_capture_times[i].tv_sec, last_capture_times[i].tv_usec, - delta_time.delta, - delay - ); - - if (sleep_time > 0) { - Debug(4, "usleeping (%d)", sleep_time); - usleep(sleep_time); - } - } // end if has a last_capture time - last_capture_times[i] = now; + // You have to add back in the previous sleep time + sleep_time = delay - (delta_time.delta - sleep_time); + Debug(4, "Sleep time is %d from now:%d.%d last:%d.%d delta %d delay: %d", + sleep_time, + now.tv_sec, now.tv_usec, + last_capture_times[i].tv_sec, last_capture_times[i].tv_usec, + delta_time.delta, + delay + ); + + if (sleep_time > 0) { + Debug(4, "usleeping (%d)", sleep_time); + usleep(sleep_time); + } + } // end if has a last_capture time + last_capture_times[i] = now; + } // end if delay } // end foreach n_monitors if ((result < 0) or zm_reload) { @@ -351,16 +344,10 @@ int main(int argc, char *argv[]) { } } // end while ! zm_terminate and connected - for (std::unique_ptr &analysis_thread: analysis_threads) - analysis_thread->Stop(); - for (size_t i = 0; i < monitors.size(); i++) { monitors[i]->Close(); } - // Killoff the analysis threads. Don't need them spinning while we try to reconnect - analysis_threads.clear(); - delete [] alarm_capture_delays; delete [] capture_delays; delete [] last_capture_times; @@ -385,7 +372,7 @@ int main(int argc, char *argv[]) { for (std::shared_ptr &monitor : monitors) { static char sql[ZM_SQL_SML_BUFSIZ]; snprintf(sql, sizeof(sql), - "INSERT INTO Monitor_Status (MonitorId,Status) VALUES (%d, 'Connected') ON DUPLICATE KEY UPDATE Status='NotRunning'", + "INSERT INTO Monitor_Status (MonitorId,Status) VALUES (%d, 'NotRunning') ON DUPLICATE KEY UPDATE Status='NotRunning'", monitor->Id()); zmDbDo(sql); } From c39ec5873bcd2b310a4d21daef4cf76d4c000f71 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 17 Mar 2021 12:50:13 -0400 Subject: [PATCH 0210/1277] don't include zm_utils in decoder_thread --- src/zm_decoder_thread.cpp | 1 - 1 file changed, 1 deletion(-) diff --git a/src/zm_decoder_thread.cpp b/src/zm_decoder_thread.cpp index e8023e905..3d522830d 100644 --- a/src/zm_decoder_thread.cpp +++ b/src/zm_decoder_thread.cpp @@ -2,7 +2,6 @@ #include "zm_monitor.h" #include "zm_signal.h" -#include "zm_utils.h" //DecoderThread::DecoderThread(std::shared_ptr monitor) : DecoderThread::DecoderThread(Monitor * monitor) : From 284fe52b5f9da0c02d49da6849a1d710c477894b Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 17 Mar 2021 12:57:45 -0400 Subject: [PATCH 0211/1277] fix double stop/free of decoder and analysis threads --- src/zm_monitor.cpp | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 38a2814a3..b8d2fa4fc 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -3180,9 +3180,11 @@ int Monitor::Close() { packetqueue.clear(); if (decoder) { delete decoder; + decoder = nullptr; } if (analysis_thread) { delete analysis_thread; + analysis_thread = nullptr; } std::lock_guard lck(event_mutex); if (event) { From ec8e0f5997ca6790888e52de275044ad3c393991 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 17 Mar 2021 13:09:54 -0400 Subject: [PATCH 0212/1277] replace while(1) with while(not zm_terminate) so that these scripts exit cleanly --- scripts/zmcontrol.pl.in | 12 ++++++++++-- scripts/zmstats.pl.in | 11 +++++++++-- scripts/zmtrigger.pl.in | 11 +++++++++-- scripts/zmwatch.pl.in | 11 +++++++++-- 4 files changed, 37 insertions(+), 8 deletions(-) diff --git a/scripts/zmcontrol.pl.in b/scripts/zmcontrol.pl.in index 474148df3..d1f50cf43 100644 --- a/scripts/zmcontrol.pl.in +++ b/scripts/zmcontrol.pl.in @@ -138,6 +138,14 @@ if ( $options{command} ) { Fatal("Can't load ZoneMinder::Control::$protocol\n$Module::Load::Conditional::ERROR"); } + my $zm_terminate = 0; + sub TermHandler { + Info('Received TERM, exiting'); + $zm_terminate = 1; + } + $SIG{TERM} = \&TermHandler; + $SIG{INT} = \&TermHandler; + Info("Control server $id/$protocol starting at " .strftime('%y/%m/%d %H:%M:%S', localtime()) ); @@ -166,7 +174,7 @@ if ( $options{command} ) { my $win = $rin; my $ein = $win; my $timeout = MAX_COMMAND_WAIT; - while ( 1 ) { + while (!$zm_terminate) { my $nfound = select(my $rout = $rin, undef, undef, $timeout); if ( $nfound > 0 ) { if ( vec($rout, fileno(SERVER), 1) ) { @@ -202,7 +210,7 @@ if ( $options{command} ) { } else { Debug('Select timed out'); } - } # end while forever + } # end while !$zm_terminate Info("Control server $id/$protocol exiting"); unlink($sock_file); $control->close(); diff --git a/scripts/zmstats.pl.in b/scripts/zmstats.pl.in index 81fdd3129..7cdf93b5a 100644 --- a/scripts/zmstats.pl.in +++ b/scripts/zmstats.pl.in @@ -28,13 +28,20 @@ delete @ENV{qw(IFS CDPATH ENV BASH_ENV)}; logInit(); logSetSignal(); +my $zm_terminate = 0; +sub TermHandler { + Info('Received TERM, exiting'); + $zm_terminate = 1; +} +$SIG{TERM} = \&TermHandler; +$SIG{INT} = \&TermHandler; Info('Stats Daemon starting in '.START_DELAY.' seconds'); sleep(START_DELAY); my $dbh = zmDbConnect(); -while( 1 ) { +while (!$zm_terminate) { while ( ! ( $dbh and $dbh->ping() ) ) { Info('Reconnecting to db'); if ( !($dbh = zmDbConnect()) ) { @@ -95,7 +102,7 @@ while( 1 ) { zmDbDo('DELETE FROM Sessions WHERE access < ? LIMIT 100', time - (60*60*24*7)); sleep($Config{ZM_STATS_UPDATE_INTERVAL}); -} # end while (1) +} # end while (!$zm_terminate) Info('Stats Daemon exiting'); exit(); diff --git a/scripts/zmtrigger.pl.in b/scripts/zmtrigger.pl.in index 5fc632165..c409bde33 100644 --- a/scripts/zmtrigger.pl.in +++ b/scripts/zmtrigger.pl.in @@ -87,6 +87,13 @@ delete @ENV{qw(IFS CDPATH ENV BASH_ENV)}; logInit(); logSetSignal(); +my $zm_terminate = 0; +sub TermHandler { + Info('Received TERM, exiting'); + $zm_terminate = 1; +} +$SIG{TERM} = \&TermHandler; +$SIG{INT} = \&TermHandler; Info('Trigger daemon starting'); @@ -118,7 +125,7 @@ my $win = $rin; my $ein = $win; my $timeout = SELECT_TIMEOUT; my %actions; -while (1) { +while (!$zm_terminate) { $rin = $base_rin; # Add the file descriptors of any spawned connections foreach my $fileno ( keys %spawned_connections ) { @@ -312,7 +319,7 @@ while (1) { # zmDbConnect will ping and reconnect if neccessary $dbh = zmDbConnect(); -} # end while ( 1 ) +} # end while (!$zm_terminate) Info('Trigger daemon exiting'); exit; diff --git a/scripts/zmwatch.pl.in b/scripts/zmwatch.pl.in index d82c74b99..f1fbd16b4 100644 --- a/scripts/zmwatch.pl.in +++ b/scripts/zmwatch.pl.in @@ -68,6 +68,13 @@ delete @ENV{qw(IFS CDPATH ENV BASH_ENV)}; logInit(); logSetSignal(); +my $zm_terminate = 0; +sub TermHandler { + Info('Received TERM, exiting'); + $zm_terminate = 1; +} +$SIG{TERM} = \&TermHandler; +$SIG{INT} = \&TermHandler; Info('Watchdog starting, pausing for '.START_DELAY.' seconds'); sleep(START_DELAY); @@ -77,7 +84,7 @@ my $sql = $Config{ZM_SERVER_ID} ? 'SELECT * FROM Monitors WHERE ServerId=?' : 'S my $sth = $dbh->prepare_cached($sql) or Fatal("Can't prepare '$sql': ".$dbh->errstr()); -while( 1 ) { +while (!$zm_terminate) { while ( ! ( $dbh and $dbh->ping() ) ) { if ( ! ( $dbh = zmDbConnect() ) ) { sleep($Config{ZM_WATCH_CHECK_INTERVAL}); @@ -192,7 +199,7 @@ while( 1 ) { } # end foreach monitor sleep($Config{ZM_WATCH_CHECK_INTERVAL}); -} # end while (1) +} # end while (!$zm_terminate) Info("Watchdog exiting"); exit(); From 079d3361a2beeb8fe317b90532c791feead5cec2 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 17 Mar 2021 15:52:55 -0400 Subject: [PATCH 0213/1277] Rework to read content_length bytes at once. Micro-optimisation --- src/zm_remote_camera_http.cpp | 292 +++++++++++++++++----------------- 1 file changed, 146 insertions(+), 146 deletions(-) diff --git a/src/zm_remote_camera_http.cpp b/src/zm_remote_camera_http.cpp index d3ba95df0..3cfdc7412 100644 --- a/src/zm_remote_camera_http.cpp +++ b/src/zm_remote_camera_http.cpp @@ -202,7 +202,7 @@ int RemoteCameraHttp::SendRequest() { * > 0 is the # of bytes read. */ -int RemoteCameraHttp::ReadData( Buffer &buffer, unsigned int bytes_expected ) { +int RemoteCameraHttp::ReadData(Buffer &buffer, unsigned int bytes_expected) { fd_set rfds; FD_ZERO(&rfds); FD_SET(sd, &rfds); @@ -210,44 +210,44 @@ int RemoteCameraHttp::ReadData( Buffer &buffer, unsigned int bytes_expected ) { struct timeval temp_timeout = timeout; int n_found = select(sd+1, &rfds, nullptr, nullptr, &temp_timeout); - if( n_found == 0 ) { - Debug( 1, "Select timed out timeout was %d secs %d usecs", temp_timeout.tv_sec, temp_timeout.tv_usec ); + if (n_found == 0) { + Debug(1, "Select timed out timeout was %d secs %d usecs", temp_timeout.tv_sec, temp_timeout.tv_usec); int error = 0; - socklen_t len = sizeof (error); - int retval = getsockopt (sd, SOL_SOCKET, SO_ERROR, &error, &len); - if(retval != 0 ) { - Debug( 1, "error getting socket error code %s", strerror(retval) ); + socklen_t len = sizeof(error); + int retval = getsockopt(sd, SOL_SOCKET, SO_ERROR, &error, &len); + if (retval != 0) { + Debug(1, "error getting socket error code %s", strerror(retval)); } - if (error != 0 ) { + if (error != 0) { return -1; } // Why are we disconnecting? It's just a timeout, meaning that data wasn't available. //Disconnect(); return 0; - } else if ( n_found < 0) { + } else if (n_found < 0) { Error("Select error: %s", strerror(errno)); return -1; } unsigned int total_bytes_to_read = 0; - if ( bytes_expected ) { + if (bytes_expected) { total_bytes_to_read = bytes_expected; } else { - if ( ioctl( sd, FIONREAD, &total_bytes_to_read ) < 0 ) { - Error( "Can't ioctl(): %s", strerror(errno) ); + if (ioctl(sd, FIONREAD, &total_bytes_to_read) < 0) { + Error("Can't ioctl(): %s", strerror(errno) ); return -1; } - if ( total_bytes_to_read == 0 ) { - if ( mode == SINGLE_IMAGE ) { + if (total_bytes_to_read == 0) { + if (mode == SINGLE_IMAGE) { int error = 0; - socklen_t len = sizeof (error); - int retval = getsockopt( sd, SOL_SOCKET, SO_ERROR, &error, &len ); - if ( retval != 0 ) { - Debug( 1, "error getting socket error code %s", strerror(retval) ); + socklen_t len = sizeof(error); + int retval = getsockopt(sd, SOL_SOCKET, SO_ERROR, &error, &len); + if (retval != 0) { + Debug(1, "error getting socket error code %s", strerror(retval)); } - if ( error != 0 ) { + if (error != 0) { return -1; } // Case where we are grabbing a single jpg, but no content-length was given, so the expectation is that we read until close. @@ -261,38 +261,38 @@ int RemoteCameraHttp::ReadData( Buffer &buffer, unsigned int bytes_expected ) { } // There can be lots of bytes available. I've seen 4MB or more. This will vastly inflate our buffer size unnecessarily. - if ( total_bytes_to_read > ZM_NETWORK_BUFSIZ ) { + if (total_bytes_to_read > ZM_NETWORK_BUFSIZ) { total_bytes_to_read = ZM_NETWORK_BUFSIZ; - Debug(4, "Just getting 32K" ); + Debug(4, "Just getting 32K"); } else { - Debug(4, "Just getting %d", total_bytes_to_read ); + Debug(4, "Just getting %d", total_bytes_to_read); } } // end if bytes_expected or not - Debug( 4, "Expecting %d bytes", total_bytes_to_read ); + Debug(4, "Expecting %d bytes", total_bytes_to_read); int total_bytes_read = 0; do { - int bytes_read = buffer.read_into( sd, total_bytes_to_read ); - if ( bytes_read < 0 ) { - Error( "Read error: %s", strerror(errno) ); - return( -1 ); - } else if ( bytes_read == 0 ) { - Debug( 2, "Socket closed" ); + int bytes_read = buffer.read_into(sd, total_bytes_to_read); + if (bytes_read < 0) { + Error("Read error: %s", strerror(errno)); + return -1; + } else if (bytes_read == 0) { + Debug(2, "Socket closed"); //Disconnect(); // Disconnect is done outside of ReadData now. - return( -1 ); - } else if ( (unsigned int)bytes_read < total_bytes_to_read ) { - Error( "Incomplete read, expected %d, got %d", total_bytes_to_read, bytes_read ); - return( -1 ); + return -1; + } else if ((unsigned int)bytes_read < total_bytes_to_read) { + Debug(1, "Incomplete read, expected %d, got %d", total_bytes_to_read, bytes_read); + } else { + Debug(3, "Read %d bytes", bytes_read); } - Debug( 3, "Read %d bytes", bytes_read ); total_bytes_read += bytes_read; total_bytes_to_read -= bytes_read; - } while ( total_bytes_to_read ); + } while (total_bytes_to_read); - Debug(4, buffer); + Debug(4, "buffer: %s", buffer); return total_bytes_read; -} +} // end readData int RemoteCameraHttp::GetData() { time_t start_time = time(nullptr); @@ -497,36 +497,36 @@ int RemoteCameraHttp::GetResponse() { return( -1 ); } - if ( content_length ) { - while ( ((long)buffer.size() < content_length ) && ! zm_terminate ) { - Debug(3, "Need more data buffer %d < content length %d", buffer.size(), content_length ); - int bytes_read = GetData(); + if (content_length) { + while (((long)buffer.size() < content_length ) and !zm_terminate) { + Debug(3, "Need more data buffer %d < content length %d", buffer.size(), content_length); + int bytes_read = ReadData(buffer, content_length - buffer.size()); - if ( bytes_read < 0 ) { - Error( "Unable to read content" ); - return( -1 ); + if (bytes_read < 0) { + Error("Unable to read content"); + return -1; } bytes += bytes_read; } - Debug( 3, "Got end of image by length, content-length = %d", content_length ); + Debug(3, "Got end of image by length, content-length = %d", content_length); } else { - while ( !content_length ) { + while (!content_length) { buffer_len = GetData(); - if ( buffer_len < 0 ) { - Error( "Unable to read content" ); - return( -1 ); + if (buffer_len < 0) { + Error("Unable to read content"); + return -1; } bytes += buffer_len; static RegExpr *content_expr = 0; - if ( mode == MULTI_IMAGE ) { - if ( !content_expr ) { + if (mode == MULTI_IMAGE) { + if (!content_expr) { char content_pattern[256] = ""; - snprintf( content_pattern, sizeof(content_pattern), "^(.+?)(?:\r?\n)*(?:--)?%s\r?\n", content_boundary ); - content_expr = new RegExpr( content_pattern, PCRE_DOTALL ); + snprintf(content_pattern, sizeof(content_pattern), "^(.+?)(?:\r?\n)*(?:--)?%s\r?\n", content_boundary); + content_expr = new RegExpr(content_pattern, PCRE_DOTALL); } - if ( content_expr->Match( buffer, buffer.size() ) == 2 ) { + if (content_expr->Match( buffer, buffer.size()) == 2) { content_length = content_expr->MatchLength( 1 ); - Debug( 3, "Got end of image by pattern, content-length = %d", content_length ); + Debug(3, "Got end of image by pattern, content-length = %d", content_length); } } } @@ -623,7 +623,7 @@ int RemoteCameraHttp::GetResponse() { case HEADERCONT : { buffer_len = GetData(); - if ( buffer_len < 0 ) { + if (buffer_len < 0) { Error("Unable to read header"); return -1; } @@ -634,13 +634,13 @@ int RemoteCameraHttp::GetResponse() { int header_len = buffer.size(); bool all_headers = false; - while ( true ) { + while (true and !zm_terminate) { int crlf_len = memspn(header_ptr, "\r\n", header_len); - if ( n_headers ) { + if (n_headers) { if ( - (crlf_len == 2 && !strncmp(header_ptr, "\n\n", crlf_len )) + (crlf_len == 2 && !strncmp(header_ptr, "\n\n", crlf_len)) || - (crlf_len == 4 && !strncmp( header_ptr, "\r\n\r\n", crlf_len )) + (crlf_len == 4 && !strncmp(header_ptr, "\r\n\r\n", crlf_len)) ) { Debug(3, "Have double linefeed, done headers"); *header_ptr = '\0'; @@ -650,8 +650,8 @@ int RemoteCameraHttp::GetResponse() { break; } } - if ( crlf_len ) { - if ( header_len == crlf_len ) { + if (crlf_len) { + if (header_len == crlf_len) { break; } else { *header_ptr = '\0'; @@ -661,22 +661,22 @@ int RemoteCameraHttp::GetResponse() { } Debug(6, "%s", header_ptr); - if ( (crlf = mempbrk(header_ptr, "\r\n", header_len)) ) { + if ((crlf = mempbrk(header_ptr, "\r\n", header_len))) { //headers[n_headers++] = header_ptr; n_headers++; - if ( !http_header && (strncasecmp(header_ptr, http_match, http_match_len) == 0) ) { + if (!http_header && (strncasecmp(header_ptr, http_match, http_match_len) == 0)) { http_header = header_ptr+http_match_len; - Debug( 6, "Got http header '%s'", header_ptr ); + Debug(6, "Got http header '%s'", header_ptr); } else if ( !connection_header && (strncasecmp(header_ptr, connection_match, connection_match_len) == 0) ) { connection_header = header_ptr+connection_match_len; - Debug( 6, "Got connection header '%s'", header_ptr ); + Debug(6, "Got connection header '%s'", header_ptr); } else if ( !content_length_header && (strncasecmp(header_ptr, content_length_match, content_length_match_len) == 0) ) { content_length_header = header_ptr+content_length_match_len; - Debug( 6, "Got content length header '%s'", header_ptr ); + Debug(6, "Got content length header '%s'", header_ptr); } else if ( !authenticate_header && (strncasecmp(header_ptr, authenticate_match, authenticate_match_len) == 0) ) { authenticate_header = header_ptr; - Debug( 6, "Got authenticate header '%s'", header_ptr ); + Debug(6, "Got authenticate header '%s'", header_ptr); } else if ( !content_type_header && (strncasecmp(header_ptr, content_type_match, content_type_match_len) == 0) ) { content_type_header = header_ptr+content_type_match_len; Debug(6, "Got content type header '%s'", header_ptr); @@ -691,10 +691,10 @@ int RemoteCameraHttp::GetResponse() { } } // end while search for headers - if ( all_headers ) { + if (all_headers) { char *start_ptr, *end_ptr; - if ( !http_header ) { + if (!http_header) { Error("Unable to extract HTTP status from header"); return -1; } @@ -703,7 +703,7 @@ int RemoteCameraHttp::GetResponse() { end_ptr = start_ptr+strspn(start_ptr, "10."); // FIXME Why are we memsetting every time? Can we not do it once? - memset(http_version, 0, sizeof(http_version)); + //memset(http_version, 0, sizeof(http_version)); strncpy(http_version, start_ptr, end_ptr-start_ptr); start_ptr = end_ptr; @@ -718,12 +718,12 @@ int RemoteCameraHttp::GetResponse() { start_ptr += strspn(start_ptr, " "); strcpy(status_mesg, start_ptr); - if ( status == 401 ) { - if ( mNeedAuth ) { + if (status == 401) { + if (mNeedAuth) { Error("Failed authentication"); return -1; } - if ( !authenticate_header ) { + if (!authenticate_header) { Error("Failed authentication, but don't have an authentication header."); return -1; } @@ -732,7 +732,7 @@ int RemoteCameraHttp::GetResponse() { Debug(2, "Checking for digest auth in %s", authenticate_header); mAuthenticator->checkAuthResponse(Header); - if ( mAuthenticator->auth_method() == zm::AUTH_DIGEST ) { + if (mAuthenticator->auth_method() == zm::AUTH_DIGEST) { Debug(2, "Need Digest Authentication"); request = stringtf("GET %s HTTP/%s\r\n", path.c_str(), config.http_version); request += stringtf("User-Agent: %s/%s\r\n", config.http_ua, ZM_VERSION); @@ -747,26 +747,26 @@ int RemoteCameraHttp::GetResponse() { } else { Debug(2, "Need some other kind of Authentication"); } - } else if ( status < 200 || status > 299 ) { + } else if (status < 200 || status > 299) { Error("Invalid response status %s: %s", status_code, status_mesg); return -1; } Debug(3, "Got status '%d' (%s), http version %s", status, status_mesg, http_version); - if ( connection_header ) { + if (connection_header) { memset(connection_type, 0, sizeof(connection_type)); start_ptr = connection_header + strspn(connection_header, " "); // FIXME Should we not use strncpy? strcpy(connection_type, start_ptr); Debug(3, "Got connection '%s'", connection_type); } - if ( content_length_header ) { + if (content_length_header) { start_ptr = content_length_header + strspn(content_length_header, " "); - content_length = atoi( start_ptr ); + content_length = atoi(start_ptr); Debug(3, "Got content length '%d'", content_length); } - if ( content_type_header ) { - memset(content_type, 0, sizeof(content_type)); + if (content_type_header) { + //memset(content_type, 0, sizeof(content_type)); start_ptr = content_type_header + strspn(content_type_header, " "); if ( (end_ptr = strchr(start_ptr, ';')) ) { strncpy(content_type, start_ptr, end_ptr-start_ptr); @@ -774,7 +774,7 @@ int RemoteCameraHttp::GetResponse() { start_ptr = end_ptr + strspn(end_ptr, "; "); - if ( strncasecmp(start_ptr, boundary_match, boundary_match_len) == 0 ) { + if (strncasecmp(start_ptr, boundary_match, boundary_match_len) == 0) { start_ptr += boundary_match_len; start_ptr += strspn(start_ptr, "-"); content_boundary_len = sprintf(content_boundary, "--%s", start_ptr); @@ -788,24 +788,24 @@ int RemoteCameraHttp::GetResponse() { } } // end if content_type_header - if ( !strcasecmp(content_type, "image/jpeg") || !strcasecmp(content_type, "image/jpg") ) { + if (!strcasecmp(content_type, "image/jpeg") || !strcasecmp(content_type, "image/jpg")) { // Single image mode = SINGLE_IMAGE; format = JPEG; state = CONTENT; - } else if ( !strcasecmp(content_type, "image/x-rgb") ) { + } else if (!strcasecmp(content_type, "image/x-rgb")) { // Single image mode = SINGLE_IMAGE; format = X_RGB; state = CONTENT; - } else if ( !strcasecmp(content_type, "image/x-rgbz") ) { + } else if (!strcasecmp(content_type, "image/x-rgbz")) { // Single image mode = SINGLE_IMAGE; format = X_RGBZ; state = CONTENT; - } else if ( !strcasecmp(content_type, "multipart/x-mixed-replace") ) { + } else if (!strcasecmp(content_type, "multipart/x-mixed-replace")) { // Image stream, so start processing - if ( !content_boundary[0] ) { + if (!content_boundary[0]) { Error("No content boundary found in header '%s'", content_type_header); return -1; } @@ -817,8 +817,8 @@ int RemoteCameraHttp::GetResponse() { //// MPEG stream, coming soon! //} else { - Error( "Unrecognised content type '%s'", content_type ); - return( -1 ); + Error("Unrecognised content type '%s'", content_type); + return -1; } } else { Debug(3, "Unable to extract entire header from stream, continuing"); @@ -844,24 +844,24 @@ int RemoteCameraHttp::GetResponse() { int subheader_len = buffer.size(); bool all_headers = false; - while( true ) { - int crlf_len = memspn( subheader_ptr, "\r\n", subheader_len ); - if ( n_subheaders ) { - if ( (crlf_len == 2 && !strncmp( subheader_ptr, "\n\n", crlf_len )) || (crlf_len == 4 && !strncmp( subheader_ptr, "\r\n\r\n", crlf_len )) ) { + while (true and !zm_terminate) { + int crlf_len = memspn(subheader_ptr, "\r\n", subheader_len); + if (n_subheaders) { + if ( (crlf_len == 2 && !strncmp(subheader_ptr, "\n\n", crlf_len)) || (crlf_len == 4 && !strncmp( subheader_ptr, "\r\n\r\n", crlf_len )) ) { *subheader_ptr = '\0'; subheader_ptr += crlf_len; - subheader_len -= buffer.consume( subheader_ptr-(char *)buffer ); + subheader_len -= buffer.consume(subheader_ptr-(char *)buffer); all_headers = true; break; } } - if ( crlf_len ) { - if ( subheader_len == crlf_len ) { + if (crlf_len) { + if (subheader_len == crlf_len) { break; } else { *subheader_ptr = '\0'; subheader_ptr += crlf_len; - subheader_len -= buffer.consume( subheader_ptr-(char *)buffer ); + subheader_len -= buffer.consume(subheader_ptr-(char *)buffer); } } @@ -905,29 +905,29 @@ int RemoteCameraHttp::GetResponse() { } } - if ( all_headers && boundary_header ) { + if (all_headers && boundary_header) { char *start_ptr/*, *end_ptr*/; - Debug( 3, "Got boundary '%s'", boundary_header ); + Debug(3, "Got boundary '%s'", boundary_header); - if ( subcontent_length_header[0] ) { - start_ptr = subcontent_length_header + strspn( subcontent_length_header, " " ); - content_length = atoi( start_ptr ); - Debug( 3, "Got subcontent length '%d'", content_length ); + if (subcontent_length_header[0]) { + start_ptr = subcontent_length_header + strspn(subcontent_length_header, " "); + content_length = atoi(start_ptr); + Debug(3, "Got subcontent length '%d'", content_length); } - if ( subcontent_type_header[0] ) { - memset( content_type, 0, sizeof(content_type) ); - start_ptr = subcontent_type_header + strspn( subcontent_type_header, " " ); - strcpy( content_type, start_ptr ); - Debug( 3, "Got subcontent type '%s'", content_type ); + if (subcontent_type_header[0]) { + //memset(content_type, 0, sizeof(content_type)); + start_ptr = subcontent_type_header + strspn(subcontent_type_header, " "); + strcpy(content_type, start_ptr); + Debug(3, "Got subcontent type '%s'", content_type); } state = CONTENT; } else { - Debug( 3, "Unable to extract subheader from stream, retrying" ); + Debug(3, "Unable to extract subheader from stream, retrying"); buffer_len = GetData(); - if ( buffer_len < 0 ) { - Error( "Unable to read subheader" ); - return( -1 ); + if (buffer_len < 0) { + Error("Unable to read subheader"); + return -1; } bytes += buffer_len; state = SUBHEADERCONT; @@ -942,90 +942,90 @@ int RemoteCameraHttp::GetResponse() { *semicolon = '\0'; } - if ( !strcasecmp( content_type, "image/jpeg" ) || !strcasecmp( content_type, "image/jpg" ) ) { + if (!strcasecmp(content_type, "image/jpeg") || !strcasecmp(content_type, "image/jpg")) { format = JPEG; - } else if ( !strcasecmp( content_type, "image/x-rgb" ) ) { + } else if (!strcasecmp(content_type, "image/x-rgb")) { format = X_RGB; - } else if ( !strcasecmp( content_type, "image/x-rgbz" ) ) { + } else if (!strcasecmp(content_type, "image/x-rgbz")) { format = X_RGBZ; } else { - Error( "Found unsupported content type '%s'", content_type ); - return( -1 ); + Error("Found unsupported content type '%s'", content_type); + return -1; } // This is an early test for jpeg content, so we can bail early - if ( format == JPEG && buffer.size() >= 2 ) { - if ( buffer[0] != 0xff || buffer[1] != 0xd8 ) { - Error( "Found bogus jpeg header '%02x%02x'", buffer[0], buffer[1] ); - return( -1 ); + if (format == JPEG and buffer.size() >= 2) { + if (buffer[0] != 0xff or buffer[1] != 0xd8) { + Error("Found bogus jpeg header '%02x%02x'", buffer[0], buffer[1]); + return -1; } } - if ( content_length ) { - while ( ( (long)buffer.size() < content_length ) && ! zm_terminate ) { + if (content_length) { + while (((long)buffer.size() < content_length) and !zm_terminate) { Debug(4, "getting more data"); - int bytes_read = GetData(); - if ( bytes_read < 0 ) { + int bytes_read = ReadData(buffer, content_length-buffer.size()); + if (bytes_read < 0) { Error("Unable to read content"); return -1; } bytes += bytes_read; } - Debug( 3, "Got end of image by length, content-length = %d", content_length ); + Debug(3, "Got end of image by length, content-length = %d", content_length); } else { // Read until we find the end of image or the stream closes. - while ( !content_length && !zm_terminate ) { + while (!content_length && !zm_terminate) { Debug(4, "!content_length, ReadData"); - buffer_len = ReadData( buffer ); - if ( buffer_len < 0 ) { - Error( "Unable to read content" ); - return( -1 ); + buffer_len = ReadData(buffer); + if (buffer_len < 0) { + Error("Unable to read content"); + return -1; } bytes += buffer_len; int buffer_size = buffer.size(); - if ( buffer_len ) { + if (buffer_len) { // Got some data - if ( mode == MULTI_IMAGE ) { + if (mode == MULTI_IMAGE) { // Look for the boundary marker, determine content length using it's position - if ( char *start_ptr = (char *)memstr( (char *)buffer, "\r\n--", buffer_size ) ) { + if (char *start_ptr = (char *)memstr( (char *)buffer, "\r\n--", buffer_size)) { content_length = start_ptr - (char *)buffer; - Debug( 2, "Got end of image by pattern (crlf--), content-length = %d", content_length ); + Debug(2, "Got end of image by pattern (crlf--), content-length = %d", content_length); } else { - Debug( 2, "Did not find end of image by patten (crlf--) yet, content-length = %d", content_length ); + Debug(2, "Did not find end of image by patten (crlf--) yet, content-length = %d", content_length); } } // end if MULTI_IMAGE } else { content_length = buffer_size; - Debug( 2, "Got end of image by closure, content-length = %d", content_length ); - if ( mode == SINGLE_IMAGE ) { + Debug(2, "Got end of image by closure, content-length = %d", content_length); + if (mode == SINGLE_IMAGE) { char *end_ptr = (char *)buffer+buffer_size; // strip off any last line feeds - while( *end_ptr == '\r' || *end_ptr == '\n' ) { + while (*end_ptr == '\r' || *end_ptr == '\n') { content_length--; end_ptr--; } - if ( end_ptr != ((char *)buffer+buffer_size) ) { - Debug( 2, "Trimmed end of image, new content-length = %d", content_length ); + if (end_ptr != ((char *)buffer+buffer_size)) { + Debug(2, "Trimmed end of image, new content-length = %d", content_length); } } // end if SINGLE_IMAGE } // end if read some data } // end while ! content_length } // end if content_length - if ( mode == SINGLE_IMAGE ) { + if (mode == SINGLE_IMAGE) { state = HEADER; Disconnect(); } else { state = SUBHEADER; } - if ( format == JPEG && buffer.size() >= 2 ) { - if ( buffer[0] != 0xff || buffer[1] != 0xd8 ) { - Error( "Found bogus jpeg header '%02x%02x'", buffer[0], buffer[1] ); - return( -1 ); + if (format == JPEG && buffer.size() >= 2) { + if (buffer[0] != 0xff || buffer[1] != 0xd8) { + Error("Found bogus jpeg header '%02x%02x'", buffer[0], buffer[1]); + return -1; } } From fb28c6b365314ec35442bf3dc9f56add39ef4976 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 17 Mar 2021 15:53:14 -0400 Subject: [PATCH 0214/1277] Fix login in Decode for non-ffmpeg monitors --- src/zm_monitor.cpp | 122 ++++++++++++++++++++++----------------------- 1 file changed, 61 insertions(+), 61 deletions(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index b8d2fa4fc..46b5014a5 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -2551,7 +2551,7 @@ int Monitor::Capture() { } } else { captureResult = camera->Capture(*packet); - Debug(4, "Back from capture result=%d image %d", captureResult, image_count); + Debug(4, "Back from capture result=%d image count %d", captureResult, image_count); if ( captureResult < 0 ) { Debug(2, "failed capture"); @@ -2749,85 +2749,85 @@ bool Monitor::Decode() { if (!packet_lock) return false; ZMPacket *packet = packet_lock->packet_; packetqueue.increment_it(decoder_it); - if (packet->image or (packet->codec_type != AVMEDIA_TYPE_VIDEO)) { + if (packet->codec_type != AVMEDIA_TYPE_VIDEO) { delete packet_lock; return true; // Don't need decode } int ret = 0; - if (packet->packet.size and !packet->in_frame) { + if ((!packet->image) and packet->packet.size and !packet->in_frame) { // Allocate the image first so that it can be used by hwaccel // We don't actually care about camera colours, pixel order etc. We care about the desired settings // //capture_image = packet->image = new Image(width, height, camera->Colours(), camera->SubpixelOrder()); ret = packet->decode(camera->getVideoCodecContext()); - } else { - Debug(1, "No packet.size(%d) or packet->in_frame(%p). Not decoding", packet->packet.size, packet->in_frame); + if (ret > 0) { + if (packet->in_frame and !packet->image) { + packet->image = new Image(camera_width, camera_height, camera->Colours(), camera->SubpixelOrder()); + packet->get_image(); + } + } else { + Debug(1, "No packet.size(%d) or packet->in_frame(%p). Not decoding", packet->packet.size, packet->in_frame); + } } Image* capture_image = nullptr; unsigned int index = image_count % image_buffer_count; - if (ret > 0) { - if (packet->in_frame and !packet->image) { - packet->image = new Image(camera_width, camera_height, camera->Colours(), camera->SubpixelOrder()); - packet->get_image(); + + if (packet->image) { + capture_image = packet->image; + + /* Deinterlacing */ + if ( deinterlacing_value ) { + if ( deinterlacing_value == 1 ) { + capture_image->Deinterlace_Discard(); + } else if ( deinterlacing_value == 2 ) { + capture_image->Deinterlace_Linear(); + } else if ( deinterlacing_value == 3 ) { + capture_image->Deinterlace_Blend(); + } else if ( deinterlacing_value == 4 ) { + capture_image->Deinterlace_4Field(next_buffer.image, (deinterlacing>>8)&0xff); + } else if ( deinterlacing_value == 5 ) { + capture_image->Deinterlace_Blend_CustomRatio((deinterlacing>>8)&0xff); + } } - if (packet->image) { - capture_image = packet->image; - - /* Deinterlacing */ - if ( deinterlacing_value ) { - if ( deinterlacing_value == 1 ) { - capture_image->Deinterlace_Discard(); - } else if ( deinterlacing_value == 2 ) { - capture_image->Deinterlace_Linear(); - } else if ( deinterlacing_value == 3 ) { - capture_image->Deinterlace_Blend(); - } else if ( deinterlacing_value == 4 ) { - capture_image->Deinterlace_4Field(next_buffer.image, (deinterlacing>>8)&0xff); - } else if ( deinterlacing_value == 5 ) { - capture_image->Deinterlace_Blend_CustomRatio((deinterlacing>>8)&0xff); - } + if ( orientation != ROTATE_0 ) { + Debug(2, "Doing rotation"); + switch ( orientation ) { + case ROTATE_0 : + // No action required + break; + case ROTATE_90 : + case ROTATE_180 : + case ROTATE_270 : + capture_image->Rotate((orientation-1)*90); + break; + case FLIP_HORI : + case FLIP_VERT : + capture_image->Flip(orientation==FLIP_HORI); + break; } + } // end if have rotation - if ( orientation != ROTATE_0 ) { - Debug(2, "Doing rotation"); - switch ( orientation ) { - case ROTATE_0 : - // No action required - break; - case ROTATE_90 : - case ROTATE_180 : - case ROTATE_270 : - capture_image->Rotate((orientation-1)*90); - break; - case FLIP_HORI : - case FLIP_VERT : - capture_image->Flip(orientation==FLIP_HORI); - break; - } - } // end if have rotation + if (privacy_bitmask) { + Debug(1, "Applying privacy"); + capture_image->MaskPrivacy(privacy_bitmask); + } - if (privacy_bitmask) { - Debug(1, "Applying privacy"); - capture_image->MaskPrivacy(privacy_bitmask); - } + if (config.timestamp_on_capture) { + Debug(1, "Timestampprivacy"); + TimestampImage(packet->image, packet->timestamp); + } - if (config.timestamp_on_capture) { - Debug(1, "Timestampprivacy"); - TimestampImage(packet->image, packet->timestamp); - } - - if (!ref_image.Buffer()) { - // First image, so assign it to ref image - Debug(1, "Assigning ref image %dx%d size: %d", width, height, camera->ImageSize()); - ref_image.Assign(width, height, camera->Colours(), camera->SubpixelOrder(), - packet->image->Buffer(), camera->ImageSize()); - } - image_buffer[index].image->Assign(*(packet->image)); - *(image_buffer[index].timestamp) = *(packet->timestamp); - } // end if have image - } // end if did decoding + if (!ref_image.Buffer()) { + // First image, so assign it to ref image + Debug(1, "Assigning ref image %dx%d size: %d", width, height, camera->ImageSize()); + ref_image.Assign(width, height, camera->Colours(), camera->SubpixelOrder(), + packet->image->Buffer(), camera->ImageSize()); + } + image_buffer[index].image->Assign(*(packet->image)); + *(image_buffer[index].timestamp) = *(packet->timestamp); + } // end if have image packet->decoded = true; delete packet_lock; From ccb1bc1a7dbafa38b9693a5b31f6d96f9852940a Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 17 Mar 2021 16:11:31 -0400 Subject: [PATCH 0215/1277] Have to wait until we are finished with the packet before unlocking. --- src/zm_monitor.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 46b5014a5..bb7eda5e4 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -2829,11 +2829,11 @@ bool Monitor::Decode() { *(image_buffer[index].timestamp) = *(packet->timestamp); } // end if have image packet->decoded = true; - delete packet_lock; shared_data->signal = ( capture_image and signal_check_points ) ? CheckSignal(capture_image) : true; shared_data->last_write_index = index; shared_data->last_write_time = packet->timestamp->tv_sec; + delete packet_lock; return true; } // end bool Monitor::Decode() From 1b876f24f9d91ce86167beeee3f73bc0f34091f1 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 17 Mar 2021 17:14:46 -0400 Subject: [PATCH 0216/1277] Must have Id as well in order to know which monitor to control --- web/api/app/Controller/MonitorsController.php | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/web/api/app/Controller/MonitorsController.php b/web/api/app/Controller/MonitorsController.php index 2a6374222..184e17bb3 100644 --- a/web/api/app/Controller/MonitorsController.php +++ b/web/api/app/Controller/MonitorsController.php @@ -344,7 +344,7 @@ class MonitorsController extends AppController { public function daemonControl($id, $command, $daemon=null) { // Need to see if it is local or remote $monitor = $this->Monitor->find('first', array( - 'fields' => array('Type', 'Function', 'Device', 'ServerId'), + 'fields' => array('Id', 'Type', 'Function', 'Device', 'ServerId'), 'conditions' => array('Id' => $id) )); $monitor = $monitor['Monitor']; From f4506a8f350f7fbd7e4b785735d5a2a04e9d62ff Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Wed, 17 Mar 2021 23:41:00 -0400 Subject: [PATCH 0217/1277] We always need an analysis thread. --- src/zm_monitor.cpp | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index bb7eda5e4..b17bf97b1 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -3157,12 +3157,10 @@ int Monitor::PrimeCapture() { if (!decoder_it) decoder_it = packetqueue.get_video_it(false); if (!decoder) decoder = new DecoderThread(this); } - if (function != MONITOR) { - if (!analysis_it) analysis_it = packetqueue.get_video_it(false); - if (!analysis_thread) { - Debug(1, "Starting an analysis thread for monitor (%d)", id); - analysis_thread = new AnalysisThread(this); - } + if (!analysis_it) analysis_it = packetqueue.get_video_it(false); + if (!analysis_thread) { + Debug(1, "Starting an analysis thread for monitor (%d)", id); + analysis_thread = new AnalysisThread(this); } } else { From edefbfcad68bdf1c8ece4da0ac04f2e0ffd123bc Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 18 Mar 2021 09:24:27 -0400 Subject: [PATCH 0218/1277] Remove assumptions about Analysis being about motion detection. Fixes mem leaks in Monitor mode --- src/zm_monitor.cpp | 6 ++---- src/zm_monitor.h | 4 ---- 2 files changed, 2 insertions(+), 8 deletions(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index b17bf97b1..8fc0a507a 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -1797,10 +1797,6 @@ void Monitor::UpdateAnalysisFPS() { // If there isn't then we keep pre-event + alarm frames. = pre_event_count bool Monitor::Analyse() { - if (!Enabled()) { - Warning("Shouldn't be doing Analyse when not Enabled"); - return false; - } // if have event, send frames until we find a video packet, at which point do analysis. Adaptive skip should only affect which frames we do analysis on. @@ -3153,10 +3149,12 @@ int Monitor::PrimeCapture() { audio_fifo = new Fifo(shared_data->audio_fifo_path, true); } } // end if rtsp_server + if (decoding_enabled) { if (!decoder_it) decoder_it = packetqueue.get_video_it(false); if (!decoder) decoder = new DecoderThread(this); } + if (!analysis_it) analysis_it = packetqueue.get_video_it(false); if (!analysis_thread) { Debug(1, "Starting an analysis thread for monitor (%d)", id); diff --git a/src/zm_monitor.h b/src/zm_monitor.h index 7969d2eef..f12910597 100644 --- a/src/zm_monitor.h +++ b/src/zm_monitor.h @@ -445,10 +445,6 @@ public: } inline const char *EventPrefix() const { return event_prefix; } inline bool Ready() const { - if ( function <= MONITOR ) { - Error("Should not be calling Ready if the function doesn't include motion detection"); - return false; - } if ( image_count >= ready_count ) { return true; } From 4cb38a119e16e06c943df113d1606450ee658198 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 18 Mar 2021 14:09:15 -0400 Subject: [PATCH 0219/1277] Fix saving Filters and other objects. Apparently comparing 0 to NOW() doesn't work. --- web/includes/Object.php | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/web/includes/Object.php b/web/includes/Object.php index 9c9a3c095..da104e5db 100644 --- a/web/includes/Object.php +++ b/web/includes/Object.php @@ -302,14 +302,20 @@ class ZM_Object { # Set defaults. Note that we only replace "" with null, not other values # because for example if we want to clear TimestampFormat, we clear it, but the default is a string value foreach ( $this->defaults as $field => $default ) { - if ( (!property_exists($this, $field)) or ($this->{$field} === '') ) { - if ( is_array($default) ) { + if (!property_exists($this, $field)) { + if (is_array($default)) { $this->{$field} = $default['default']; - } else if ( $default == null ) { + } else { + $this->{$field} = $default; + } + } else if ($this->{$field} === '') { + if (is_array($default)) { + $this->{$field} = $default['default']; + } else if ($default == null) { $this->{$field} = $default; } } - } + } # end foreach default $fields = array_filter( $this->defaults, @@ -339,10 +345,10 @@ class ZM_Object { ') VALUES ('. implode(', ', array_map(function($field){return $this->$field() == 'NOW()' ? 'NOW()' : '?';}, $fields)).')'; - $values = array_values(array_map( - function($field){return $this->$field();}, - array_filter($fields, function($field){ return $this->$field() != 'NOW()';}) - )); + # For some reason comparing 0 to 'NOW()' returns false; So we do this. + $filtered = array_filter($fields, function($field){ return ( (!$this->$field()) or ($this->$field() != 'NOW()'));}); + $mapped = array_map(function($field){return $this->$field();}, $filtered); + $values = array_values($mapped); if ( dbQuery($sql, $values) ) { $this->{'Id'} = dbInsertId(); return true; From 7a4c34ec7e7c5d4a11f375553594e89af2d0cb92 Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Thu, 18 Mar 2021 12:59:14 +0100 Subject: [PATCH 0220/1277] RemoteCameraHttp: Fix a log message --- src/zm_remote_camera_http.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/zm_remote_camera_http.cpp b/src/zm_remote_camera_http.cpp index 3cfdc7412..010fc4fbc 100644 --- a/src/zm_remote_camera_http.cpp +++ b/src/zm_remote_camera_http.cpp @@ -289,7 +289,7 @@ int RemoteCameraHttp::ReadData(Buffer &buffer, unsigned int bytes_expected) { total_bytes_to_read -= bytes_read; } while (total_bytes_to_read); - Debug(4, "buffer: %s", buffer); + Debug(4, "buffer size: %d", static_cast(buffer)); return total_bytes_read; } // end readData From d1bbfdaf6b3067985a6669dedb9906fd4d1b8cb9 Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Thu, 18 Mar 2021 13:00:36 +0100 Subject: [PATCH 0221/1277] Build: Enable -Wconditionally-supported on GCC We want to have warnings if we use some implementation-specific features of GCC to be able to keep compatibility with clang. --- cmake/compiler/gcc/settings.cmake | 1 + 1 file changed, 1 insertion(+) diff --git a/cmake/compiler/gcc/settings.cmake b/cmake/compiler/gcc/settings.cmake index 87ff6e939..497db5a17 100644 --- a/cmake/compiler/gcc/settings.cmake +++ b/cmake/compiler/gcc/settings.cmake @@ -1,6 +1,7 @@ target_compile_options(zm-warning-interface INTERFACE -Wall + -Wconditionally-supported -Wextra -Wformat-security -Wno-cast-function-type From 7889b515d306f3ddaceac8a166463e6e2594df8f Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 18 Mar 2021 14:46:01 -0400 Subject: [PATCH 0222/1277] Add groovy and hirsuit builds --- utils/packpack/startpackpack.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/utils/packpack/startpackpack.sh b/utils/packpack/startpackpack.sh index d464089d3..75aee96f3 100755 --- a/utils/packpack/startpackpack.sh +++ b/utils/packpack/startpackpack.sh @@ -347,7 +347,7 @@ elif [ "${OS}" == "debian" ] || [ "${OS}" == "ubuntu" ] || [ "${OS}" == "raspbia setdebpkgname movecrud - if [ "${DIST}" == "focal" ] || [ "${DIST}" == "buster" ]; then + if [ "${DIST}" == "focal" ] || "${DIST}" == "groovy" || "${DIST}" == "hirsuit" || [ "${DIST}" == "buster" ]; then ln -sfT distros/ubuntu2004 debian elif [ "${DIST}" == "beowulf" ]; then ln -sfT distros/beowulf debian From e39c293a772ac62e1e99b0202113c7a7b4d9b1da Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 18 Mar 2021 15:58:45 -0400 Subject: [PATCH 0223/1277] fix eslint --- web/skins/classic/views/js/snapshot.js | 1 - 1 file changed, 1 deletion(-) diff --git a/web/skins/classic/views/js/snapshot.js b/web/skins/classic/views/js/snapshot.js index 9db096277..9ddefe0f7 100644 --- a/web/skins/classic/views/js/snapshot.js +++ b/web/skins/classic/views/js/snapshot.js @@ -27,7 +27,6 @@ function manageDelConfirmModalBtns() { } function initPage() { - // enable or disable buttons based on current selection and user rights /* renameBtn.prop('disabled', !canEdit.Events); From dd714a018fc7ac50adce3ad22440f1c56d417917 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 18 Mar 2021 16:18:21 -0400 Subject: [PATCH 0224/1277] add populating RtspServer --- utils/packpack/startpackpack.sh | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/utils/packpack/startpackpack.sh b/utils/packpack/startpackpack.sh index 75aee96f3..e18dbd127 100755 --- a/utils/packpack/startpackpack.sh +++ b/utils/packpack/startpackpack.sh @@ -117,6 +117,17 @@ commonprep () { exit 1 fi fi + + if [ -e "build/RtspServer" ]; then + echo "Found existing RtspServer..." + else + echo "Cloning RtspServer ..." + git clone https://github.com/ZoneMinder/RtspServer + if [ $? -ne 0 ]; then + echo "ERROR: CakePHP-Enum-Behavior tarball retreival failed..." + exit 1 + fi + fi } # Uncompress the submodule tarballs and move them into place @@ -137,6 +148,13 @@ movecrud () { rmdir web/api/app/Plugin/CakePHP-Enum-Behavior mv -f CakePHP-Enum-Behavior-${CEBVER} web/api/app/Plugin/CakePHP-Enum-Behavior fi + if [ -e "dep/RtspServer/CMakeLists.txt" ]; then + echo "RtspServer already installed..." + else + echo "Copying RtspServer..." + rmdir dep/RtspServer + cp -Rpd build/RtspServer dep/RtspServer + fi } # previsouly part of installzm.sh From 86e7c440872a3ed416b9338b39a126eef815962e Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 18 Mar 2021 16:20:19 -0400 Subject: [PATCH 0225/1277] add missing [] --- utils/packpack/startpackpack.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/utils/packpack/startpackpack.sh b/utils/packpack/startpackpack.sh index e18dbd127..950f5fcf2 100755 --- a/utils/packpack/startpackpack.sh +++ b/utils/packpack/startpackpack.sh @@ -365,7 +365,7 @@ elif [ "${OS}" == "debian" ] || [ "${OS}" == "ubuntu" ] || [ "${OS}" == "raspbia setdebpkgname movecrud - if [ "${DIST}" == "focal" ] || "${DIST}" == "groovy" || "${DIST}" == "hirsuit" || [ "${DIST}" == "buster" ]; then + if [ "${DIST}" == "focal" ] || [ "${DIST}" == "groovy" ] || [ "${DIST}" == "hirsuit" ] || [ "${DIST}" == "buster" ]; then ln -sfT distros/ubuntu2004 debian elif [ "${DIST}" == "beowulf" ]; then ln -sfT distros/beowulf debian From 70e61740d3438901d7128707765bd7a1207b995d Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 18 Mar 2021 16:41:12 -0400 Subject: [PATCH 0226/1277] Fix eslint --- web/skins/classic/views/js/montage.js | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/web/skins/classic/views/js/montage.js b/web/skins/classic/views/js/montage.js index 1a3c901d4..a1260effc 100644 --- a/web/skins/classic/views/js/montage.js +++ b/web/skins/classic/views/js/montage.js @@ -271,7 +271,9 @@ function reloadWebSite(ndx) { } function takeSnapshot() { - monitor_ids = monitorData.map( monitor=> { return 'monitor_ids[]='+monitor.id; }); + monitor_ids = monitorData.map((monitor)=>{ + return 'monitor_ids[]='+monitor.id; + }); window.location = '?view=snapshot&action=create&'+monitor_ids.join('&'); } From 53133ba0519ece3ff637ee454af1afa2f4f7deb8 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 18 Mar 2021 17:43:31 -0400 Subject: [PATCH 0227/1277] add token as an alternative to jwt_token --- src/zm_rtsp_server_authenticator.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/zm_rtsp_server_authenticator.h b/src/zm_rtsp_server_authenticator.h index 55a7784e7..9908327d3 100644 --- a/src/zm_rtsp_server_authenticator.h +++ b/src/zm_rtsp_server_authenticator.h @@ -46,6 +46,9 @@ class ZM_RtspServer_Authenticator : public xop::Authenticator { if (query.has("jwt_token")) { const QueryParameter *jwt_token = query.get("jwt_token"); user = zmLoadTokenUser(jwt_token->firstValue().c_str(), false); + } else if (query.has("token")) { + const QueryParameter *jwt_token = query.get("token"); + user = zmLoadTokenUser(jwt_token->firstValue().c_str(), false); } else if (strcmp(config.auth_relay, "none") == 0) { if (query.has("username")) { std::string username = query.get("username")->firstValue(); From 65ed17ab76c15ef70b48eac2f8d2895d28f1b8e8 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 18 Mar 2021 17:46:00 -0400 Subject: [PATCH 0228/1277] fix dest of clone of RtspServer --- utils/packpack/startpackpack.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/utils/packpack/startpackpack.sh b/utils/packpack/startpackpack.sh index 950f5fcf2..ef370aea5 100755 --- a/utils/packpack/startpackpack.sh +++ b/utils/packpack/startpackpack.sh @@ -122,7 +122,7 @@ commonprep () { echo "Found existing RtspServer..." else echo "Cloning RtspServer ..." - git clone https://github.com/ZoneMinder/RtspServer + git clone https://github.com/ZoneMinder/RtspServer build/RtspServer if [ $? -ne 0 ]; then echo "ERROR: CakePHP-Enum-Behavior tarball retreival failed..." exit 1 From b71db319a029ad775a2c5881b3d0b31a81a56ad3 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Thu, 18 Mar 2021 17:56:39 -0400 Subject: [PATCH 0229/1277] need to rm -r RtspServer --- utils/packpack/startpackpack.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/utils/packpack/startpackpack.sh b/utils/packpack/startpackpack.sh index ef370aea5..0ba1d2d07 100755 --- a/utils/packpack/startpackpack.sh +++ b/utils/packpack/startpackpack.sh @@ -124,7 +124,7 @@ commonprep () { echo "Cloning RtspServer ..." git clone https://github.com/ZoneMinder/RtspServer build/RtspServer if [ $? -ne 0 ]; then - echo "ERROR: CakePHP-Enum-Behavior tarball retreival failed..." + echo "ERROR: RtspServer clone failed..." exit 1 fi fi @@ -152,7 +152,7 @@ movecrud () { echo "RtspServer already installed..." else echo "Copying RtspServer..." - rmdir dep/RtspServer + rm -r dep/RtspServer cp -Rpd build/RtspServer dep/RtspServer fi } From 33f98a0d40e5867984526a6156bc6f1ca6b13098 Mon Sep 17 00:00:00 2001 From: SirLouen Date: Sat, 20 Mar 2021 20:02:52 +0100 Subject: [PATCH 0230/1277] Issue #3197 Add RECORD to Event_Close_Mode time --- src/zm_monitor.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 8fc0a507a..9fb398bdb 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -1979,7 +1979,7 @@ bool Monitor::Analyse() { Debug(2, "Have event %" PRIu64 " in mocord", event->Id()); if (section_length && ( ( timestamp->tv_sec - video_store_data->recording.tv_sec ) >= section_length ) - && ( (function == MOCORD && (event_close_mode != CLOSE_TIME)) || ! ( timestamp->tv_sec % section_length ) ) + && ( ( ( (function == MOCORD) || (function == RECORD) ) && (event_close_mode != CLOSE_TIME)) || ! ( timestamp->tv_sec % section_length ) ) ) { Info("%s: %03d - Closing event %" PRIu64 ", section end forced %d - %d = %d >= %d", From db7e9edcab43b5adae6550f2a590a7cc471f24ba Mon Sep 17 00:00:00 2001 From: SirLouen Date: Sun, 21 Mar 2021 00:00:36 +0100 Subject: [PATCH 0231/1277] Issue #3197 Improvement --- src/zm_monitor.cpp | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 9fb398bdb..58977dc2c 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -1977,11 +1977,10 @@ bool Monitor::Analyse() { if (event) { Debug(2, "Have event %" PRIu64 " in mocord", event->Id()); - if (section_length - && ( ( timestamp->tv_sec - video_store_data->recording.tv_sec ) >= section_length ) - && ( ( ( (function == MOCORD) || (function == RECORD) ) && (event_close_mode != CLOSE_TIME)) || ! ( timestamp->tv_sec % section_length ) ) - ) { - + if (section_length && ( ( timestamp->tv_sec - video_store_data->recording.tv_sec ) >= section_length ) + && ( ( (function == MOCORD) && (event_close_mode != CLOSE_TIME) ) || ( (function == RECORD) && (event_close_mode == CLOSE_TIME) ) + || ! ( timestamp->tv_sec % section_length ) ) ) + { Info("%s: %03d - Closing event %" PRIu64 ", section end forced %d - %d = %d >= %d", name, image_count, event->Id(), timestamp->tv_sec, video_store_data->recording.tv_sec, From a042e4bf77557bd5f59b9bbed6849e7b37070cb0 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sat, 20 Mar 2021 19:27:18 -0400 Subject: [PATCH 0232/1277] spacing --- src/zm_monitor.cpp | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 8fc0a507a..1c47f59e7 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -1947,8 +1947,8 @@ bool Monitor::Analyse() { motion_frame_skip, capture_fps, analysis_fps_limit); } - if ( !(analysis_image_count % (motion_frame_skip+1)) ) { - if ( snap->image ) { + if (!(analysis_image_count % (motion_frame_skip+1))) { + if (snap->image) { // Get new score. motion_score = DetectMotion(*(snap->image), zoneSet); @@ -1971,17 +1971,14 @@ bool Monitor::Analyse() { } // end if motion_score } // end if active and doing motion detection - if (function == RECORD or function == MOCORD) { // If doing record, check to see if we need to close the event or not. - if (event) { Debug(2, "Have event %" PRIu64 " in mocord", event->Id()); if (section_length && ( ( timestamp->tv_sec - video_store_data->recording.tv_sec ) >= section_length ) && ( (function == MOCORD && (event_close_mode != CLOSE_TIME)) || ! ( timestamp->tv_sec % section_length ) ) ) { - Info("%s: %03d - Closing event %" PRIu64 ", section end forced %d - %d = %d >= %d", name, image_count, event->Id(), timestamp->tv_sec, video_store_data->recording.tv_sec, From 68f9c7c9e67a34a123e20d4ef11b09e17a53352a Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sat, 20 Mar 2021 19:27:53 -0400 Subject: [PATCH 0233/1277] introduce a _last_error member to the object for reporting errors saving. --- web/includes/Object.php | 41 +++++++++++++++++++---------------------- 1 file changed, 19 insertions(+), 22 deletions(-) diff --git a/web/includes/Object.php b/web/includes/Object.php index da104e5db..4ed4fadf0 100644 --- a/web/includes/Object.php +++ b/web/includes/Object.php @@ -5,6 +5,7 @@ require_once('database.php'); $object_cache = array(); class ZM_Object { + protected $_last_error; public function __construct($IdOrRow = NULL) { $class = get_class($this); @@ -207,8 +208,8 @@ class ZM_Object { public function changes($new_values, $defaults=null) { $changes = array(); - if ( $defaults ) { - foreach ( $defaults as $field => $type ) { + if ($defaults) { + foreach ($defaults as $field => $type) { if ( isset($new_values[$field]) ) { # Will have already been handled above continue; @@ -247,47 +248,40 @@ class ZM_Object { } else if ( $this->$field() != $value ) { $changes[$field] = $value; } - } else if ( property_exists($this, $field) ) { + } else if (property_exists($this, $field)) { $type = (array_key_exists($field, $this->defaults) && is_array($this->defaults[$field])) ? $this->defaults[$field]['type'] : 'scalar'; - if ( $type == 'set' ) { + if ($type == 'set') { $old_value = is_array($this->$field) ? $this->$field : ($this->$field ? explode(',', $this->$field) : array()); $new_value = is_array($value) ? $value : ($value ? explode(',', $value) : array()); $diff = array_recursive_diff($old_value, $new_value); - if ( count($diff) ) { - $changes[$field] = $new_value; - } + if (count($diff)) $changes[$field] = $new_value; # Input might be a command separated string, or an array } else { - if ( array_key_exists($field, $this->defaults) && is_array($this->defaults[$field]) && isset($this->defaults[$field]['filter_regexp']) ) { - if ( is_array($this->defaults[$field]['filter_regexp']) ) { - foreach ( $this->defaults[$field]['filter_regexp'] as $regexp ) { + if (array_key_exists($field, $this->defaults) && is_array($this->defaults[$field]) && isset($this->defaults[$field]['filter_regexp'])) { + if (is_array($this->defaults[$field]['filter_regexp'])) { + foreach ($this->defaults[$field]['filter_regexp'] as $regexp) { $value = preg_replace($regexp, '', trim($value)); } } else { $value = preg_replace($this->defaults[$field]['filter_regexp'], '', trim($value)); } } - if ( $this->{$field} != $value ) { - $changes[$field] = $value; - } + if ($this->{$field} != $value) $changes[$field] = $value; } - } else if ( array_key_exists($field, $this->defaults) ) { - if ( is_array($this->defaults[$field]) and isset($this->defaults[$field]['default']) ) { + } else if (array_key_exists($field, $this->defaults)) { + if (is_array($this->defaults[$field]) and isset($this->defaults[$field]['default'])) { $default = $this->defaults[$field]['default']; } else { $default = $this->defaults[$field]; } - if ( $default != $value ) { - $changes[$field] = $value; - } + if ($default != $value) $changes[$field] = $value; } } # end foreach newvalue - return $changes; } # end public function changes @@ -335,8 +329,7 @@ class ZM_Object { $sql = 'UPDATE `'.$table.'` SET '.implode(', ', array_map(function($field) {return '`'.$field.'`=?';}, $fields)).' WHERE Id=?'; $values = array_map(function($field){ return $this->{$field};}, $fields); $values[] = $this->{'Id'}; - if ( dbQuery($sql, $values) ) - return true; + if (dbQuery($sql, $values)) return true; } else { unset($fields['Id']); @@ -349,11 +342,12 @@ class ZM_Object { $filtered = array_filter($fields, function($field){ return ( (!$this->$field()) or ($this->$field() != 'NOW()'));}); $mapped = array_map(function($field){return $this->$field();}, $filtered); $values = array_values($mapped); - if ( dbQuery($sql, $values) ) { + if (dbQuery($sql, $values)) { $this->{'Id'} = dbInsertId(); return true; } } + $this->_last_error = dbError($sql); return false; } // end function save @@ -427,5 +421,8 @@ class ZM_Object { public function remove_from_cache() { return ZM_Object::_remove_from_cache(get_class(), $this); } + public function get_last_error() { + return $this->_last_error; + } } # end class Object ?> From 8a1d13b6cd132aad8fe5c09d4e4c0d405cfb9865 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sun, 21 Mar 2021 09:17:03 -0400 Subject: [PATCH 0234/1277] perror => Error() --- src/zm_fifo.cpp | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/zm_fifo.cpp b/src/zm_fifo.cpp index 35867cfc9..10a8cc365 100644 --- a/src/zm_fifo.cpp +++ b/src/zm_fifo.cpp @@ -86,7 +86,7 @@ bool Fifo::open() { } long pipe_size = (long)fcntl(raw_fd, F_GETPIPE_SZ); if (pipe_size == -1) { - perror("get pipe size failed."); + Error("get pipe size failed."); } Debug(1, "default pipe size: %ld\n", pipe_size); #endif @@ -106,7 +106,7 @@ bool Fifo::writePacket(ZMPacket &packet) { Debug(2, "Writing header ZM %u %" PRId64, packet.packet.size, packet.pts); // Going to write a brief header - if ( fprintf(outfile, "ZM %u %" PRId64 "\n", packet.packet.size, packet.pts) < 0 ) { + if (fprintf(outfile, "ZM %u %" PRId64 "\n", packet.packet.size, packet.pts) < 0) { if (errno != EAGAIN) { Error("Problem during writing: %s", strerror(errno)); } else { @@ -121,6 +121,7 @@ bool Fifo::writePacket(ZMPacket &packet) { } return true; } + bool Fifo::writePacket(std::string filename, ZMPacket &packet) { bool on_blocking_abort = true; FILE *outfile = nullptr; From d0adaeaabe47aabb5867178a5b07fa21b7881fde Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sun, 21 Mar 2021 09:18:12 -0400 Subject: [PATCH 0235/1277] rework user saving action to use User object. Implement a duplicate username check. Deprecate php < 5.3 due to lack of bcrypt password hashing functions. Hence deprecate the use of mysql PASSWORD() --- web/includes/actions/user.php | 101 ++++++++++++++++------------------ 1 file changed, 48 insertions(+), 53 deletions(-) diff --git a/web/includes/actions/user.php b/web/includes/actions/user.php index 372108355..b13471231 100644 --- a/web/includes/actions/user.php +++ b/web/includes/actions/user.php @@ -18,77 +18,72 @@ // Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. // -if ( $action == 'Save' ) { - if ( canEdit('System') ) { - if ( !empty($_REQUEST['uid']) ) { - $dbUser = dbFetchOne('SELECT * FROM Users WHERE Id=?', NULL, array($_REQUEST['uid'])); - } else { - $dbUser = array(); - } +global $error_message; - $types = array(); - if ( isset($_REQUEST['newUser']['MonitorIds']) and is_array($_REQUEST['newUser']['MonitorIds']) ) +if ($action == 'Save') { + require_once('includes/User.php'); + $uid = isset($_REQUEST['uid']) ? validInt($_REQUEST['uid']) : 0; + $dbUser = new ZM\User($uid); + + if (canEdit('System')) { + # Need to check for uniqueness of Username + $user_with_my_username = ZM\User::find_one(array('Username'=>$_REQUEST['newUser']['Username'])); + if ($user_with_my_username and + ( ( $uid and ($user_with_my_username->Id() != $uid) ) or !$uid) + ) { + $error_message = 'There already exists a user with this Username
'; + unset($_REQUEST['redirect']); + return; + } + # What other tests should we do? + + if (isset($_REQUEST['newUser']['MonitorIds']) and is_array($_REQUEST['newUser']['MonitorIds'])) $_REQUEST['newUser']['MonitorIds'] = implode(',', $_REQUEST['newUser']['MonitorIds']); - if ( !$_REQUEST['newUser']['Password'] ) + if (!empty($_REQUEST['newUser']['Password'])) { + $_REQUEST['newUser']['Password'] = password_hash($_REQUEST['newUser']['Password'], PASSWORD_BCRYPT); + } else { unset($_REQUEST['newUser']['Password']); - - $changes = getFormChanges($dbUser, $_REQUEST['newUser'], $types); - - if ( isset($_REQUEST['newUser']['Password']) ) { - if ( function_exists('password_hash') ) { - $pass_hash = '"'.password_hash($_REQUEST['newUser']['Password'], PASSWORD_BCRYPT).'"'; - } else { - $pass_hash = ' PASSWORD('.dbEscape($_REQUEST['newUser']['Password']).') '; - ZM\Info('Cannot use bcrypt as you are using PHP < 5.3'); - } - - if ( $_REQUEST['newUser']['Password'] ) { - $changes['Password'] = 'Password = '.$pass_hash; - } else { - unset($changes['Password']); - } } + $changes = $dbUser->changes($_REQUEST['newUser']); + ZM\Debug("Changes: " . print_r($changes, true)); - if ( count($changes) ) { - if ( !empty($_REQUEST['uid']) ) { - dbQuery('UPDATE Users SET '.implode(', ', $changes).' WHERE Id = ?', array($_REQUEST['uid'])); - # If we are updating the logged in user, then update our session user data. - if ( $user and ( $dbUser['Username'] == $user['Username'] ) ) { + if (count($changes)) { + if (!$dbUser->save($changes)) { + $error_message = $dbUser->get_last_error(); + unset($_REQUEST['redirect']); + return; + } + + if ($uid) { + if ($user and ($dbUser->Username() == $user['Username'])) { # We are the logged in user, need to update the $user object and generate a new auth_hash $sql = 'SELECT * FROM Users WHERE Enabled=1 AND Id=?'; - $user = dbFetchOne($sql, NULL, array($_REQUEST['uid'])); + $user = dbFetchOne($sql, NULL, array($uid)); # Have to update auth hash in session zm_session_start(); generateAuthHash(ZM_AUTH_HASH_IPS, true); session_write_close(); } - } else { - dbQuery('INSERT INTO Users SET '.implode(', ', $changes)); } } # end if changes - } else if ( ZM_USER_SELF_EDIT and ( $_REQUEST['uid'] == $user['Id'] ) ) { - $uid = $user['Id']; - - $dbUser = dbFetchOne('SELECT Id, Password, Language FROM Users WHERE Id = ?', NULL, array($uid)); - - $types = array(); - $changes = getFormChanges($dbUser, $_REQUEST['newUser'], $types); - - if ( function_exists('password_hash') ) { - $pass_hash = '"'.password_hash($_REQUEST['newUser']['Password'], PASSWORD_BCRYPT).'"'; + } else if (ZM_USER_SELF_EDIT and ($uid == $user['Id'])) { + if (!empty($_REQUEST['newUser']['Password'])) { + $_REQUEST['newUser']['Password'] = password_hash($_REQUEST['newUser']['Password'], PASSWORD_BCRYPT); } else { - $pass_hash = ' PASSWORD('.dbEscape($_REQUEST['newUser']['Password']).') '; - ZM\Info ('Cannot use bcrypt as you are using PHP < 5.3'); + unset($_REQUEST['newUser']['Password']); } + $fields = array('Password'=>'', 'Language'=>'', 'HomeView'=>''); + ZM\Debug("changes: ".print_r(array_intersect_key($_REQUEST['newUser'], $fields),true)); + $changes = $dbUser->changes(array_intersect_key($_REQUEST['newUser'], $fields)); + ZM\Debug("changes: ".print_r($changes, true)); - if ( !empty($_REQUEST['newUser']['Password']) ) { - $changes['Password'] = 'Password = '.$pass_hash; - } else { - unset($changes['Password']); - } - if ( count($changes) ) { - dbQuery('UPDATE Users SET '.implode(', ', $changes).' WHERE Id=?', array($uid)); + if (count($changes)) { + if (!$dbUser->save($changes)) { + $error_message = $dbUser->get_last_error(); + unset($_REQUEST['redirect']); + return; + } # We are the logged in user, need to update the $user object and generate a new auth_hash $sql = 'SELECT * FROM Users WHERE Enabled=1 AND Id=?'; From cc455e5d7402999f037bbfee4ed1b4e07fdfa7c9 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sun, 21 Mar 2021 09:18:47 -0400 Subject: [PATCH 0236/1277] fix require=>require_once for User.php. Use getBodyTopHTML so that we get the error reporting --- web/skins/classic/views/user.php | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/web/skins/classic/views/user.php b/web/skins/classic/views/user.php index 981b205e2..e14c2e1f2 100644 --- a/web/skins/classic/views/user.php +++ b/web/skins/classic/views/user.php @@ -25,7 +25,7 @@ if ( !canEdit('System') && !$selfEdit ) { return; } -require('includes/User.php'); +require_once('includes/User.php'); if ( $_REQUEST['uid'] ) { if ( !($newUser = new ZM\User($_REQUEST['uid'])) ) { @@ -54,9 +54,9 @@ foreach ( dbFetchAll($sql) as $monitor ) { $focusWindow = true; xhtmlHeaders(__FILE__, translate('User').' - '.$newUser->Username()); +echo getBodyTopHTML(); +echo getNavBarHTML(); ?> - -
From 284837d53669df5b3f75b88b7651c552875a24c9 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sun, 21 Mar 2021 09:19:21 -0400 Subject: [PATCH 0237/1277] quotes, spaces. Also move setting redirect to to where we actually do the redirect so that actions can remove the redirect if there was an error to report. --- web/index.php | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/web/index.php b/web/index.php index bf71b059c..9683e5b31 100644 --- a/web/index.php +++ b/web/index.php @@ -98,7 +98,7 @@ if ( isset($_GET['skin']) ) { $skin = 'classic'; } -if ( ! is_dir("skins/$skin") ) { +if (!is_dir('skins/'.$skin) ) { $skins = array_map('basename', glob('skins/*', GLOB_ONLYDIR)); if ( !in_array($skin, $skins) ) { @@ -117,10 +117,10 @@ if ( isset($_GET['css']) ) { $css = 'classic'; } -if ( !is_dir("skins/$skin/css/$css") ) { +if (!is_dir("skins/$skin/css/$css")) { $css_skins = array_map('basename', glob('skins/'.$skin.'/css/*', GLOB_ONLYDIR)); - if ( count($css_skins) ) { - if ( !in_array($css, $css_skins) ) { + if (count($css_skins)) { + if (!in_array($css, $css_skins)) { ZM\Error("Invalid skin css '$css' setting to " . $css_skins[0]); $css = $css_skins[0]; } else { @@ -137,7 +137,7 @@ define('ZM_SKIN_PATH', "skins/$skin"); define('ZM_SKIN_NAME', $skin); $skinBase = array(); // To allow for inheritance of skins -if ( !file_exists(ZM_SKIN_PATH) ) +if (!file_exists(ZM_SKIN_PATH)) ZM\Fatal("Invalid skin '$skin'"); $skinBase[] = $skin; @@ -183,9 +183,6 @@ $user = null; if ( isset($_REQUEST['view']) ) $view = detaintPath($_REQUEST['view']); -if ( isset($_REQUEST['redirect']) ) - $redirect = '?view='.detaintPath($_REQUEST['redirect']); - # Add CSP Headers $cspNonce = bin2hex(zm_random_bytes(16)); @@ -265,6 +262,8 @@ if ( ZM_OPT_USE_AUTH and (!isset($user)) and ($view != 'login') and ($view != 'n $request = null; } +if ( isset($_REQUEST['redirect']) ) + $redirect = '?view='.detaintPath($_REQUEST['redirect']); if ( $redirect ) { ZM\Debug("Redirecting to $redirect"); From a57473a146b6222cd281fe5d0fe295ee81519ba1 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sun, 21 Mar 2021 09:19:31 -0400 Subject: [PATCH 0238/1277] remove debug --- web/skins/classic/views/options.php | 1 - 1 file changed, 1 deletion(-) diff --git a/web/skins/classic/views/options.php b/web/skins/classic/views/options.php index 2be9e9b52..2745979b7 100644 --- a/web/skins/classic/views/options.php +++ b/web/skins/classic/views/options.php @@ -156,7 +156,6 @@ foreach ( array_map('basename', glob('skins/'.$skin.'/css/*', GLOB_ONLYDIR)) as $userMonitors[] = $monitors[$monitorId]['Name']; } } - ZM\Debug("monitors: ".$user_row['Username'] . ' ' . $user_row['MonitorIds']. ' :' . print_r($userMonitors, true)); ?> From 07cf4697148b6f9a96fb7f869d0308be5bbb57e4 Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Sun, 21 Mar 2021 14:33:20 +0100 Subject: [PATCH 0239/1277] dep: Update RtspServer --- dep/RtspServer | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dep/RtspServer b/dep/RtspServer index 20846b25f..c81ec629f 160000 --- a/dep/RtspServer +++ b/dep/RtspServer @@ -1 +1 @@ -Subproject commit 20846b25ffc0c9a1de1b6701ca99425ef39e9f3f +Subproject commit c81ec629f091b77e2948b1e6113064de7e9bc9e3 From 6d9a4ed661b94300865ab48f7ed08835d50b31f8 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sun, 21 Mar 2021 12:28:33 -0400 Subject: [PATCH 0240/1277] If the analysis thread is falling behind, we can't count the packets after it in the number of packets to keep in queue. So figure out how many there are and add that to the max_video_packet count to keep so that we always have enough to satisfy pre_event_count --- src/zm_packetqueue.cpp | 36 ++++++++++++++++++++++++------------ 1 file changed, 24 insertions(+), 12 deletions(-) diff --git a/src/zm_packetqueue.cpp b/src/zm_packetqueue.cpp index 5dbc4ea00..7cc19debf 100644 --- a/src/zm_packetqueue.cpp +++ b/src/zm_packetqueue.cpp @@ -120,7 +120,7 @@ void PacketQueue::clearPackets(ZMPacket *add_packet) { // // So start at the beginning, counting video packets until the next keyframe. // Then if deleting those packets doesn't break 1 and 2, then go ahead and delete them. - if ( ! ( + if (! ( add_packet->packet.stream_index == video_stream_id and add_packet->keyframe @@ -138,6 +138,20 @@ void PacketQueue::clearPackets(ZMPacket *add_packet) { } std::unique_lock lck(mutex); + // If ananlysis_it isn't at the end, we need to keep that many additional packets + int tail_count = 0; + if (pktQueue.back() != add_packet) { + Debug(1, "Ours is not the back"); + packetqueue_iterator it = pktQueue.end(); + --it; + while (*it != add_packet) { + if ((*it)->packet.stream_index == video_stream_id) + ++tail_count; + --it; + } + } + Debug(1, "Tail count is %d", tail_count); + packetqueue_iterator it = pktQueue.begin(); packetqueue_iterator next_front = pktQueue.begin(); int video_packets_to_delete = 0; // This is a count of how many packets we will delete so we know when to stop looking @@ -145,33 +159,31 @@ void PacketQueue::clearPackets(ZMPacket *add_packet) { // First packet is special because we know it is a video keyframe and only need to check for lock ZMPacket *zm_packet = *it; ZMLockedPacket *lp = new ZMLockedPacket(zm_packet); - if ( lp->trylock() ) { + if (lp->trylock()) { ++it; delete lp; // Since we have many packets in the queue, we should NOT be pointing at end so don't need to test for that - while ( *it != add_packet ) { + while (*it != add_packet) { zm_packet = *it; lp = new ZMLockedPacket(zm_packet); - if ( !lp->trylock() ) { - break; - } + if (!lp->trylock()) break; delete lp; - if ( is_there_an_iterator_pointing_to_packet(zm_packet) ) { + if (is_there_an_iterator_pointing_to_packet(zm_packet)) { Warning("Found iterator at beginning of queue. Some thread isn't keeping up"); break; } - if ( zm_packet->packet.stream_index == video_stream_id ) { - if ( zm_packet->keyframe ) { + if (zm_packet->packet.stream_index == video_stream_id) { + if (zm_packet->keyframe) { Debug(3, "Have a video keyframe so setting next front to it"); next_front = it; } ++video_packets_to_delete; - Debug(4, "Counted %d video packets. Which would leave %d in packetqueue", - video_packets_to_delete, packet_counts[video_stream_id]-video_packets_to_delete); - if (packet_counts[video_stream_id] - video_packets_to_delete <= max_video_packet_count) { + Debug(4, "Counted %d video packets. Which would leave %d in packetqueue tail count is %d", + video_packets_to_delete, packet_counts[video_stream_id]-video_packets_to_delete, tail_count); + if (packet_counts[video_stream_id] - video_packets_to_delete <= max_video_packet_count + tail_count) { break; } } From 8a1284e2fa2a7c3ff09f27bc53d377044fadccf4 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sun, 21 Mar 2021 12:30:56 -0400 Subject: [PATCH 0241/1277] Can't use a decimal step. Has to be any because browsers suck. --- web/skins/classic/views/js/zone.js | 2 +- web/skins/classic/views/zone.php | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/web/skins/classic/views/js/zone.js b/web/skins/classic/views/js/zone.js index 67ff28d1d..6282cbf8a 100644 --- a/web/skins/classic/views/js/zone.js +++ b/web/skins/classic/views/js/zone.js @@ -202,7 +202,7 @@ function toPercent(field, maxValue) { field.value = 100; } } - field.setAttribute('step', 0.01); + field.setAttribute('step', 'any'); field.setAttribute('max', 100); } diff --git a/web/skins/classic/views/zone.php b/web/skins/classic/views/zone.php index 2c41a885c..7a4761a50 100644 --- a/web/skins/classic/views/zone.php +++ b/web/skins/classic/views/zone.php @@ -206,7 +206,7 @@ if ( count($other_zones) ) { array('data-on-change'=>'applyZoneUnits', 'id'=>'newZone[Units]') ); # Used later for number inputs - $step = $newZone['Units'] == 'Percent' ? ' step="0.01" max="100"' : ''; + $step = $newZone['Units'] == 'Percent' ? ' step="any" max="100"' : ''; ?> From 7f9c9c6624b4ac3cf74fd986ea1d831a651362f3 Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Sun, 21 Mar 2021 21:40:41 +0100 Subject: [PATCH 0242/1277] web: make eslint happy --- web/skins/classic/views/js/events.js | 4 ++-- web/skins/classic/views/js/filter.js | 6 ++---- web/skins/classic/views/js/snapshots.js | 3 +-- 3 files changed, 5 insertions(+), 8 deletions(-) diff --git a/web/skins/classic/views/js/events.js b/web/skins/classic/views/js/events.js index 40adaba20..9f6d34f75 100644 --- a/web/skins/classic/views/js/events.js +++ b/web/skins/classic/views/js/events.js @@ -58,8 +58,8 @@ function processRows(rows) { var emailed = row.Emailed == yesString ? emailedString : ''; row.Id = '' + eid + ''; - row.Name = '' + row.Name + '' - + '
' + archived + emailed + '
'; + row.Name = '' + row.Name + '' + + '
' + archived + emailed + '
'; if ( canEdit.Monitors ) row.Monitor = '' + row.Monitor + ''; if ( canEdit.Events ) row.Cause = '' + row.Cause + ''; if ( row.Notes.indexOf('detected:') >= 0 ) { diff --git a/web/skins/classic/views/js/filter.js b/web/skins/classic/views/js/filter.js index c59e755ff..61c01a2f2 100644 --- a/web/skins/classic/views/js/filter.js +++ b/web/skins/classic/views/js/filter.js @@ -40,10 +40,8 @@ function validateForm(form) { var have_endtime_term = false; for ( var i = 0; i < rows.length; i++ ) { if ( - ( form.elements['filter[Query][terms][' + i + '][attr]'].value == 'EndDateTime' ) - || - ( form.elements['filter[Query][terms][' + i + '][attr]'].value == 'EndTime' ) - || + ( form.elements['filter[Query][terms][' + i + '][attr]'].value == 'EndDateTime' ) || + ( form.elements['filter[Query][terms][' + i + '][attr]'].value == 'EndTime' ) || ( form.elements['filter[Query][terms][' + i + '][attr]'].value == 'EndDate' ) ) { have_endtime_term = true; diff --git a/web/skins/classic/views/js/snapshots.js b/web/skins/classic/views/js/snapshots.js index dff843374..b0e83001d 100644 --- a/web/skins/classic/views/js/snapshots.js +++ b/web/skins/classic/views/js/snapshots.js @@ -48,7 +48,6 @@ function ajaxRequest(params) { function processRows(rows) { $j.each(rows, function(ndx, row) { - var id = row.Id; row.Id = '' + id + ''; row.Name = '' + row.Name + ''; @@ -192,7 +191,7 @@ function initPage() { window.location.reload(true); }); -/* + /* // Manage the ARCHIVE button document.getElementById("archiveBtn").addEventListener("click", function onArchiveClick(evt) { var selections = getIdSelections(); From 7e86e1ef401cdf17193739ecf64afd5368231daa Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Sun, 7 Mar 2021 16:17:17 +0100 Subject: [PATCH 0243/1277] utils: Make TimevalToString thread-safe --- src/zm_utils.cpp | 31 ++++++++++++++++--------------- src/zm_utils.h | 2 +- 2 files changed, 17 insertions(+), 16 deletions(-) diff --git a/src/zm_utils.cpp b/src/zm_utils.cpp index 3fce11d1c..71f99ef9e 100644 --- a/src/zm_utils.cpp +++ b/src/zm_utils.cpp @@ -22,7 +22,7 @@ #include "zm_config.h" #include "zm_logger.h" #include -#include +#include #include #include /* Definition of AT_* constants */ #include @@ -43,7 +43,7 @@ std::string trimSet(std::string str, std::string trimset) { // Trim Both leading and trailing sets size_t startpos = str.find_first_not_of(trimset); // Find the first character position after excluding leading blank spaces size_t endpos = str.find_last_not_of(trimset); // Find the first character position from reverse af - + // if all spaces or empty return an empty string if ( ( std::string::npos == startpos ) || ( std::string::npos == endpos ) ) return std::string(""); @@ -148,7 +148,7 @@ const std::string base64Encode(const std::string &inString) { selection = remainder | (*inPtr >> 4); remainder = (*inPtr++ & 0x0f) << 2; outString += base64_table[selection]; - + if ( *inPtr ) { selection = remainder | (*inPtr >> 6); outString += base64_table[selection]; @@ -175,7 +175,7 @@ int split(const char* string, const char delim, std::vector& items) return -2; std::string str(string); - + while ( true ) { size_t pos = str.find(delim); items.push_back(str.substr(0, pos)); @@ -242,7 +242,7 @@ void hwcaps_detect() { } else { sse_version = 0; Debug(1, "Detected a x86\\x86-64 processor"); - } + } #elif defined(__arm__) // ARM processor in 32bit mode // To see if it supports NEON, we need to get that information from the kernel @@ -279,7 +279,7 @@ void* sse2_aligned_memcpy(void* dest, const void* src, size_t bytes) { "sse2_copy_iter:\n\t" "movdqa (%0),%%xmm0\n\t" "movdqa 0x10(%0),%%xmm1\n\t" - "movdqa 0x20(%0),%%xmm2\n\t" + "movdqa 0x20(%0),%%xmm2\n\t" "movdqa 0x30(%0),%%xmm3\n\t" "movdqa 0x40(%0),%%xmm4\n\t" "movdqa 0x50(%0),%%xmm5\n\t" @@ -328,16 +328,17 @@ void timespec_diff(struct timespec *start, struct timespec *end, struct timespec } } -char *timeval_to_string( struct timeval tv ) { - time_t nowtime; - struct tm *nowtm; - static char tmbuf[20], buf[28]; +std::string TimevalToString(timeval tv) { + tm now = {}; + std::array tm_buf = {}; - nowtime = tv.tv_sec; - nowtm = localtime(&nowtime); - strftime(tmbuf, sizeof tmbuf, "%Y-%m-%d %H:%M:%S", nowtm); - snprintf(buf, sizeof buf-1, "%s.%06ld", tmbuf, tv.tv_usec); - return buf; + localtime_r(&tv.tv_sec, &now); + size_t tm_buf_len = strftime(tm_buf.data(), tm_buf.size(), "%Y-%m-%d %H:%M:%S", &now); + if (tm_buf_len == 0) { + return ""; + } + + return stringtf("%s.%06ld", tm_buf.data(), tv.tv_usec); } std::string UriDecode( const std::string &encoded ) { diff --git a/src/zm_utils.h b/src/zm_utils.h index 4eba32b6a..41b6789fe 100644 --- a/src/zm_utils.h +++ b/src/zm_utils.h @@ -63,7 +63,7 @@ void hwcaps_detect(); extern unsigned int sse_version; extern unsigned int neonversion; -char *timeval_to_string( struct timeval tv ); +std::string TimevalToString(timeval tv); std::string UriDecode( const std::string &encoded ); void touch( const char *pathname ); From 4e8c7d1f7cac1936c08ae6932726895c5079cfd1 Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Sun, 7 Mar 2021 16:35:05 +0100 Subject: [PATCH 0244/1277] Eliminate non-thread-safe calls to localtime localtime uses an internal static storage to which a pointer is given as return value. Due to this it is not safe to call localtime from multiple threads since the same static storage is used. Use localtime_r instead which allows to pass in a tm struct. Fixes: https://github.com/ZoneMinder/zoneminder/security/code-scanning/24 https://github.com/ZoneMinder/zoneminder/security/code-scanning/25 https://github.com/ZoneMinder/zoneminder/security/code-scanning/26 https://github.com/ZoneMinder/zoneminder/security/code-scanning/27 https://github.com/ZoneMinder/zoneminder/security/code-scanning/28 https://github.com/ZoneMinder/zoneminder/security/code-scanning/30 https://github.com/ZoneMinder/zoneminder/security/code-scanning/31 https://github.com/ZoneMinder/zoneminder/security/code-scanning/33 https://github.com/ZoneMinder/zoneminder/security/code-scanning/58 https://github.com/ZoneMinder/zoneminder/security/code-scanning/59 https://github.com/ZoneMinder/zoneminder/security/code-scanning/63 https://github.com/ZoneMinder/zoneminder/security/code-scanning/64 https://github.com/ZoneMinder/zoneminder/security/code-scanning/65 --- src/zm_event.cpp | 33 +++++++++++++++++---------------- src/zm_event.h | 12 +++++++----- src/zm_eventstream.cpp | 18 ++++++++++-------- src/zm_image.cpp | 6 ++++-- src/zm_logger.cpp | 3 ++- src/zm_monitor.cpp | 3 ++- src/zm_user.cpp | 11 ++++++----- src/zmu.cpp | 6 ++++-- 8 files changed, 52 insertions(+), 40 deletions(-) diff --git a/src/zm_event.cpp b/src/zm_event.cpp index f0b2da694..9ab13fafc 100644 --- a/src/zm_event.cpp +++ b/src/zm_event.cpp @@ -71,8 +71,8 @@ Event::Event( std::string notes; createNotes(notes); - struct timeval now; - gettimeofday(&now, 0); + timeval now = {}; + gettimeofday(&now, nullptr); if ( !start_time.tv_sec ) { Warning("Event has zero time, setting to now"); @@ -80,12 +80,12 @@ Event::Event( } else if ( start_time.tv_sec > now.tv_sec ) { char buffer[26]; char buffer_now[26]; - struct tm* tm_info; + tm tm_info = {}; - tm_info = localtime(&start_time.tv_sec); - strftime(buffer, 26, "%Y:%m:%d %H:%M:%S", tm_info); - tm_info = localtime(&now.tv_sec); - strftime(buffer_now, 26, "%Y:%m:%d %H:%M:%S", tm_info); + localtime_r(&start_time.tv_sec, &tm_info); + strftime(buffer, 26, "%Y:%m:%d %H:%M:%S", &tm_info); + localtime_r(&now.tv_sec, &tm_info); + strftime(buffer_now, 26, "%Y:%m:%d %H:%M:%S", &tm_info); Error( "StartDateTime in the future starttime %u.%u >? now %u.%u difference %d\n%s\n%s", @@ -661,16 +661,17 @@ bool Event::SetPath(Storage *storage) { return false; } - struct tm *stime = localtime(&start_time.tv_sec); + tm stime = {}; + localtime_r(&start_time.tv_sec, &stime); if ( scheme == Storage::DEEP ) { int dt_parts[6]; - dt_parts[0] = stime->tm_year-100; - dt_parts[1] = stime->tm_mon+1; - dt_parts[2] = stime->tm_mday; - dt_parts[3] = stime->tm_hour; - dt_parts[4] = stime->tm_min; - dt_parts[5] = stime->tm_sec; + dt_parts[0] = stime.tm_year-100; + dt_parts[1] = stime.tm_mon+1; + dt_parts[2] = stime.tm_mday; + dt_parts[3] = stime.tm_hour; + dt_parts[4] = stime.tm_min; + dt_parts[5] = stime.tm_sec; std::string date_path; std::string time_path; @@ -685,7 +686,7 @@ bool Event::SetPath(Storage *storage) { if ( i == 2 ) date_path = path; } - time_path = stringtf("%02d/%02d/%02d", stime->tm_hour, stime->tm_min, stime->tm_sec); + time_path = stringtf("%02d/%02d/%02d", stime.tm_hour, stime.tm_min, stime.tm_sec); // Create event id symlink std::string id_file = stringtf("%s/.%" PRIu64, date_path.c_str(), id); @@ -695,7 +696,7 @@ bool Event::SetPath(Storage *storage) { } } else if ( scheme == Storage::MEDIUM ) { path += stringtf("/%04d-%02d-%02d", - stime->tm_year+1900, stime->tm_mon+1, stime->tm_mday + stime.tm_year+1900, stime.tm_mon+1, stime.tm_mday ); if ( mkdir(path.c_str(), 0755) and ( errno != EEXIST ) ) { Error("Can't mkdir %s: %s", path.c_str(), strerror(errno)); diff --git a/src/zm_event.h b/src/zm_event.h index 1eb3f2a7f..5e972eca2 100644 --- a/src/zm_event.h +++ b/src/zm_event.h @@ -138,18 +138,20 @@ class Event { bool SetPath(Storage *storage); public: - static const char *getSubPath(struct tm *time) { + static const char *getSubPath(tm time) { static char subpath[PATH_MAX] = ""; snprintf(subpath, sizeof(subpath), "%02d/%02d/%02d/%02d/%02d/%02d", - time->tm_year-100, time->tm_mon+1, time->tm_mday, - time->tm_hour, time->tm_min, time->tm_sec); + time.tm_year-100, time.tm_mon+1, time.tm_mday, + time.tm_hour, time.tm_min, time.tm_sec); return subpath; } static const char *getSubPath(time_t *time) { - return Event::getSubPath(localtime(time)); + tm time_tm = {}; + localtime_r(time, &time_tm); + return Event::getSubPath(time_tm); } - const char* getEventFile(void) const { + const char* getEventFile() const { return video_file.c_str(); } diff --git a/src/zm_eventstream.cpp b/src/zm_eventstream.cpp index 5ed424b7f..8610f2f04 100644 --- a/src/zm_eventstream.cpp +++ b/src/zm_eventstream.cpp @@ -171,33 +171,35 @@ bool EventStream::loadEventData(uint64_t event_id) { const char *storage_path = storage->Path(); if ( event_data->scheme == Storage::DEEP ) { - struct tm *event_time = localtime(&event_data->start_time); + tm event_time = {}; + localtime_r(&event_data->start_time, &event_time); if ( storage_path[0] == '/' ) snprintf(event_data->path, sizeof(event_data->path), "%s/%u/%02d/%02d/%02d/%02d/%02d/%02d", storage_path, event_data->monitor_id, - event_time->tm_year-100, event_time->tm_mon+1, event_time->tm_mday, - event_time->tm_hour, event_time->tm_min, event_time->tm_sec); + event_time.tm_year-100, event_time.tm_mon+1, event_time.tm_mday, + event_time.tm_hour, event_time.tm_min, event_time.tm_sec); else snprintf(event_data->path, sizeof(event_data->path), "%s/%s/%u/%02d/%02d/%02d/%02d/%02d/%02d", staticConfig.PATH_WEB.c_str(), storage_path, event_data->monitor_id, - event_time->tm_year-100, event_time->tm_mon+1, event_time->tm_mday, - event_time->tm_hour, event_time->tm_min, event_time->tm_sec); + event_time.tm_year-100, event_time.tm_mon+1, event_time.tm_mday, + event_time.tm_hour, event_time.tm_min, event_time.tm_sec); } else if ( event_data->scheme == Storage::MEDIUM ) { - struct tm *event_time = localtime(&event_data->start_time); + tm event_time = {}; + localtime_r(&event_data->start_time, &event_time); if ( storage_path[0] == '/' ) snprintf(event_data->path, sizeof(event_data->path), "%s/%u/%04d-%02d-%02d/%" PRIu64, storage_path, event_data->monitor_id, - event_time->tm_year+1900, event_time->tm_mon+1, event_time->tm_mday, + event_time.tm_year+1900, event_time.tm_mon+1, event_time.tm_mday, event_data->event_id); else snprintf(event_data->path, sizeof(event_data->path), "%s/%s/%u/%04d-%02d-%02d/%" PRIu64, staticConfig.PATH_WEB.c_str(), storage_path, event_data->monitor_id, - event_time->tm_year+1900, event_time->tm_mon+1, event_time->tm_mday, + event_time.tm_year+1900, event_time.tm_mon+1, event_time.tm_mday, event_data->event_id); } else { diff --git a/src/zm_image.cpp b/src/zm_image.cpp index 014ae783c..2e0c82670 100644 --- a/src/zm_image.cpp +++ b/src/zm_image.cpp @@ -1186,7 +1186,8 @@ cinfo->out_color_space = JCS_RGB; // This is a lot of stuff to allocate on the stack. Recommend char *timebuf[64]; char timebuf[64], msbuf[64]; - strftime(timebuf, sizeof timebuf, "%Y:%m:%d %H:%M:%S", localtime(&(timestamp.tv_sec))); + tm timestamp_tm = {}; + strftime(timebuf, sizeof timebuf, "%Y:%m:%d %H:%M:%S", localtime_r(×tamp.tv_sec, ×tamp_tm)); snprintf(msbuf, sizeof msbuf, "%06d",(int)(timestamp.tv_usec)); // we only use milliseconds because that's all defined in exif, but this is the whole microseconds because we have it unsigned char exiftimes[82] = { 0x45, 0x78, 0x69, 0x66, 0x00, 0x00, 0x49, 0x49, 0x2A, 0x00, 0x08, 0x00, 0x00, 0x00, 0x01, 0x00, @@ -2132,7 +2133,8 @@ void Image::Annotate( void Image::Timestamp( const char *label, const time_t when, const Coord &coord, const int size ) { char time_text[64]; - strftime(time_text, sizeof(time_text), "%y/%m/%d %H:%M:%S", localtime(&when)); + tm when_tm = {}; + strftime(time_text, sizeof(time_text), "%y/%m/%d %H:%M:%S", localtime_r(&when, &when_tm)); if ( label ) { // Assume label is max 64, + ' - ' + 64 chars of time_text char text[132]; diff --git a/src/zm_logger.cpp b/src/zm_logger.cpp index 3d0c0c2c5..d425764ae 100644 --- a/src/zm_logger.cpp +++ b/src/zm_logger.cpp @@ -447,7 +447,8 @@ void Logger::logPrint(bool hex, const char * const filepath, const int line, con } else { #endif char *timePtr = timeString; - timePtr += strftime(timePtr, sizeof(timeString), "%x %H:%M:%S", localtime(&timeVal.tv_sec)); + tm now_tm = {}; + timePtr += strftime(timePtr, sizeof(timeString), "%x %H:%M:%S", localtime_r(&timeVal.tv_sec, &now_tm)); snprintf(timePtr, sizeof(timeString)-(timePtr-timeString), ".%06ld", timeVal.tv_usec); #if 0 } diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index 1ca51bdb9..f846e569f 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -2842,7 +2842,8 @@ void Monitor::TimestampImage(Image *ts_image, const struct timeval *ts_time) con // Expand the strftime macros first char label_time_text[256]; - strftime(label_time_text, sizeof(label_time_text), label_format, localtime(&ts_time->tv_sec)); + tm ts_tm = {}; + strftime(label_time_text, sizeof(label_time_text), label_format, localtime_r(&ts_time->tv_sec, &ts_tm)); char label_text[1024]; const char *s_ptr = label_time_text; char *d_ptr = label_text; diff --git a/src/zm_user.cpp b/src/zm_user.cpp index 512718ccb..13cf7c1ac 100644 --- a/src/zm_user.cpp +++ b/src/zm_user.cpp @@ -245,18 +245,19 @@ User *zmLoadAuthUser(const char *auth, bool use_remote_addr) { const char *pass = dbrow[2]; time_t our_now = now; + tm now_tm = {}; for ( unsigned int i = 0; i < hours; i++, our_now -= 3600 ) { - struct tm *now_tm = localtime(&our_now); + localtime_r(&our_now, &now_tm); snprintf(auth_key, sizeof(auth_key)-1, "%s%s%s%s%d%d%d%d", config.auth_hash_secret, user, pass, remote_addr, - now_tm->tm_hour, - now_tm->tm_mday, - now_tm->tm_mon, - now_tm->tm_year); + now_tm.tm_hour, + now_tm.tm_mday, + now_tm.tm_mon, + now_tm.tm_year); #if HAVE_DECL_MD5 MD5((unsigned char *)auth_key, strlen(auth_key), md5sum); diff --git a/src/zmu.cpp b/src/zmu.cpp index 0bfe90b3b..384af99a3 100644 --- a/src/zmu.cpp +++ b/src/zmu.cpp @@ -506,8 +506,10 @@ int main(int argc, char *argv[]) { struct timeval timestamp = monitor->GetTimestamp(image_idx); if ( verbose ) { char timestamp_str[64] = "None"; - if ( timestamp.tv_sec ) - strftime(timestamp_str, sizeof(timestamp_str), "%Y-%m-%d %H:%M:%S", localtime(×tamp.tv_sec)); + if ( timestamp.tv_sec ) { + tm tm_info = {}; + strftime(timestamp_str, sizeof(timestamp_str), "%Y-%m-%d %H:%M:%S", localtime_r(×tamp.tv_sec, &tm_info)); + } if ( image_idx == -1 ) printf("Time of last image capture: %s.%02ld\n", timestamp_str, timestamp.tv_usec/10000); else From 67d7872e9a3a8db8fc550d1a9f5f7eedceb9f018 Mon Sep 17 00:00:00 2001 From: Peter Keresztes Schmidt Date: Sun, 7 Mar 2021 16:44:08 +0100 Subject: [PATCH 0245/1277] Eliminate non-thread-safe calls to gmtime gmtime uses an internal static storage to which a pointer is given as return value. Due to this it is not safe to call gmtime from multiple threads since the same static storage is used. Use gmtime_r instead which allows to pass in a tm struct. Fixes: https://github.com/ZoneMinder/zoneminder/security/code-scanning/32 --- src/zms.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/zms.cpp b/src/zms.cpp index 89b20e30c..70ec38fdc 100644 --- a/src/zms.cpp +++ b/src/zms.cpp @@ -241,8 +241,9 @@ int main(int argc, const char *argv[], char **envp) { time_t now = time(nullptr); char date_string[64]; + tm now_tm = {}; strftime(date_string, sizeof(date_string)-1, - "%a, %d %b %Y %H:%M:%S GMT", gmtime(&now)); + "%a, %d %b %Y %H:%M:%S GMT", gmtime_r(&now, &now_tm)); fputs("Last-Modified: ", stdout); fputs(date_string, stdout); From 858ae8b11fa283816db1f04d853265ca092ab692 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sun, 21 Mar 2021 13:16:55 -0400 Subject: [PATCH 0246/1277] fix alignment and min width of datetime column in logs view --- web/skins/classic/css/base/views/log.css | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/web/skins/classic/css/base/views/log.css b/web/skins/classic/css/base/views/log.css index 918c1097c..0c6f618c8 100644 --- a/web/skins/classic/css/base/views/log.css +++ b/web/skins/classic/css/base/views/log.css @@ -16,3 +16,14 @@ tr.log-dbg td { font-style: italic; } +th[data-field="DateTime"], +th[data-field="Component"], +th[data-field="Code"], +th[data-field="Message"], +th[data-field="File"], +th[data-field="Line"] { + text-align: left; +} +th[data-field="DateTime"] { + min-width: 135px; +} From be653980f34d2dc41a53b7ac19033edd707c57f3 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Sun, 21 Mar 2021 18:17:10 -0400 Subject: [PATCH 0247/1277] fix eslint --- web/skins/classic/views/js/snapshots.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/web/skins/classic/views/js/snapshots.js b/web/skins/classic/views/js/snapshots.js index b0e83001d..8df31d5c3 100644 --- a/web/skins/classic/views/js/snapshots.js +++ b/web/skins/classic/views/js/snapshots.js @@ -267,7 +267,7 @@ function initPage() { }) .fail(logAjaxFail); }); -*/ + */ // Manage the DELETE button document.getElementById("deleteBtn").addEventListener("click", function onDeleteClick(evt) { if ( ! canEdit.Events ) { From 6d5cbe258365b53785c0ee3c45a97e7c0026632a Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 11:02:32 -0400 Subject: [PATCH 0248/1277] Make incorrect dimensions non-fatal if the monitor dimensions are larger than what is expected, so at least there is enough ram to store the image --- src/zm_local_camera.cpp | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/src/zm_local_camera.cpp b/src/zm_local_camera.cpp index b5c0ab665..b464c4d96 100644 --- a/src/zm_local_camera.cpp +++ b/src/zm_local_camera.cpp @@ -798,10 +798,10 @@ void LocalCamera::Initialise() { ); if ( v4l2_data.fmt.fmt.pix.width != width ) { - Warning("Failed to set requested width"); + Warning("Failed to set requested width"); } if ( v4l2_data.fmt.fmt.pix.height != height ) { - Warning("Failed to set requested height"); + Warning("Failed to set requested height"); } /* Buggy driver paranoia. */ @@ -2087,8 +2087,11 @@ int LocalCamera::Capture(ZMPacket &zm_packet) { buffer_bytesused = v4l2_data.bufptr->bytesused; bytes += buffer_bytesused; - if ( (v4l2_data.fmt.fmt.pix.width * v4l2_data.fmt.fmt.pix.height) != (width * height) ) { - Fatal("Captured image dimensions differ: V4L2: %dx%d monitor: %dx%d", + if ( (v4l2_data.fmt.fmt.pix.width * v4l2_data.fmt.fmt.pix.height) > (width * height) ) { + Fatal("Captured image dimensions larger than image buffer: V4L2: %dx%d monitor: %dx%d", + v4l2_data.fmt.fmt.pix.width, v4l2_data.fmt.fmt.pix.height, width, height); + } else if ( (v4l2_data.fmt.fmt.pix.width * v4l2_data.fmt.fmt.pix.height) != (width * height) ) { + Error("Captured image dimensions differ: V4L2: %dx%d monitor: %dx%d", v4l2_data.fmt.fmt.pix.width, v4l2_data.fmt.fmt.pix.height, width, height); } } // end if v4l2 From 3f3bc50acb9cea70e901aae0649319b37eb91d3a Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 12:04:32 -0400 Subject: [PATCH 0249/1277] Add keep_keyframes setting. When NOT doing passthrough we don't actually have to store all packets since last keyframe, so don't do it. SImplifies clearPackets() logic a lot and will save ram for those people. --- src/zm_packetqueue.cpp | 40 +++++++++++++++++++++++++++++----------- src/zm_packetqueue.h | 2 ++ 2 files changed, 31 insertions(+), 11 deletions(-) diff --git a/src/zm_packetqueue.cpp b/src/zm_packetqueue.cpp index 7cc19debf..152f51972 100644 --- a/src/zm_packetqueue.cpp +++ b/src/zm_packetqueue.cpp @@ -31,7 +31,8 @@ PacketQueue::PacketQueue(): max_video_packet_count(-1), max_stream_id(-1), packet_counts(nullptr), - deleting(false) + deleting(false), + keep_keyframes(false) { } @@ -120,23 +121,39 @@ void PacketQueue::clearPackets(ZMPacket *add_packet) { // // So start at the beginning, counting video packets until the next keyframe. // Then if deleting those packets doesn't break 1 and 2, then go ahead and delete them. - if (! ( + if (keep_keyframes and ! ( add_packet->packet.stream_index == video_stream_id - and - add_packet->keyframe - and - (packet_counts[video_stream_id] > max_video_packet_count) - and - *(pktQueue.begin()) != add_packet - ) + and + add_packet->keyframe + and + (packet_counts[video_stream_id] > max_video_packet_count) + and + *(pktQueue.begin()) != add_packet + ) ) { - Debug(3, "stream index %d ?= video_stream_id %d, keyframe %d, counts %d > max %d at begin %d", - add_packet->packet.stream_index, video_stream_id, add_packet->keyframe, packet_counts[video_stream_id], max_video_packet_count, + Debug(3, "stream index %d ?= video_stream_id %d, keyframe %d, keep_keyframes %d, counts %d > max %d at begin %d", + add_packet->packet.stream_index, video_stream_id, add_packet->keyframe, keep_keyframes, packet_counts[video_stream_id], max_video_packet_count, ( *(pktQueue.begin()) != add_packet ) ); return; } std::unique_lock lck(mutex); + if (!keep_keyframes) { + // If not doing passthrough, we don't care about starting with a keyframe so logic is simpler + while ((*pktQueue.begin() != add_packet) and (packet_counts[video_stream_id] > max_video_packet_count)) { + ZMPacket *zm_packet = *pktQueue.begin(); + ZMLockedPacket *lp = new ZMLockedPacket(zm_packet); + if (!lp->trylock()) break; + delete lp; + + pktQueue.pop_front(); + packet_counts[zm_packet->packet.stream_index] -= 1; + Debug(1, "Deleting a packet with stream index:%d image_index:%d with keyframe:%d, video frames in queue:%d max: %d, queuesize:%d", + zm_packet->packet.stream_index, zm_packet->image_index, zm_packet->keyframe, packet_counts[video_stream_id], max_video_packet_count, pktQueue.size()); + delete zm_packet; + } // end while + return; + } // If ananlysis_it isn't at the end, we need to keep that many additional packets int tail_count = 0; @@ -156,6 +173,7 @@ void PacketQueue::clearPackets(ZMPacket *add_packet) { packetqueue_iterator next_front = pktQueue.begin(); int video_packets_to_delete = 0; // This is a count of how many packets we will delete so we know when to stop looking + // First packet is special because we know it is a video keyframe and only need to check for lock ZMPacket *zm_packet = *it; ZMLockedPacket *lp = new ZMLockedPacket(zm_packet); diff --git a/src/zm_packetqueue.h b/src/zm_packetqueue.h index e759dc777..8caa0a527 100644 --- a/src/zm_packetqueue.h +++ b/src/zm_packetqueue.h @@ -38,6 +38,7 @@ class PacketQueue { int max_stream_id; int *packet_counts; /* packet count for each stream_id, to keep track of how many video vs audio packets are in the queue */ bool deleting; + bool keep_keyframes; std::list iterators; std::mutex mutex; @@ -51,6 +52,7 @@ class PacketQueue { int addStream(); void setMaxVideoPackets(int p); + void setKeepKeyframes(bool k) { keep_keyframes = k; }; bool queuePacket(ZMPacket* packet); ZMLockedPacket * popPacket(); From 2d4b4b60223c142a027fdb09374d2f42518dcdc6 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 12:05:05 -0400 Subject: [PATCH 0250/1277] If we already tried decoding a packet, don't try again. Also we really shouldn't be decoding in videostore. --- src/zm_videostore.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/zm_videostore.cpp b/src/zm_videostore.cpp index ba19f6f57..201192678 100644 --- a/src/zm_videostore.cpp +++ b/src/zm_videostore.cpp @@ -979,7 +979,7 @@ int VideoStore::writeVideoFramePacket(ZMPacket *zm_packet) { ); } else if ( !zm_packet->in_frame ) { Debug(4, "Have no in_frame"); - if ( zm_packet->packet.size ) { + if (zm_packet->packet.size and !zm_packet->decoded) { Debug(4, "Decoding"); if ( !zm_packet->decode(video_in_ctx) ) { Debug(2, "unable to decode yet."); From b8b20917be8e41435a1ac4daab2681ebed18d570 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 12:05:22 -0400 Subject: [PATCH 0251/1277] setKeepKeyframes when not PASSTHROUGH --- src/zm_monitor.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/zm_monitor.cpp b/src/zm_monitor.cpp index f846e569f..83c041ed4 100644 --- a/src/zm_monitor.cpp +++ b/src/zm_monitor.cpp @@ -582,6 +582,7 @@ void Monitor::Load(MYSQL_ROW dbrow, bool load_zones=true, Purpose p = QUERY) { warmup_count = atoi(dbrow[col]); col++; pre_event_count = atoi(dbrow[col]); col++; packetqueue.setMaxVideoPackets(pre_event_count); + packetqueue.setKeepKeyframes(videowriter == PASSTHROUGH); post_event_count = atoi(dbrow[col]); col++; stream_replay_buffer = atoi(dbrow[col]); col++; alarm_frame_count = atoi(dbrow[col]); col++; @@ -2574,7 +2575,7 @@ int Monitor::Capture() { // Don't want to do analysis on it, but we won't due to signal return -1; } else if ( captureResult > 0 ) { - // If we captured, let's assume signal, ::Decode will detect further + // If we captured, let's assume signal, Decode will detect further if (!decoding_enabled) { shared_data->last_write_index = index; shared_data->last_write_time = packet->timestamp->tv_sec; From c347261e19688731f024088638405676ff1d92d9 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 12:05:36 -0400 Subject: [PATCH 0252/1277] Change default of ImageBufferCount to 3 --- web/includes/Monitor.php | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/web/includes/Monitor.php b/web/includes/Monitor.php index 9470c68f3..eb9efed29 100644 --- a/web/includes/Monitor.php +++ b/web/includes/Monitor.php @@ -83,7 +83,7 @@ class Monitor extends ZM_Object { 'LabelX' => 0, 'LabelY' => 0, 'LabelSize' => 1, - 'ImageBufferCount' => 20, + 'ImageBufferCount' => 3, 'WarmupCount' => 0, 'PreEventCount' => 5, 'PostEventCount' => 5, From fa08240a4d3c2d12a9d01e2ad8a79ca676ae5153 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 12:06:25 -0400 Subject: [PATCH 0253/1277] Fix set() and __call to use the default value when set value is ''. Fixes issues in monitor view when changing type --- web/includes/Object.php | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/web/includes/Object.php b/web/includes/Object.php index 4ed4fadf0..337136836 100644 --- a/web/includes/Object.php +++ b/web/includes/Object.php @@ -39,6 +39,7 @@ class ZM_Object { public function __call($fn, array $args){ $type = (array_key_exists($fn, $this->defaults) && is_array($this->defaults[$fn])) ? $this->defaults[$fn]['type'] : 'scalar'; + if ( count($args) ) { if ( $type == 'set' and is_array($args[0]) ) { $this->{$fn} = implode(',', $args[0]); @@ -51,7 +52,15 @@ class ZM_Object { $this->{$fn} = preg_replace($this->defaults[$fn]['filter_regexp'], '', $args[0]); } } else { - $this->{$fn} = $args[0]; + if ( $args[0] == '' and array_key_exists($fn, $this->defaults) ) { + if ( is_array($this->defaults[$fn]) ) { + $this->{$fn} = $this->defaults[$fn]['default']; + } else { + $this->{$fn} = $this->defaults[$fn]; + } + } else { + $this->{$fn} = $args[0]; + } } } @@ -175,7 +184,7 @@ class ZM_Object { $this->$field($value); } else { if ( is_array($value) ) { -# perhaps should turn into a comma-separated string + # perhaps should turn into a comma-separated string $this->{$field} = implode(',', $value); } else if ( is_string($value) ) { if ( array_key_exists($field, $this->defaults) && is_array($this->defaults[$field]) && isset($this->defaults[$field]['filter_regexp']) ) { @@ -186,8 +195,14 @@ class ZM_Object { } else { $this->{$field} = preg_replace($this->defaults[$field]['filter_regexp'], '', trim($value)); } + } else if ( $value == '' and array_key_exists($field, $this->defaults) ) { + if ( is_array($this->defaults[$field]) ) { + $this->{$field} = $this->defaults[$field]['default']; + } else { + $this->{$field} = $this->defaults[$field]; + } } else { - $this->{$field} = trim($value); + $this->{$field} = $value; } } else if ( is_integer($value) ) { $this->{$field} = $value; From 613ed1faf2295d38ef08df0adadb17cff8543f71 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 12:06:48 -0400 Subject: [PATCH 0254/1277] Update estimated ram use when we use the dropdown to change resolution --- web/skins/classic/views/js/monitor.js | 1 + 1 file changed, 1 insertion(+) diff --git a/web/skins/classic/views/js/monitor.js b/web/skins/classic/views/js/monitor.js index 1603df783..854ba627b 100644 --- a/web/skins/classic/views/js/monitor.js +++ b/web/skins/classic/views/js/monitor.js @@ -48,6 +48,7 @@ function updateMonitorDimensions(element) { form.elements['newMonitor[Height]'].value = dimensions[1]; } } + update_estimated_ram_use(); return false; } From c7b22dae818654a557061ffc743d08e745f97d78 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 12:07:23 -0400 Subject: [PATCH 0255/1277] get rid of nextId entirely. Don't want to use it anywhere other than setting the name --- web/skins/classic/views/monitor.php | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/web/skins/classic/views/monitor.php b/web/skins/classic/views/monitor.php index a2402b8bc..fb7e2bde8 100644 --- a/web/skins/classic/views/monitor.php +++ b/web/skins/classic/views/monitor.php @@ -37,16 +37,14 @@ $mid = null; $monitor = null; if ( !empty($_REQUEST['mid']) ) { $mid = validInt($_REQUEST['mid']); - $monitor = new ZM\Monitor($mid); if ( $monitor and ZM_OPT_X10 ) $x10Monitor = dbFetchOne('SELECT * FROM TriggersX10 WHERE MonitorId = ?', NULL, array($mid)); } if ( !$monitor ) { - $nextId = getTableAutoInc('Monitors'); $monitor = new ZM\Monitor(); - $monitor->Name(translate('Monitor').'-'.$nextId); + $monitor->Name(translate('Monitor').'-'.getTableAutoInc('Monitors')); $monitor->WebColour(random_colour()); } # end if $_REQUEST['mid'] From 9cb58873324f99de0992b14a7907ec7146a9e1f0 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 12:41:30 -0400 Subject: [PATCH 0256/1277] better debug logging when choosing codec --- src/zm_videostore.cpp | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/src/zm_videostore.cpp b/src/zm_videostore.cpp index 1ffc69e90..4d4bf7e6d 100644 --- a/src/zm_videostore.cpp +++ b/src/zm_videostore.cpp @@ -155,7 +155,7 @@ bool VideoStore::open() { wanted_codec = AV_CODEC_ID_H264; // FIXME what is the optimal codec? Probably low latency h264 which is effectively mjpeg } else { - Debug(2, "Codec wanted %d", wanted_codec); + Debug(2, "Codec wanted %d %s", wanted_codec, avcodec_get_name((AVCodecID)wanted_codec)); } std::string wanted_encoder = monitor->Encoder(); @@ -167,7 +167,12 @@ bool VideoStore::open() { } } if ( codec_data[i].codec_id != wanted_codec ) { - Debug(1, "Not the right codec %d != %d", codec_data[i].codec_id, wanted_codec); + Debug(1, "Not the right codec %d %s != %d %s", + codec_data[i].codec_id, + avcodec_get_name(codec_data[i].codec_id), + wanted_codec, + avcodec_get_name((AVCodecID)wanted_codec) + ); continue; } From 3fdd2bff7f733e36f569b8699838fe8d6bb20319 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 12:59:18 -0400 Subject: [PATCH 0257/1277] Have to use === because apparently null == NOW() according to php. --- web/includes/Object.php | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/web/includes/Object.php b/web/includes/Object.php index 337136836..0cf5ca850 100644 --- a/web/includes/Object.php +++ b/web/includes/Object.php @@ -351,7 +351,7 @@ class ZM_Object { $sql = 'INSERT INTO `'.$table. '` ('.implode(', ', array_map(function($field) {return '`'.$field.'`';}, $fields)). ') VALUES ('. - implode(', ', array_map(function($field){return $this->$field() == 'NOW()' ? 'NOW()' : '?';}, $fields)).')'; + implode(', ', array_map(function($field){return (($this->$field() === 'NOW()') ? 'NOW()' : '?');}, $fields)).')'; # For some reason comparing 0 to 'NOW()' returns false; So we do this. $filtered = array_filter($fields, function($field){ return ( (!$this->$field()) or ($this->$field() != 'NOW()'));}); From be0841832ec9498ba41f38bd05f17d0f5001e81c Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 12:59:41 -0400 Subject: [PATCH 0258/1277] enable reporting saving errors back to web ui. --- web/includes/actions/filter.php | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/web/includes/actions/filter.php b/web/includes/actions/filter.php index 9216ef723..08eb8c020 100644 --- a/web/includes/actions/filter.php +++ b/web/includes/actions/filter.php @@ -18,9 +18,11 @@ // Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. // +global $error_message; // Event scope actions, view permissions only required if ( !canView('Events') ) { - ZM\Warning('You do not have permission to view Events.'); + $error_message = 'You do not have permission to view Events.'; + ZM\Warning($error_message); return; } @@ -65,12 +67,13 @@ if ( isset($_REQUEST['object']) and ( $_REQUEST['object'] == 'filter' ) ) { $changes = $filter->changes($_REQUEST['filter']); ZM\Debug('Changes: ' . print_r($changes,true)); - if ( $_REQUEST['Id'] and ( $action == 'Save' ) ) { - if ( $filter->Background() ) - $filter->control('stop'); - $filter->save($changes); + if ($filter->Id() and ($action == 'Save')) { + if ($filter->Background()) $filter->control('stop'); + if (!$filter->save($changes)) { + $error_message = $filter->get_last_error(); + return; + } } else { - if ( $action == 'execute' ) { if ( count($changes) ) { $filter->Name('_TempFilter'.time()); @@ -79,7 +82,10 @@ if ( isset($_REQUEST['object']) and ( $_REQUEST['object'] == 'filter' ) ) { } else if ( $action == 'SaveAs' ) { $filter->Id(null); } - $filter->save($changes); + if (!$filter->save($changes)) { + $error_message = $filter->get_last_error(); + return; + } // We update the request id so that the newly saved filter is auto-selected $_REQUEST['Id'] = $filter->Id(); From 1dc32a0eed9d17c446112612d8bb0169d6b668ce Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 13:00:08 -0400 Subject: [PATCH 0259/1277] Use getBodyTopHTML so that we get error reporting --- web/skins/classic/views/filter.php | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/web/skins/classic/views/filter.php b/web/skins/classic/views/filter.php index a3619d74b..d16ebdef3 100644 --- a/web/skins/classic/views/filter.php +++ b/web/skins/classic/views/filter.php @@ -187,10 +187,10 @@ foreach ( dbFetchAll('SELECT Id, Name, MonitorId FROM Zones ORDER BY lower(`Name } xhtmlHeaders(__FILE__, translate('EventFilter')); +echo getBodyTopHTML(); +echo $navbar = getNavBarHTML(); ?> -
-
From d84e4a14fb724da4a0fd5656b12eb46810093cd5 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 17:43:43 -0400 Subject: [PATCH 0260/1277] Use a define when setting pipe size --- src/zm_fifo.cpp | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/zm_fifo.cpp b/src/zm_fifo.cpp index 10a8cc365..f807494ce 100644 --- a/src/zm_fifo.cpp +++ b/src/zm_fifo.cpp @@ -27,6 +27,7 @@ #include #define RAW_BUFFER 512 +#define PIPE_SIZE 1024*1024 void Fifo::file_create_if_missing( const char * path, @@ -80,7 +81,7 @@ bool Fifo::open() { } } #ifdef __linux__ - int ret = fcntl(raw_fd, F_SETPIPE_SZ, 1024 * 1024); + int ret = fcntl(raw_fd, F_SETPIPE_SZ, PIPE_SIZE); if (ret < 0) { Error("set pipe size failed."); } @@ -143,6 +144,7 @@ bool Fifo::writePacket(std::string filename, ZMPacket &packet) { } } + Debug(4, "Writing packet of size %d pts %" PRId64, packet.packet.size, packet.pts); if (fwrite(packet.packet.data, packet.packet.size, 1, outfile) != 1) { Debug(1, "Unable to write to '%s': %s", filename.c_str(), strerror(errno)); fclose(outfile); From d2efb51b0d72082454f1121d1b9332dd6738ac5b Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 17:44:19 -0400 Subject: [PATCH 0261/1277] rearrange a bit, put back setting width and height on video source even though it isn't used yet. Remove signal blocking stuff that we don't use --- src/zm_rtsp_server.cpp | 30 +++++++++--------------------- 1 file changed, 9 insertions(+), 21 deletions(-) diff --git a/src/zm_rtsp_server.cpp b/src/zm_rtsp_server.cpp index 7fe8e916a..a84f85b20 100644 --- a/src/zm_rtsp_server.cpp +++ b/src/zm_rtsp_server.cpp @@ -129,6 +129,10 @@ int main(int argc, char *argv[]) { zmDbConnect(); zmLoadDBConfig(); logInit(log_id_string); + if (!config.min_rtsp_port) { + Debug(1, "Not starting RTSP server because min_rtsp_port not set"); + exit(-1); + } hwcaps_detect(); @@ -152,18 +156,6 @@ int main(int argc, char *argv[]) { zmSetDefaultTermHandler(); zmSetDefaultDieHandler(); - sigset_t block_set; - sigemptyset(&block_set); - - sigaddset(&block_set, SIGHUP); - sigaddset(&block_set, SIGUSR1); - sigaddset(&block_set, SIGUSR2); - - if (!config.min_rtsp_port) { - Debug(1, "Not starting RTSP server because min_rtsp_port not set"); - exit(-1); - } - std::shared_ptr eventLoop(new xop::EventLoop()); std::shared_ptr rtspServer = xop::RtspServer::Create(eventLoop.get()); @@ -183,7 +175,6 @@ int main(int argc, char *argv[]) { std::list sources; while (!zm_terminate) { - for (size_t i = 0; i < monitors.size(); i++) { std::shared_ptr monitor = monitors[i]; @@ -242,13 +233,10 @@ int main(int argc, char *argv[]) { Error("Unable to create source"); } sources.push_back(videoSource); - -#if 0 - if (video_source) { - video_source->setWidth(monitor->Width()); - video_source->setHeight(monitor->Height()); + if (videoSource) { + videoSource->setWidth(monitor->Width()); + videoSource->setHeight(monitor->Height()); } -#endif std::string audioFifoPath = monitor->GetAudioFifoPath(); if (audioFifoPath.empty()) { @@ -288,7 +276,7 @@ int main(int argc, char *argv[]) { logInit(log_id_string); zm_reload = false; } // end if zm_reload - } // end while ! zm_terminate + } // end while !zm_terminate Info("RTSP Server shutting down"); for (size_t i = 0; i < monitors.size(); i++) { @@ -300,7 +288,7 @@ int main(int argc, char *argv[]) { } // end foreach monitor for ( std::list::iterator it = sources.begin(); it != sources.end(); ++it ) { - Debug(1, "RTSPServerThread::stopping source"); + Debug(1, "RTSPServerThread::stopping source"); (*it)->Stop(); } while (sources.size()) { From cfff9723af8dc78138af6746fbf3d9182985a6ef Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 17:44:45 -0400 Subject: [PATCH 0262/1277] Remove h264markers cuz it's in zm_rtsp_server_frame.h --- src/zm_rtsp_server_fifo_h264_source.h | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/zm_rtsp_server_fifo_h264_source.h b/src/zm_rtsp_server_fifo_h264_source.h index d73aa1730..8ad005959 100644 --- a/src/zm_rtsp_server_fifo_h264_source.h +++ b/src/zm_rtsp_server_fifo_h264_source.h @@ -19,8 +19,6 @@ // H264 ZoneMinder FramedSource // --------------------------------- #if HAVE_RTSP_SERVER -const char H264marker[] = {0,0,0,1}; -const char H264shortmarker[] = {0,0,1}; class H26X_ZoneMinderFifoSource : public ZoneMinderFifoVideoSource { public: H26X_ZoneMinderFifoSource( From 8bc22880a49100ff8a6b55dba4adc14c6c141745 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 17:45:07 -0400 Subject: [PATCH 0263/1277] use modern threads. Add a separate thread for sending data. --- src/zm_rtsp_server_fifo_source.cpp | 84 +++++++++++++++++++++--------- src/zm_rtsp_server_fifo_source.h | 24 ++++++--- 2 files changed, 74 insertions(+), 34 deletions(-) diff --git a/src/zm_rtsp_server_fifo_source.cpp b/src/zm_rtsp_server_fifo_source.cpp index 1b3b4f73d..677eb534a 100644 --- a/src/zm_rtsp_server_fifo_source.cpp +++ b/src/zm_rtsp_server_fifo_source.cpp @@ -31,34 +31,53 @@ ZoneMinderFifoSource::ZoneMinderFifoSource( m_fifo(fifo), m_fd(-1) { - memset(&m_thid, 0, sizeof(m_thid)); - memset(&m_mutex, 0, sizeof(m_mutex)); - pthread_mutex_init(&m_mutex, nullptr); - pthread_create(&m_thid, nullptr, threadStub, this); + read_thread_ = std::thread(&ZoneMinderFifoSource::ReadRun, this); + write_thread_ = std::thread(&ZoneMinderFifoSource::WriteRun, this); } ZoneMinderFifoSource::~ZoneMinderFifoSource() { Debug(1, "Deleting Fifo Source"); - stop = 1; - pthread_join(m_thid, nullptr); + Stop(); + if (read_thread_.joinable()) + read_thread_.join(); + if (write_thread_.joinable()) + write_thread_.join(); Debug(1, "Deleting Fifo Source done"); - pthread_mutex_destroy(&m_mutex); } // thread mainloop -void* ZoneMinderFifoSource::thread() { - stop = 0; - - while (!stop) { - if (getNextFrame() < 0) sleep(1); +void ZoneMinderFifoSource::ReadRun() { + while (!stop_) { + if (getNextFrame() < 0) { + Debug(1, "Sleeping"); + sleep(1); + } + } +} +void ZoneMinderFifoSource::WriteRun() { + while (!stop_) { + std::unique_lock lck(mutex_); + if (m_nalQueue.empty()) { + Debug(3, "waiting"); + condition_.wait(lck); + } + + while (!m_nalQueue.empty()) { + NAL_Frame *nal = m_nalQueue.front(); + m_nalQueue.pop(); + + Debug(3, "Pushing nal of size %d at %" PRId64, nal->size(), nal->pts()); + PushFrame(nal->buffer(), nal->size(), nal->pts()); + Debug(3, "Done Pushing nal"); + delete nal; + } } - return nullptr; } // read from monitor int ZoneMinderFifoSource::getNextFrame() { - if (zm_terminate or stop) { - Debug(1, "Terminating %d %d", zm_terminate, stop); + if (zm_terminate or stop_) { + Debug(1, "Terminating %d %d", zm_terminate, (stop_==true?1:0)); return -1; } @@ -84,7 +103,7 @@ int ZoneMinderFifoSource::getNextFrame() { return -1; } - Debug(1, "%s bytes read %d bytes, buffer size %u", m_fifo.c_str(), bytes_read, m_buffer.size()); + Debug(3, "%s bytes read %d bytes, buffer size %u", m_fifo.c_str(), bytes_read, m_buffer.size()); while (m_buffer.size()) { unsigned int data_size = 0; int64_t pts; @@ -142,23 +161,36 @@ int ZoneMinderFifoSource::getNextFrame() { int bytes_needed = data_size - (m_buffer.size() - header_size); if (bytes_needed > 0) { Debug(4, "Need another %d bytes. Trying to read them", bytes_needed); - int bytes_read = m_buffer.read_into(m_fd, bytes_needed); - if ( bytes_read != bytes_needed ) { - Debug(1, "Failed to read another %d bytes, got %d.", bytes_needed, bytes_read); - return -1; - } + while (bytes_needed) { + int bytes_read = m_buffer.read_into(m_fd, bytes_needed); + if (bytes_read <= 0) { + Debug(1, "Failed to read another %d bytes, got %d.", bytes_needed, bytes_read); + return -1; + } + + if (bytes_read != bytes_needed) { + Debug(4, "Failed to read another %d bytes, got %d.", bytes_needed, bytes_read); + } + bytes_needed -= bytes_read; + } // end while bytes_neeeded } m_buffer.consume(header_size); unsigned char *packet_start = m_buffer.head(); size_t bytes_remaining = data_size; std::list< std::pair > framesList = this->splitFrames(packet_start, bytes_remaining); - Debug(3, "Got %d frames, consuming %d bytes, remaining %d", framesList.size(), header_size + data_size, bytes_remaining); + Debug(3, "Got %d frames, consuming %d bytes, remaining %d", framesList.size(), data_size, bytes_remaining); m_buffer.consume(data_size); - while (framesList.size()) { - std::pair nal = framesList.front(); - framesList.pop_front(); - PushFrame(nal.first, nal.second, pts); + + { + std::unique_lock lck(mutex_); + while (framesList.size()) { + std::pair nal = framesList.front(); + framesList.pop_front(); + NAL_Frame *Nal = new NAL_Frame(nal.first, nal.second, pts); + m_nalQueue.push(Nal); + } } + condition_.notify_all(); } // end while m_buffer.size() return 1; } diff --git a/src/zm_rtsp_server_fifo_source.h b/src/zm_rtsp_server_fifo_source.h index e1c60cfdf..ff2f7d7ed 100644 --- a/src/zm_rtsp_server_fifo_source.h +++ b/src/zm_rtsp_server_fifo_source.h @@ -13,8 +13,10 @@ #include "zm_config.h" #include "zm_ffmpeg.h" #include "zm_define.h" +#include "zm_rtsp_server_frame.h" #include #include +#include #include #if HAVE_RTSP_SERVER @@ -24,7 +26,10 @@ class ZoneMinderFifoSource { public: - void Stop() { stop=1; }; + void Stop() { + stop_ = true; + condition_.notify_all(); + }; ZoneMinderFifoSource( std::shared_ptr& rtspServer, @@ -35,8 +40,9 @@ class ZoneMinderFifoSource { virtual ~ZoneMinderFifoSource(); protected: - static void* threadStub(void* clientData) { return ((ZoneMinderFifoSource*) clientData)->thread();}; - void* thread(); + void ReadRun(); + void WriteRun(); + int getNextFrame(); virtual void PushFrame(const uint8_t *data, size_t size, int64_t pts) = 0; // split packet in frames @@ -45,11 +51,12 @@ class ZoneMinderFifoSource { protected: - int m_width; - int m_height; - pthread_t m_thid; - pthread_mutex_t m_mutex; - int stop; + std::mutex mutex_; + std::condition_variable condition_; + + std::thread read_thread_; + std::thread write_thread_; + std::atomic stop_; std::shared_ptr& m_rtspServer; xop::MediaSessionId m_sessionId; @@ -58,6 +65,7 @@ class ZoneMinderFifoSource { int m_fd; Buffer m_buffer; AVRational m_timeBase; + std::queue m_nalQueue; }; #endif // HAVE_RTSP_SERVER From 96a96f48656e1360e2406e8736d437af37826bad Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 17:45:44 -0400 Subject: [PATCH 0264/1277] add jwt token as a std:;string --- src/zm_rtsp_server_authenticator.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/zm_rtsp_server_authenticator.h b/src/zm_rtsp_server_authenticator.h index 9908327d3..5793feb32 100644 --- a/src/zm_rtsp_server_authenticator.h +++ b/src/zm_rtsp_server_authenticator.h @@ -45,10 +45,10 @@ class ZM_RtspServer_Authenticator : public xop::Authenticator { if (query.has("jwt_token")) { const QueryParameter *jwt_token = query.get("jwt_token"); - user = zmLoadTokenUser(jwt_token->firstValue().c_str(), false); + user = zmLoadTokenUser(jwt_token->firstValue(), false); } else if (query.has("token")) { const QueryParameter *jwt_token = query.get("token"); - user = zmLoadTokenUser(jwt_token->firstValue().c_str(), false); + user = zmLoadTokenUser(jwt_token->firstValue(), false); } else if (strcmp(config.auth_relay, "none") == 0) { if (query.has("username")) { std::string username = query.get("username")->firstValue(); From f306febb5fb732faad2283cc36ac8f8e583548cc Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 17:46:13 -0400 Subject: [PATCH 0265/1277] use find_one when loading Server which can use caching --- web/ajax/log.php | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/web/ajax/log.php b/web/ajax/log.php index 7dfb11d6a..4fed0088b 100644 --- a/web/ajax/log.php +++ b/web/ajax/log.php @@ -168,7 +168,7 @@ function queryRequest() { foreach ( $results as $row ) { $row['DateTime'] = strftime('%Y-%m-%d %H:%M:%S', intval($row['TimeKey'])); - $Server = new ZM\Server($row['ServerId']); + $Server = ZM\Server::find_one(array('Id'=>$row['ServerId'])); $row['Server'] = $Server->Name(); // First strip out any html tags // Second strip out all characters that are not ASCII 32-126 (yes, 126) From 0a8b89fecdc81fbaea58edd8225ecc2bf2ffb7cb Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 17:46:39 -0400 Subject: [PATCH 0266/1277] Use pts instead of timestamp in our nal --- src/zm_rtsp_server_frame.h | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/src/zm_rtsp_server_frame.h b/src/zm_rtsp_server_frame.h index 99c1223f4..26843d691 100644 --- a/src/zm_rtsp_server_frame.h +++ b/src/zm_rtsp_server_frame.h @@ -15,17 +15,14 @@ const char H264shortmarker[] = {0,0,1}; class NAL_Frame { public: - NAL_Frame(unsigned char * buffer, size_t size, timeval timestamp) : + NAL_Frame(unsigned char * buffer, size_t size, int64 pts) : m_buffer(nullptr), m_size(size), - m_timestamp(timestamp), + m_pts(pts), m_ref_count(1) { m_buffer = new unsigned char[m_size]; memcpy(m_buffer, buffer, m_size); }; - NAL_Frame(unsigned char* buffer, size_t size) : m_buffer(buffer), m_size(size) { - gettimeofday(&m_timestamp, NULL); - }; NAL_Frame& operator=(const NAL_Frame&); ~NAL_Frame() { delete[] m_buffer; @@ -37,6 +34,7 @@ class NAL_Frame { unsigned char *nal() const { return m_buffer+4; }; size_t size() const { return m_size; }; size_t nal_size() const { return m_size-4; }; + int64_t pts() const { return m_pts; }; bool check() const { // Look for marker at beginning unsigned char *marker = (unsigned char*)memmem(m_buffer, sizeof(H264marker), H264marker, sizeof(H264marker)); @@ -68,7 +66,7 @@ class NAL_Frame { unsigned char* m_buffer; size_t m_size; public: - timeval m_timestamp; + int64 m_pts; private: int m_ref_count; }; From 16b33536ed0fbed2483fc617dc1dd78ba2de530f Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 17:46:56 -0400 Subject: [PATCH 0267/1277] Turn off second Log in header --- web/skins/classic/includes/functions.php | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/web/skins/classic/includes/functions.php b/web/skins/classic/includes/functions.php index 1e490fd46..81742e41a 100644 --- a/web/skins/classic/includes/functions.php +++ b/web/skins/classic/includes/functions.php @@ -266,7 +266,7 @@ function getNormalNavBarHTML($running, $user, $bandwidth_options, $view, $skin) echo getDbConHTML(); echo getStorageHTML(); echo getShmHTML(); - echo getLogIconHTML(); + #echo getLogIconHTML(); ?> From e0893ef7ab1010e2df95524e636b98733e2cc572 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 17:49:46 -0400 Subject: [PATCH 0268/1277] update RtspServer --- dep/RtspServer | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dep/RtspServer b/dep/RtspServer index c81ec629f..8ef9169c4 160000 --- a/dep/RtspServer +++ b/dep/RtspServer @@ -1 +1 @@ -Subproject commit c81ec629f091b77e2948b1e6113064de7e9bc9e3 +Subproject commit 8ef9169c473da2f77752b5c43ae2dd7a82596876 From 7743445323dc28f2e5487007e00834a141ab0617 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 21:30:56 -0400 Subject: [PATCH 0269/1277] Handle when there isn't a server Id --- web/ajax/log.php | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/web/ajax/log.php b/web/ajax/log.php index 4fed0088b..c5181e49b 100644 --- a/web/ajax/log.php +++ b/web/ajax/log.php @@ -169,7 +169,8 @@ function queryRequest() { foreach ( $results as $row ) { $row['DateTime'] = strftime('%Y-%m-%d %H:%M:%S', intval($row['TimeKey'])); $Server = ZM\Server::find_one(array('Id'=>$row['ServerId'])); - $row['Server'] = $Server->Name(); + + $row['Server'] = $Server ? $Server->Name() : ''; // First strip out any html tags // Second strip out all characters that are not ASCII 32-126 (yes, 126) $row['Message'] = preg_replace('/[^\x20-\x7E]/', '', strip_tags($row['Message'])); From e51fe9eb4aaa3488993cade4905063fadcb9d569 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Mon, 22 Mar 2021 21:31:09 -0400 Subject: [PATCH 0270/1277] add tail_count to encode option --- src/zm_packetqueue.cpp | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/src/zm_packetqueue.cpp b/src/zm_packetqueue.cpp index 152f51972..450087015 100644 --- a/src/zm_packetqueue.cpp +++ b/src/zm_packetqueue.cpp @@ -138,22 +138,6 @@ void PacketQueue::clearPackets(ZMPacket *add_packet) { return; } std::unique_lock lck(mutex); - if (!keep_keyframes) { - // If not doing passthrough, we don't care about starting with a keyframe so logic is simpler - while ((*pktQueue.begin() != add_packet) and (packet_counts[video_stream_id] > max_video_packet_count)) { - ZMPacket *zm_packet = *pktQueue.begin(); - ZMLockedPacket *lp = new ZMLockedPacket(zm_packet); - if (!lp->trylock()) break; - delete lp; - - pktQueue.pop_front(); - packet_counts[zm_packet->packet.stream_index] -= 1; - Debug(1, "Deleting a packet with stream index:%d image_index:%d with keyframe:%d, video frames in queue:%d max: %d, queuesize:%d", - zm_packet->packet.stream_index, zm_packet->image_index, zm_packet->keyframe, packet_counts[video_stream_id], max_video_packet_count, pktQueue.size()); - delete zm_packet; - } // end while - return; - } // If ananlysis_it isn't at the end, we need to keep that many additional packets int tail_count = 0; @@ -169,11 +153,27 @@ void PacketQueue::clearPackets(ZMPacket *add_packet) { } Debug(1, "Tail count is %d", tail_count); + if (!keep_keyframes) { + // If not doing passthrough, we don't care about starting with a keyframe so logic is simpler + while ((*pktQueue.begin() != add_packet) and (packet_counts[video_stream_id] > max_video_packet_count + tail_count)) { + ZMPacket *zm_packet = *pktQueue.begin(); + ZMLockedPacket *lp = new ZMLockedPacket(zm_packet); + if (!lp->trylock()) break; + delete lp; + + pktQueue.pop_front(); + packet_counts[zm_packet->packet.stream_index] -= 1; + Debug(1, "Deleting a packet with stream index:%d image_index:%d with keyframe:%d, video frames in queue:%d max: %d, queuesize:%d", + zm_packet->packet.stream_index, zm_packet->image_index, zm_packet->keyframe, packet_counts[video_stream_id], max_video_packet_count, pktQueue.size()); + delete zm_packet; + } // end while + return; + } + packetqueue_iterator it = pktQueue.begin(); packetqueue_iterator next_front = pktQueue.begin(); int video_packets_to_delete = 0; // This is a count of how many packets we will delete so we know when to stop looking - // First packet is special because we know it is a video keyframe and only need to check for lock ZMPacket *zm_packet = *it; ZMLockedPacket *lp = new ZMLockedPacket(zm_packet); From 456afac0ec9c7628ded4a3b819c5047aaefba756 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 23 Mar 2021 10:07:55 -0400 Subject: [PATCH 0271/1277] Must initialize stop_ --- src/zm_rtsp_server_fifo_source.cpp | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/zm_rtsp_server_fifo_source.cpp b/src/zm_rtsp_server_fifo_source.cpp index 677eb534a..82d60b72a 100644 --- a/src/zm_rtsp_server_fifo_source.cpp +++ b/src/zm_rtsp_server_fifo_source.cpp @@ -25,6 +25,7 @@ ZoneMinderFifoSource::ZoneMinderFifoSource( xop::MediaChannelId channelId, std::string fifo ) : + stop_(false), m_rtspServer(rtspServer), m_sessionId(sessionId), m_channelId(channelId), @@ -47,6 +48,7 @@ ZoneMinderFifoSource::~ZoneMinderFifoSource() { // thread mainloop void ZoneMinderFifoSource::ReadRun() { + if (!stop_) Warning("bad value for stop_ in ReadRun"); while (!stop_) { if (getNextFrame() < 0) { Debug(1, "Sleeping"); @@ -55,6 +57,7 @@ void ZoneMinderFifoSource::ReadRun() { } } void ZoneMinderFifoSource::WriteRun() { + if (!stop_) Warning("bad value for stop_ in WriteRun"); while (!stop_) { std::unique_lock lck(mutex_); if (m_nalQueue.empty()) { From 1bfc61a5b786577929aa43db252ec3cb0cdb7221 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 23 Mar 2021 10:32:59 -0400 Subject: [PATCH 0272/1277] change log level to debug when monitor is not decoding --- src/zm_stream.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/zm_stream.cpp b/src/zm_stream.cpp index 869341114..ee7ce26a7 100644 --- a/src/zm_stream.cpp +++ b/src/zm_stream.cpp @@ -73,7 +73,7 @@ bool StreamBase::checkInitialised() { return false; } if ((monitor->GetType() == Monitor::FFMPEG) and !monitor->DecodingEnabled() ) { - Error("Monitor is not decoding."); + Debug(1, "Monitor is not decoding."); return false; } return true; From 7a533686de4a2ea92f84c35c8f3a4542f4f41bd7 Mon Sep 17 00:00:00 2001 From: Isaac Connor Date: Tue, 23 Mar 2021 13:43:08 -0400 Subject: [PATCH 0273/1277] remove our version of jwt-cpp --- dep/jwt-cpp/CMakeLists.txt | 6 - dep/jwt-cpp/Doxyfile | 2494 --- dep/jwt-cpp/LICENSE | 21 - dep/jwt-cpp/README.md | 208 - dep/jwt-cpp/include/jwt-cpp/base.h | 208 - dep/jwt-cpp/include/jwt-cpp/jwt.h | 3037 --- dep/jwt-cpp/include/nlohmann/json.hpp | 25447 ---------------------- dep/jwt-cpp/include/picojson/picojson.h | 1200 - 8 files changed, 32621 deletions(-) delete mode 100644 dep/jwt-cpp/CMakeLists.txt delete mode 100644 dep/jwt-cpp/Doxyfile delete mode 100644 dep/jwt-cpp/LICENSE delete mode 100644 dep/jwt-cpp/README.md delete mode 100644 dep/jwt-cpp/include/jwt-cpp/base.h delete mode 100644 dep/jwt-cpp/include/jwt-cpp/jwt.h delete mode 100644 dep/jwt-cpp/include/nlohmann/json.hpp delete mode 100644 dep/jwt-cpp/include/picojson/picojson.h diff --git a/dep/jwt-cpp/CMakeLists.txt b/dep/jwt-cpp/CMakeLists.txt deleted file mode 100644 index 81ddc84a1..000000000 --- a/dep/jwt-cpp/CMakeLists.txt +++ /dev/null @@ -1,6 +0,0 @@ -add_library(jwt-cpp INTERFACE) -add_library(jwt-cpp::jwt-cpp ALIAS jwt-cpp) - -target_include_directories(jwt-cpp - INTERFACE - ${CMAKE_CURRENT_SOURCE_DIR}/include) diff --git a/dep/jwt-cpp/Doxyfile b/dep/jwt-cpp/Doxyfile deleted file mode 100644 index 0e912fd79..000000000 --- a/dep/jwt-cpp/Doxyfile +++ /dev/null @@ -1,2494 +0,0 @@ -# Doxyfile 1.8.13 - -# This file describes the settings to be used by the documentation system -# doxygen (www.doxygen.org) for a project. -# -# All text after a double hash (##) is considered a comment and is placed in -# front of the TAG it is preceding. -# -# All text after a single hash (#) is considered a comment and will be ignored. -# The format is: -# TAG = value [value, ...] -# For lists, items can also be appended using: -# TAG += value [value, ...] -# Values that contain spaces should be placed between quotes (\" \"). - -#--------------------------------------------------------------------------- -# Project related configuration options -#--------------------------------------------------------------------------- - -# This tag specifies the encoding used for all characters in the config file -# that follow. The default is UTF-8 which is also the encoding used for all text -# before the first occurrence of this tag. Doxygen uses libiconv (or the iconv -# built into libc) for the transcoding. See http://www.gnu.org/software/libiconv -# for the list of possible encodings. -# The default value is: UTF-8. - -DOXYFILE_ENCODING = UTF-8 - -# The PROJECT_NAME tag is a single word (or a sequence of words surrounded by -# double-quotes, unless you are using Doxywizard) that should identify the -# project for which the documentation is generated. This name is used in the -# title of most generated pages and in a few other places. -# The default value is: My Project. - -PROJECT_NAME = "JWT-C++" - -# The PROJECT_NUMBER tag can be used to enter a project or revision number. This -# could be handy for archiving the generated documentation or if some version -# control system is used. - -PROJECT_NUMBER = 0.5.0 - -# Using the PROJECT_BRIEF tag one can provide an optional one line description -# for a project that appears at the top of each page and should give viewer a -# quick idea about the purpose of the project. Keep the description short. - -PROJECT_BRIEF = - -# With the PROJECT_LOGO tag one can specify a logo or an icon that is included -# in the documentation. The maximum height of the logo should not exceed 55 -# pixels and the maximum width should not exceed 200 pixels. Doxygen will copy -# the logo to the output directory. - -PROJECT_LOGO = - -# The OUTPUT_DIRECTORY tag is used to specify the (relative or absolute) path -# into which the generated documentation will be written. If a relative path is -# entered, it will be relative to the location where doxygen was started. If -# left blank the current directory will be used. - -OUTPUT_DIRECTORY = docs - -# If the CREATE_SUBDIRS tag is set to YES then doxygen will create 4096 sub- -# directories (in 2 levels) under the output directory of each output format and -# will distribute the generated files over these directories. Enabling this -# option can be useful when feeding doxygen a huge amount of source files, where -# putting all generated files in the same directory would otherwise causes -# performance problems for the file system. -# The default value is: NO. - -CREATE_SUBDIRS = NO - -# If the ALLOW_UNICODE_NAMES tag is set to YES, doxygen will allow non-ASCII -# characters to appear in the names of generated files. If set to NO, non-ASCII -# characters will be escaped, for example _xE3_x81_x84 will be used for Unicode -# U+3044. -# The default value is: NO. - -ALLOW_UNICODE_NAMES = NO - -# The OUTPUT_LANGUAGE tag is used to specify the language in which all -# documentation generated by doxygen is written. Doxygen will use this -# information to generate all constant output in the proper language. -# Possible values are: Afrikaans, Arabic, Armenian, Brazilian, Catalan, Chinese, -# Chinese-Traditional, Croatian, Czech, Danish, Dutch, English (United States), -# Esperanto, Farsi (Persian), Finnish, French, German, Greek, Hungarian, -# Indonesian, Italian, Japanese, Japanese-en (Japanese with English messages), -# Korean, Korean-en (Korean with English messages), Latvian, Lithuanian, -# Macedonian, Norwegian, Persian (Farsi), Polish, Portuguese, Romanian, Russian, -# Serbian, Serbian-Cyrillic, Slovak, Slovene, Spanish, Swedish, Turkish, -# Ukrainian and Vietnamese. -# The default value is: English. - -OUTPUT_LANGUAGE = English - -# If the BRIEF_MEMBER_DESC tag is set to YES, doxygen will include brief member -# descriptions after the members that are listed in the file and class -# documentation (similar to Javadoc). Set to NO to disable this. -# The default value is: YES. - -BRIEF_MEMBER_DESC = YES - -# If the REPEAT_BRIEF tag is set to YES, doxygen will prepend the brief -# description of a member or function before the detailed description -# -# Note: If both HIDE_UNDOC_MEMBERS and BRIEF_MEMBER_DESC are set to NO, the -# brief descriptions will be completely suppressed. -# The default value is: YES. - -REPEAT_BRIEF = YES - -# This tag implements a quasi-intelligent brief description abbreviator that is -# used to form the text in various listings. Each string in this list, if found -# as the leading text of the brief description, will be stripped from the text -# and the result, after processing the whole list, is used as the annotated -# text. Otherwise, the brief description is used as-is. If left blank, the -# following values are used ($name is automatically replaced with the name of -# the entity):The $name class, The $name widget, The $name file, is, provides, -# specifies, contains, represents, a, an and the. - -ABBREVIATE_BRIEF = "The $name class" \ - "The $name widget" \ - "The $name file" \ - is \ - provides \ - specifies \ - contains \ - represents \ - a \ - an \ - the - -# If the ALWAYS_DETAILED_SEC and REPEAT_BRIEF tags are both set to YES then -# doxygen will generate a detailed section even if there is only a brief -# description. -# The default value is: NO. - -ALWAYS_DETAILED_SEC = NO - -# If the INLINE_INHERITED_MEMB tag is set to YES, doxygen will show all -# inherited members of a class in the documentation of that class as if those -# members were ordinary class members. Constructors, destructors and assignment -# operators of the base classes will not be shown. -# The default value is: NO. - -INLINE_INHERITED_MEMB = NO - -# If the FULL_PATH_NAMES tag is set to YES, doxygen will prepend the full path -# before files name in the file list and in the header files. If set to NO the -# shortest path that makes the file name unique will be used -# The default value is: YES. - -FULL_PATH_NAMES = YES - -# The STRIP_FROM_PATH tag can be used to strip a user-defined part of the path. -# Stripping is only done if one of the specified strings matches the left-hand -# part of the path. The tag can be used to show relative paths in the file list. -# If left blank the directory from which doxygen is run is used as the path to -# strip. -# -# Note that you can specify absolute paths here, but also relative paths, which -# will be relative from the directory where doxygen is started. -# This tag requires that the tag FULL_PATH_NAMES is set to YES. - -STRIP_FROM_PATH = - -# The STRIP_FROM_INC_PATH tag can be used to strip a user-defined part of the -# path mentioned in the documentation of a class, which tells the reader which -# header file to include in order to use a class. If left blank only the name of -# the header file containing the class definition is used. Otherwise one should -# specify the list of include paths that are normally passed to the compiler -# using the -I flag. - -STRIP_FROM_INC_PATH = - -# If the SHORT_NAMES tag is set to YES, doxygen will generate much shorter (but -# less readable) file names. This can be useful is your file systems doesn't -# support long names like on DOS, Mac, or CD-ROM. -# The default value is: NO. - -SHORT_NAMES = NO - -# If the JAVADOC_AUTOBRIEF tag is set to YES then doxygen will interpret the -# first line (until the first dot) of a Javadoc-style comment as the brief -# description. If set to NO, the Javadoc-style will behave just like regular Qt- -# style comments (thus requiring an explicit @brief command for a brief -# description.) -# The default value is: NO. - -JAVADOC_AUTOBRIEF = NO - -# If the QT_AUTOBRIEF tag is set to YES then doxygen will interpret the first -# line (until the first dot) of a Qt-style comment as the brief description. If -# set to NO, the Qt-style will behave just like regular Qt-style comments (thus -# requiring an explicit \brief command for a brief description.) -# The default value is: NO. - -QT_AUTOBRIEF = NO - -# The MULTILINE_CPP_IS_BRIEF tag can be set to YES to make doxygen treat a -# multi-line C++ special comment block (i.e. a block of //! or /// comments) as -# a brief description. This used to be the default behavior. The new default is -# to treat a multi-line C++ comment block as a detailed description. Set this -# tag to YES if you prefer the old behavior instead. -# -# Note that setting this tag to YES also means that rational rose comments are -# not recognized any more. -# The default value is: NO. - -MULTILINE_CPP_IS_BRIEF = NO - -# If the INHERIT_DOCS tag is set to YES then an undocumented member inherits the -# documentation from any documented member that it re-implements. -# The default value is: YES. - -INHERIT_DOCS = YES - -# If the SEPARATE_MEMBER_PAGES tag is set to YES then doxygen will produce a new -# page for each member. If set to NO, the documentation of a member will be part -# of the file/class/namespace that contains it. -# The default value is: NO. - -SEPARATE_MEMBER_PAGES = NO - -# The TAB_SIZE tag can be used to set the number of spaces in a tab. Doxygen -# uses this value to replace tabs by spaces in code fragments. -# Minimum value: 1, maximum value: 16, default value: 4. - -TAB_SIZE = 4 - -# This tag can be used to specify a number of aliases that act as commands in -# the documentation. An alias has the form: -# name=value -# For example adding -# "sideeffect=@par Side Effects:\n" -# will allow you to put the command \sideeffect (or @sideeffect) in the -# documentation, which will result in a user-defined paragraph with heading -# "Side Effects:". You can put \n's in the value part of an alias to insert -# newlines. - -ALIASES = - -# This tag can be used to specify a number of word-keyword mappings (TCL only). -# A mapping has the form "name=value". For example adding "class=itcl::class" -# will allow you to use the command class in the itcl::class meaning. - -TCL_SUBST = - -# Set the OPTIMIZE_OUTPUT_FOR_C tag to YES if your project consists of C sources -# only. Doxygen will then generate output that is more tailored for C. For -# instance, some of the names that are used will be different. The list of all -# members will be omitted, etc. -# The default value is: NO. - -OPTIMIZE_OUTPUT_FOR_C = NO - -# Set the OPTIMIZE_OUTPUT_JAVA tag to YES if your project consists of Java or -# Python sources only. Doxygen will then generate output that is more tailored -# for that language. For instance, namespaces will be presented as packages, -# qualified scopes will look different, etc. -# The default value is: NO. - -OPTIMIZE_OUTPUT_JAVA = NO - -# Set the OPTIMIZE_FOR_FORTRAN tag to YES if your project consists of Fortran -# sources. Doxygen will then generate output that is tailored for Fortran. -# The default value is: NO. - -OPTIMIZE_FOR_FORTRAN = NO - -# Set the OPTIMIZE_OUTPUT_VHDL tag to YES if your project consists of VHDL -# sources. Doxygen will then generate output that is tailored for VHDL. -# The default value is: NO. - -OPTIMIZE_OUTPUT_VHDL = NO - -# Doxygen selects the parser to use depending on the extension of the files it -# parses. With this tag you can assign which parser to use for a given -# extension. Doxygen has a built-in mapping, but you can override or extend it -# using this tag. The format is ext=language, where ext is a file extension, and -# language is one of the parsers supported by doxygen: IDL, Java, Javascript, -# C#, C, C++, D, PHP, Objective-C, Python, Fortran (fixed format Fortran: -# FortranFixed, free formatted Fortran: FortranFree, unknown formatted Fortran: -# Fortran. In the later case the parser tries to guess whether the code is fixed -# or free formatted code, this is the default for Fortran type files), VHDL. For -# instance to make doxygen treat .inc files as Fortran files (default is PHP), -# and .f files as C (default is Fortran), use: inc=Fortran f=C. -# -# Note: For files without extension you can use no_extension as a placeholder. -# -# Note that for custom extensions you also need to set FILE_PATTERNS otherwise -# the files are not read by doxygen. - -EXTENSION_MAPPING = - -# If the MARKDOWN_SUPPORT tag is enabled then doxygen pre-processes all comments -# according to the Markdown format, which allows for more readable -# documentation. See http://daringfireball.net/projects/markdown/ for details. -# The output of markdown processing is further processed by doxygen, so you can -# mix doxygen, HTML, and XML commands with Markdown formatting. Disable only in -# case of backward compatibilities issues. -# The default value is: YES. - -MARKDOWN_SUPPORT = YES - -# When the TOC_INCLUDE_HEADINGS tag is set to a non-zero value, all headings up -# to that level are automatically included in the table of contents, even if -# they do not have an id attribute. -# Note: This feature currently applies only to Markdown headings. -# Minimum value: 0, maximum value: 99, default value: 0. -# This tag requires that the tag MARKDOWN_SUPPORT is set to YES. - -TOC_INCLUDE_HEADINGS = 0 - -# When enabled doxygen tries to link words that correspond to documented -# classes, or namespaces to their corresponding documentation. Such a link can -# be prevented in individual cases by putting a % sign in front of the word or -# globally by setting AUTOLINK_SUPPORT to NO. -# The default value is: YES. - -AUTOLINK_SUPPORT = YES - -# If you use STL classes (i.e. std::string, std::vector, etc.) but do not want -# to include (a tag file for) the STL sources as input, then you should set this -# tag to YES in order to let doxygen match functions declarations and -# definitions whose arguments contain STL classes (e.g. func(std::string); -# versus func(std::string) {}). This also make the inheritance and collaboration -# diagrams that involve STL classes more complete and accurate. -# The default value is: NO. - -BUILTIN_STL_SUPPORT = YES - -# If you use Microsoft's C++/CLI language, you should set this option to YES to -# enable parsing support. -# The default value is: NO. - -CPP_CLI_SUPPORT = NO - -# Set the SIP_SUPPORT tag to YES if your project consists of sip (see: -# http://www.riverbankcomputing.co.uk/software/sip/intro) sources only. Doxygen -# will parse them like normal C++ but will assume all classes use public instead -# of private inheritance when no explicit protection keyword is present. -# The default value is: NO. - -SIP_SUPPORT = NO - -# For Microsoft's IDL there are propget and propput attributes to indicate -# getter and setter methods for a property. Setting this option to YES will make -# doxygen to replace the get and set methods by a property in the documentation. -# This will only work if the methods are indeed getting or setting a simple -# type. If this is not the case, or you want to show the methods anyway, you -# should set this option to NO. -# The default value is: YES. - -IDL_PROPERTY_SUPPORT = YES - -# If member grouping is used in the documentation and the DISTRIBUTE_GROUP_DOC -# tag is set to YES then doxygen will reuse the documentation of the first -# member in the group (if any) for the other members of the group. By default -# all members of a group must be documented explicitly. -# The default value is: NO. - -DISTRIBUTE_GROUP_DOC = NO - -# If one adds a struct or class to a group and this option is enabled, then also -# any nested class or struct is added to the same group. By default this option -# is disabled and one has to add nested compounds explicitly via \ingroup. -# The default value is: NO. - -GROUP_NESTED_COMPOUNDS = NO - -# Set the SUBGROUPING tag to YES to allow class member groups of the same type -# (for instance a group of public functions) to be put as a subgroup of that -# type (e.g. under the Public Functions section). Set it to NO to prevent -# subgrouping. Alternatively, this can be done per class using the -# \nosubgrouping command. -# The default value is: YES. - -SUBGROUPING = YES - -# When the INLINE_GROUPED_CLASSES tag is set to YES, classes, structs and unions -# are shown inside the group in which they are included (e.g. using \ingroup) -# instead of on a separate page (for HTML and Man pages) or section (for LaTeX -# and RTF). -# -# Note that this feature does not work in combination with -# SEPARATE_MEMBER_PAGES. -# The default value is: NO. - -INLINE_GROUPED_CLASSES = NO - -# When the INLINE_SIMPLE_STRUCTS tag is set to YES, structs, classes, and unions -# with only public data fields or simple typedef fields will be shown inline in -# the documentation of the scope in which they are defined (i.e. file, -# namespace, or group documentation), provided this scope is documented. If set -# to NO, structs, classes, and unions are shown on a separate page (for HTML and -# Man pages) or section (for LaTeX and RTF). -# The default value is: NO. - -INLINE_SIMPLE_STRUCTS = NO - -# When TYPEDEF_HIDES_STRUCT tag is enabled, a typedef of a struct, union, or -# enum is documented as struct, union, or enum with the name of the typedef. So -# typedef struct TypeS {} TypeT, will appear in the documentation as a struct -# with name TypeT. When disabled the typedef will appear as a member of a file, -# namespace, or class. And the struct will be named TypeS. This can typically be -# useful for C code in case the coding convention dictates that all compound -# types are typedef'ed and only the typedef is referenced, never the tag name. -# The default value is: NO. - -TYPEDEF_HIDES_STRUCT = NO - -# The size of the symbol lookup cache can be set using LOOKUP_CACHE_SIZE. This -# cache is used to resolve symbols given their name and scope. Since this can be -# an expensive process and often the same symbol appears multiple times in the -# code, doxygen keeps a cache of pre-resolved symbols. If the cache is too small -# doxygen will become slower. If the cache is too large, memory is wasted. The -# cache size is given by this formula: 2^(16+LOOKUP_CACHE_SIZE). The valid range -# is 0..9, the default is 0, corresponding to a cache size of 2^16=65536 -# symbols. At the end of a run doxygen will report the cache usage and suggest -# the optimal cache size from a speed point of view. -# Minimum value: 0, maximum value: 9, default value: 0. - -LOOKUP_CACHE_SIZE = 0 - -#--------------------------------------------------------------------------- -# Build related configuration options -#--------------------------------------------------------------------------- - -# If the EXTRACT_ALL tag is set to YES, doxygen will assume all entities in -# documentation are documented, even if no documentation was available. Private -# class members and static file members will be hidden unless the -# EXTRACT_PRIVATE respectively EXTRACT_STATIC tags are set to YES. -# Note: This will also disable the warnings about undocumented members that are -# normally produced when WARNINGS is set to YES. -# The default value is: NO. - -EXTRACT_ALL = NO - -# If the EXTRACT_PRIVATE tag is set to YES, all private members of a class will -# be included in the documentation. -# The default value is: NO. - -EXTRACT_PRIVATE = NO - -# If the EXTRACT_PACKAGE tag is set to YES, all members with package or internal -# scope will be included in the documentation. -# The default value is: NO. - -EXTRACT_PACKAGE = NO - -# If the EXTRACT_STATIC tag is set to YES, all static members of a file will be -# included in the documentation. -# The default value is: NO. - -EXTRACT_STATIC = NO - -# If the EXTRACT_LOCAL_CLASSES tag is set to YES, classes (and structs) defined -# locally in source files will be included in the documentation. If set to NO, -# only classes defined in header files are included. Does not have any effect -# for Java sources. -# The default value is: YES. - -EXTRACT_LOCAL_CLASSES = YES - -# This flag is only useful for Objective-C code. If set to YES, local methods, -# which are defined in the implementation section but not in the interface are -# included in the documentation. If set to NO, only methods in the interface are -# included. -# The default value is: NO. - -EXTRACT_LOCAL_METHODS = NO - -# If this flag is set to YES, the members of anonymous namespaces will be -# extracted and appear in the documentation as a namespace called -# 'anonymous_namespace{file}', where file will be replaced with the base name of -# the file that contains the anonymous namespace. By default anonymous namespace -# are hidden. -# The default value is: NO. - -EXTRACT_ANON_NSPACES = NO - -# If the HIDE_UNDOC_MEMBERS tag is set to YES, doxygen will hide all -# undocumented members inside documented classes or files. If set to NO these -# members will be included in the various overviews, but no documentation -# section is generated. This option has no effect if EXTRACT_ALL is enabled. -# The default value is: NO. - -HIDE_UNDOC_MEMBERS = NO - -# If the HIDE_UNDOC_CLASSES tag is set to YES, doxygen will hide all -# undocumented classes that are normally visible in the class hierarchy. If set -# to NO, these classes will be included in the various overviews. This option -# has no effect if EXTRACT_ALL is enabled. -# The default value is: NO. - -HIDE_UNDOC_CLASSES = NO - -# If the HIDE_FRIEND_COMPOUNDS tag is set to YES, doxygen will hide all friend -# (class|struct|union) declarations. If set to NO, these declarations will be -# included in the documentation. -# The default value is: NO. - -HIDE_FRIEND_COMPOUNDS = NO - -# If the HIDE_IN_BODY_DOCS tag is set to YES, doxygen will hide any -# documentation blocks found inside the body of a function. If set to NO, these -# blocks will be appended to the function's detailed documentation block. -# The default value is: NO. - -HIDE_IN_BODY_DOCS = NO - -# The INTERNAL_DOCS tag determines if documentation that is typed after a -# \internal command is included. If the tag is set to NO then the documentation -# will be excluded. Set it to YES to include the internal documentation. -# The default value is: NO. - -INTERNAL_DOCS = NO - -# If the CASE_SENSE_NAMES tag is set to NO then doxygen will only generate file -# names in lower-case letters. If set to YES, upper-case letters are also -# allowed. This is useful if you have classes or files whose names only differ -# in case and if your file system supports case sensitive file names. Windows -# and Mac users are advised to set this option to NO. -# The default value is: system dependent. - -CASE_SENSE_NAMES = YES - -# If the HIDE_SCOPE_NAMES tag is set to NO then doxygen will show members with -# their full class and namespace scopes in the documentation. If set to YES, the -# scope will be hidden. -# The default value is: NO. - -HIDE_SCOPE_NAMES = NO - -# If the HIDE_COMPOUND_REFERENCE tag is set to NO (default) then doxygen will -# append additional text to a page's title, such as Class Reference. If set to -# YES the compound reference will be hidden. -# The default value is: NO. - -HIDE_COMPOUND_REFERENCE= NO - -# If the SHOW_INCLUDE_FILES tag is set to YES then doxygen will put a list of -# the files that are included by a file in the documentation of that file. -# The default value is: YES. - -SHOW_INCLUDE_FILES = YES - -# If the SHOW_GROUPED_MEMB_INC tag is set to YES then Doxygen will add for each -# grouped member an include statement to the documentation, telling the reader -# which file to include in order to use the member. -# The default value is: NO. - -SHOW_GROUPED_MEMB_INC = NO - -# If the FORCE_LOCAL_INCLUDES tag is set to YES then doxygen will list include -# files with double quotes in the documentation rather than with sharp brackets. -# The default value is: NO. - -FORCE_LOCAL_INCLUDES = NO - -# If the INLINE_INFO tag is set to YES then a tag [inline] is inserted in the -# documentation for inline members. -# The default value is: YES. - -INLINE_INFO = YES - -# If the SORT_MEMBER_DOCS tag is set to YES then doxygen will sort the -# (detailed) documentation of file and class members alphabetically by member -# name. If set to NO, the members will appear in declaration order. -# The default value is: YES. - -SORT_MEMBER_DOCS = YES - -# If the SORT_BRIEF_DOCS tag is set to YES then doxygen will sort the brief -# descriptions of file, namespace and class members alphabetically by member -# name. If set to NO, the members will appear in declaration order. Note that -# this will also influence the order of the classes in the class list. -# The default value is: NO. - -SORT_BRIEF_DOCS = NO - -# If the SORT_MEMBERS_CTORS_1ST tag is set to YES then doxygen will sort the -# (brief and detailed) documentation of class members so that constructors and -# destructors are listed first. If set to NO the constructors will appear in the -# respective orders defined by SORT_BRIEF_DOCS and SORT_MEMBER_DOCS. -# Note: If SORT_BRIEF_DOCS is set to NO this option is ignored for sorting brief -# member documentation. -# Note: If SORT_MEMBER_DOCS is set to NO this option is ignored for sorting -# detailed member documentation. -# The default value is: NO. - -SORT_MEMBERS_CTORS_1ST = NO - -# If the SORT_GROUP_NAMES tag is set to YES then doxygen will sort the hierarchy -# of group names into alphabetical order. If set to NO the group names will -# appear in their defined order. -# The default value is: NO. - -SORT_GROUP_NAMES = NO - -# If the SORT_BY_SCOPE_NAME tag is set to YES, the class list will be sorted by -# fully-qualified names, including namespaces. If set to NO, the class list will -# be sorted only by class name, not including the namespace part. -# Note: This option is not very useful if HIDE_SCOPE_NAMES is set to YES. -# Note: This option applies only to the class list, not to the alphabetical -# list. -# The default value is: NO. - -SORT_BY_SCOPE_NAME = NO - -# If the STRICT_PROTO_MATCHING option is enabled and doxygen fails to do proper -# type resolution of all parameters of a function it will reject a match between -# the prototype and the implementation of a member function even if there is -# only one candidate or it is obvious which candidate to choose by doing a -# simple string match. By disabling STRICT_PROTO_MATCHING doxygen will still -# accept a match between prototype and implementation in such cases. -# The default value is: NO. - -STRICT_PROTO_MATCHING = NO - -# The GENERATE_TODOLIST tag can be used to enable (YES) or disable (NO) the todo -# list. This list is created by putting \todo commands in the documentation. -# The default value is: YES. - -GENERATE_TODOLIST = YES - -# The GENERATE_TESTLIST tag can be used to enable (YES) or disable (NO) the test -# list. This list is created by putting \test commands in the documentation. -# The default value is: YES. - -GENERATE_TESTLIST = YES - -# The GENERATE_BUGLIST tag can be used to enable (YES) or disable (NO) the bug -# list. This list is created by putting \bug commands in the documentation. -# The default value is: YES. - -GENERATE_BUGLIST = YES - -# The GENERATE_DEPRECATEDLIST tag can be used to enable (YES) or disable (NO) -# the deprecated list. This list is created by putting \deprecated commands in -# the documentation. -# The default value is: YES. - -GENERATE_DEPRECATEDLIST= YES - -# The ENABLED_SECTIONS tag can be used to enable conditional documentation -# sections, marked by \if ... \endif and \cond -# ... \endcond blocks. - -ENABLED_SECTIONS = - -# The MAX_INITIALIZER_LINES tag determines the maximum number of lines that the -# initial value of a variable or macro / define can have for it to appear in the -# documentation. If the initializer consists of more lines than specified here -# it will be hidden. Use a value of 0 to hide initializers completely. The -# appearance of the value of individual variables and macros / defines can be -# controlled using \showinitializer or \hideinitializer command in the -# documentation regardless of this setting. -# Minimum value: 0, maximum value: 10000, default value: 30. - -MAX_INITIALIZER_LINES = 30 - -# Set the SHOW_USED_FILES tag to NO to disable the list of files generated at -# the bottom of the documentation of classes and structs. If set to YES, the -# list will mention the files that were used to generate the documentation. -# The default value is: YES. - -SHOW_USED_FILES = YES - -# Set the SHOW_FILES tag to NO to disable the generation of the Files page. This -# will remove the Files entry from the Quick Index and from the Folder Tree View -# (if specified). -# The default value is: YES. - -SHOW_FILES = YES - -# Set the SHOW_NAMESPACES tag to NO to disable the generation of the Namespaces -# page. This will remove the Namespaces entry from the Quick Index and from the -# Folder Tree View (if specified). -# The default value is: YES. - -SHOW_NAMESPACES = YES - -# The FILE_VERSION_FILTER tag can be used to specify a program or script that -# doxygen should invoke to get the current version for each file (typically from -# the version control system). Doxygen will invoke the program by executing (via -# popen()) the command command input-file, where command is the value of the -# FILE_VERSION_FILTER tag, and input-file is the name of an input file provided -# by doxygen. Whatever the program writes to standard output is used as the file -# version. For an example see the documentation. - -FILE_VERSION_FILTER = - -# The LAYOUT_FILE tag can be used to specify a layout file which will be parsed -# by doxygen. The layout file controls the global structure of the generated -# output files in an output format independent way. To create the layout file -# that represents doxygen's defaults, run doxygen with the -l option. You can -# optionally specify a file name after the option, if omitted DoxygenLayout.xml -# will be used as the name of the layout file. -# -# Note that if you run doxygen from a directory containing a file called -# DoxygenLayout.xml, doxygen will parse it automatically even if the LAYOUT_FILE -# tag is left empty. - -LAYOUT_FILE = - -# The CITE_BIB_FILES tag can be used to specify one or more bib files containing -# the reference definitions. This must be a list of .bib files. The .bib -# extension is automatically appended if omitted. This requires the bibtex tool -# to be installed. See also http://en.wikipedia.org/wiki/BibTeX for more info. -# For LaTeX the style of the bibliography can be controlled using -# LATEX_BIB_STYLE. To use this feature you need bibtex and perl available in the -# search path. See also \cite for info how to create references. - -CITE_BIB_FILES = - -#--------------------------------------------------------------------------- -# Configuration options related to warning and progress messages -#--------------------------------------------------------------------------- - -# The QUIET tag can be used to turn on/off the messages that are generated to -# standard output by doxygen. If QUIET is set to YES this implies that the -# messages are off. -# The default value is: NO. - -QUIET = NO - -# The WARNINGS tag can be used to turn on/off the warning messages that are -# generated to standard error (stderr) by doxygen. If WARNINGS is set to YES -# this implies that the warnings are on. -# -# Tip: Turn warnings on while writing the documentation. -# The default value is: YES. - -WARNINGS = YES - -# If the WARN_IF_UNDOCUMENTED tag is set to YES then doxygen will generate -# warnings for undocumented members. If EXTRACT_ALL is set to YES then this flag -# will automatically be disabled. -# The default value is: YES. - -WARN_IF_UNDOCUMENTED = YES - -# If the WARN_IF_DOC_ERROR tag is set to YES, doxygen will generate warnings for -# potential errors in the documentation, such as not documenting some parameters -# in a documented function, or documenting parameters that don't exist or using -# markup commands wrongly. -# The default value is: YES. - -WARN_IF_DOC_ERROR = YES - -# This WARN_NO_PARAMDOC option can be enabled to get warnings for functions that -# are documented, but have no documentation for their parameters or return -# value. If set to NO, doxygen will only warn about wrong or incomplete -# parameter documentation, but not about the absence of documentation. -# The default value is: NO. - -WARN_NO_PARAMDOC = NO - -# If the WARN_AS_ERROR tag is set to YES then doxygen will immediately stop when -# a warning is encountered. -# The default value is: NO. - -WARN_AS_ERROR = NO - -# The WARN_FORMAT tag determines the format of the warning messages that doxygen -# can produce. The string should contain the $file, $line, and $text tags, which -# will be replaced by the file and line number from which the warning originated -# and the warning text. Optionally the format may contain $version, which will -# be replaced by the version of the file (if it could be obtained via -# FILE_VERSION_FILTER) -# The default value is: $file:$line: $text. - -WARN_FORMAT = "$file:$line: $text" - -# The WARN_LOGFILE tag can be used to specify a file to which warning and error -# messages should be written. If left blank the output is written to standard -# error (stderr). - -WARN_LOGFILE = - -#--------------------------------------------------------------------------- -# Configuration options related to the input files -#--------------------------------------------------------------------------- - -# The INPUT tag is used to specify the files and/or directories that contain -# documented source files. You may enter file names like myfile.cpp or -# directories like /usr/src/myproject. Separate the files or directories with -# spaces. See also FILE_PATTERNS and EXTENSION_MAPPING -# Note: If this tag is empty the current directory is searched. - -INPUT = include README.md - -# This tag can be used to specify the character encoding of the source files -# that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses -# libiconv (or the iconv built into libc) for the transcoding. See the libiconv -# documentation (see: http://www.gnu.org/software/libiconv) for the list of -# possible encodings. -# The default value is: UTF-8. - -INPUT_ENCODING = UTF-8 - -# If the value of the INPUT tag contains directories, you can use the -# FILE_PATTERNS tag to specify one or more wildcard patterns (like *.cpp and -# *.h) to filter out the source-files in the directories. -# -# Note that for custom extensions or not directly supported extensions you also -# need to set EXTENSION_MAPPING for the extension otherwise the files are not -# read by doxygen. -# -# If left blank the following patterns are tested:*.c, *.cc, *.cxx, *.cpp, -# *.c++, *.java, *.ii, *.ixx, *.ipp, *.i++, *.inl, *.idl, *.ddl, *.odl, *.h, -# *.hh, *.hxx, *.hpp, *.h++, *.cs, *.d, *.php, *.php4, *.php5, *.phtml, *.inc, -# *.m, *.markdown, *.md, *.mm, *.dox, *.py, *.pyw, *.f90, *.f95, *.f03, *.f08, -# *.f, *.for, *.tcl, *.vhd, *.vhdl, *.ucf and *.qsf. - -FILE_PATTERNS = *.c \ - *.cc \ - *.cxx \ - *.cpp \ - *.c++ \ - *.java \ - *.ii \ - *.ixx \ - *.ipp \ - *.i++ \ - *.inl \ - *.idl \ - *.ddl \ - *.odl \ - *.h \ - *.hh \ - *.hxx \ - *.hpp \ - *.h++ \ - *.cs \ - *.d \ - *.php \ - *.php4 \ - *.php5 \ - *.phtml \ - *.inc \ - *.m \ - *.markdown \ - *.md \ - *.mm \ - *.dox \ - *.py \ - *.pyw \ - *.f90 \ - *.f95 \ - *.f03 \ - *.f08 \ - *.f \ - *.for \ - *.tcl \ - *.vhd \ - *.vhdl \ - *.ucf \ - *.qsf - -# The RECURSIVE tag can be used to specify whether or not subdirectories should -# be searched for input files as well. -# The default value is: NO. - -RECURSIVE = YES - -# The EXCLUDE tag can be used to specify files and/or directories that should be -# excluded from the INPUT source files. This way you can easily exclude a -# subdirectory from a directory tree whose root is specified with the INPUT tag. -# -# Note that relative paths are relative to the directory from which doxygen is -# run. - -EXCLUDE = include/jwt-cpp/picojson.h - -# The EXCLUDE_SYMLINKS tag can be used to select whether or not files or -# directories that are symbolic links (a Unix file system feature) are excluded -# from the input. -# The default value is: NO. - -EXCLUDE_SYMLINKS = NO - -# If the value of the INPUT tag contains directories, you can use the -# EXCLUDE_PATTERNS tag to specify one or more wildcard patterns to exclude -# certain files from those directories. -# -# Note that the wildcards are matched against the file with absolute path, so to -# exclude all test directories for example use the pattern */test/* - -EXCLUDE_PATTERNS = *nlohmann*, *picojson* - -# The EXCLUDE_SYMBOLS tag can be used to specify one or more symbol names -# (namespaces, classes, functions, etc.) that should be excluded from the -# output. The symbol name can be a fully qualified name, a word, or if the -# wildcard * is used, a substring. Examples: ANamespace, AClass, -# AClass::ANamespace, ANamespace::*Test -# -# Note that the wildcards are matched against the file with absolute path, so to -# exclude all test directories use the pattern */test/* - -EXCLUDE_SYMBOLS = jwt::details - -# The EXAMPLE_PATH tag can be used to specify one or more files or directories -# that contain example code fragments that are included (see the \include -# command). - -EXAMPLE_PATH = example - -# If the value of the EXAMPLE_PATH tag contains directories, you can use the -# EXAMPLE_PATTERNS tag to specify one or more wildcard pattern (like *.cpp and -# *.h) to filter out the source-files in the directories. If left blank all -# files are included. - -EXAMPLE_PATTERNS = * - -# If the EXAMPLE_RECURSIVE tag is set to YES then subdirectories will be -# searched for input files to be used with the \include or \dontinclude commands -# irrespective of the value of the RECURSIVE tag. -# The default value is: NO. - -EXAMPLE_RECURSIVE = NO - -# The IMAGE_PATH tag can be used to specify one or more files or directories -# that contain images that are to be included in the documentation (see the -# \image command). - -IMAGE_PATH = - -# The INPUT_FILTER tag can be used to specify a program that doxygen should -# invoke to filter for each input file. Doxygen will invoke the filter program -# by executing (via popen()) the command: -# -# -# -# where is the value of the INPUT_FILTER tag, and is the -# name of an input file. Doxygen will then use the output that the filter -# program writes to standard output. If FILTER_PATTERNS is specified, this tag -# will be ignored. -# -# Note that the filter must not add or remove lines; it is applied before the -# code is scanned, but not when the output code is generated. If lines are added -# or removed, the anchors will not be placed correctly. -# -# Note that for custom extensions or not directly supported extensions you also -# need to set EXTENSION_MAPPING for the extension otherwise the files are not -# properly processed by doxygen. - -INPUT_FILTER = - -# The FILTER_PATTERNS tag can be used to specify filters on a per file pattern -# basis. Doxygen will compare the file name with each pattern and apply the -# filter if there is a match. The filters are a list of the form: pattern=filter -# (like *.cpp=my_cpp_filter). See INPUT_FILTER for further information on how -# filters are used. If the FILTER_PATTERNS tag is empty or if none of the -# patterns match the file name, INPUT_FILTER is applied. -# -# Note that for custom extensions or not directly supported extensions you also -# need to set EXTENSION_MAPPING for the extension otherwise the files are not -# properly processed by doxygen. - -FILTER_PATTERNS = - -# If the FILTER_SOURCE_FILES tag is set to YES, the input filter (if set using -# INPUT_FILTER) will also be used to filter the input files that are used for -# producing the source files to browse (i.e. when SOURCE_BROWSER is set to YES). -# The default value is: NO. - -FILTER_SOURCE_FILES = NO - -# The FILTER_SOURCE_PATTERNS tag can be used to specify source filters per file -# pattern. A pattern will override the setting for FILTER_PATTERN (if any) and -# it is also possible to disable source filtering for a specific pattern using -# *.ext= (so without naming a filter). -# This tag requires that the tag FILTER_SOURCE_FILES is set to YES. - -FILTER_SOURCE_PATTERNS = - -# If the USE_MDFILE_AS_MAINPAGE tag refers to the name of a markdown file that -# is part of the input, its contents will be placed on the main page -# (index.html). This can be useful if you have a project on for instance GitHub -# and want to reuse the introduction page also for the doxygen output. - -USE_MDFILE_AS_MAINPAGE = README.md - -#--------------------------------------------------------------------------- -# Configuration options related to source browsing -#--------------------------------------------------------------------------- - -# If the SOURCE_BROWSER tag is set to YES then a list of source files will be -# generated. Documented entities will be cross-referenced with these sources. -# -# Note: To get rid of all source code in the generated output, make sure that -# also VERBATIM_HEADERS is set to NO. -# The default value is: NO. - -SOURCE_BROWSER = NO - -# Setting the INLINE_SOURCES tag to YES will include the body of functions, -# classes and enums directly into the documentation. -# The default value is: NO. - -INLINE_SOURCES = NO - -# Setting the STRIP_CODE_COMMENTS tag to YES will instruct doxygen to hide any -# special comment blocks from generated source code fragments. Normal C, C++ and -# Fortran comments will always remain visible. -# The default value is: YES. - -STRIP_CODE_COMMENTS = YES - -# If the REFERENCED_BY_RELATION tag is set to YES then for each documented -# function all documented functions referencing it will be listed. -# The default value is: NO. - -REFERENCED_BY_RELATION = NO - -# If the REFERENCES_RELATION tag is set to YES then for each documented function -# all documented entities called/used by that function will be listed. -# The default value is: NO. - -REFERENCES_RELATION = NO - -# If the REFERENCES_LINK_SOURCE tag is set to YES and SOURCE_BROWSER tag is set -# to YES then the hyperlinks from functions in REFERENCES_RELATION and -# REFERENCED_BY_RELATION lists will link to the source code. Otherwise they will -# link to the documentation. -# The default value is: YES. - -REFERENCES_LINK_SOURCE = YES - -# If SOURCE_TOOLTIPS is enabled (the default) then hovering a hyperlink in the -# source code will show a tooltip with additional information such as prototype, -# brief description and links to the definition and documentation. Since this -# will make the HTML file larger and loading of large files a bit slower, you -# can opt to disable this feature. -# The default value is: YES. -# This tag requires that the tag SOURCE_BROWSER is set to YES. - -SOURCE_TOOLTIPS = YES - -# If the USE_HTAGS tag is set to YES then the references to source code will -# point to the HTML generated by the htags(1) tool instead of doxygen built-in -# source browser. The htags tool is part of GNU's global source tagging system -# (see http://www.gnu.org/software/global/global.html). You will need version -# 4.8.6 or higher. -# -# To use it do the following: -# - Install the latest version of global -# - Enable SOURCE_BROWSER and USE_HTAGS in the config file -# - Make sure the INPUT points to the root of the source tree -# - Run doxygen as normal -# -# Doxygen will invoke htags (and that will in turn invoke gtags), so these -# tools must be available from the command line (i.e. in the search path). -# -# The result: instead of the source browser generated by doxygen, the links to -# source code will now point to the output of htags. -# The default value is: NO. -# This tag requires that the tag SOURCE_BROWSER is set to YES. - -USE_HTAGS = NO - -# If the VERBATIM_HEADERS tag is set the YES then doxygen will generate a -# verbatim copy of the header file for each class for which an include is -# specified. Set to NO to disable this. -# See also: Section \class. -# The default value is: YES. - -VERBATIM_HEADERS = YES - -# If the CLANG_ASSISTED_PARSING tag is set to YES then doxygen will use the -# clang parser (see: http://clang.llvm.org/) for more accurate parsing at the -# cost of reduced performance. This can be particularly helpful with template -# rich C++ code for which doxygen's built-in parser lacks the necessary type -# information. -# Note: The availability of this option depends on whether or not doxygen was -# generated with the -Duse-libclang=ON option for CMake. -# The default value is: NO. - -CLANG_ASSISTED_PARSING = NO - -# If clang assisted parsing is enabled you can provide the compiler with command -# line options that you would normally use when invoking the compiler. Note that -# the include paths will already be set by doxygen for the files and directories -# specified with INPUT and INCLUDE_PATH. -# This tag requires that the tag CLANG_ASSISTED_PARSING is set to YES. - -CLANG_OPTIONS = - -#--------------------------------------------------------------------------- -# Configuration options related to the alphabetical class index -#--------------------------------------------------------------------------- - -# If the ALPHABETICAL_INDEX tag is set to YES, an alphabetical index of all -# compounds will be generated. Enable this if the project contains a lot of -# classes, structs, unions or interfaces. -# The default value is: YES. - -ALPHABETICAL_INDEX = YES - -# The COLS_IN_ALPHA_INDEX tag can be used to specify the number of columns in -# which the alphabetical index list will be split. -# Minimum value: 1, maximum value: 20, default value: 5. -# This tag requires that the tag ALPHABETICAL_INDEX is set to YES. - -COLS_IN_ALPHA_INDEX = 5 - -# In case all classes in a project start with a common prefix, all classes will -# be put under the same header in the alphabetical index. The IGNORE_PREFIX tag -# can be used to specify a prefix (or a list of prefixes) that should be ignored -# while generating the index headers. -# This tag requires that the tag ALPHABETICAL_INDEX is set to YES. - -IGNORE_PREFIX = - -#--------------------------------------------------------------------------- -# Configuration options related to the HTML output -#--------------------------------------------------------------------------- - -# If the GENERATE_HTML tag is set to YES, doxygen will generate HTML output -# The default value is: YES. - -GENERATE_HTML = YES - -# The HTML_OUTPUT tag is used to specify where the HTML docs will be put. If a -# relative path is entered the value of OUTPUT_DIRECTORY will be put in front of -# it. -# The default directory is: html. -# This tag requires that the tag GENERATE_HTML is set to YES. - -HTML_OUTPUT = html - -# The HTML_FILE_EXTENSION tag can be used to specify the file extension for each -# generated HTML page (for example: .htm, .php, .asp). -# The default value is: .html. -# This tag requires that the tag GENERATE_HTML is set to YES. - -HTML_FILE_EXTENSION = .html - -# The HTML_HEADER tag can be used to specify a user-defined HTML header file for -# each generated HTML page. If the tag is left blank doxygen will generate a -# standard header. -# -# To get valid HTML the header file that includes any scripts and style sheets -# that doxygen needs, which is dependent on the configuration options used (e.g. -# the setting GENERATE_TREEVIEW). It is highly recommended to start with a -# default header using -# doxygen -w html new_header.html new_footer.html new_stylesheet.css -# YourConfigFile -# and then modify the file new_header.html. See also section "Doxygen usage" -# for information on how to generate the default header that doxygen normally -# uses. -# Note: The header is subject to change so you typically have to regenerate the -# default header when upgrading to a newer version of doxygen. For a description -# of the possible markers and block names see the documentation. -# This tag requires that the tag GENERATE_HTML is set to YES. - -HTML_HEADER = - -# The HTML_FOOTER tag can be used to specify a user-defined HTML footer for each -# generated HTML page. If the tag is left blank doxygen will generate a standard -# footer. See HTML_HEADER for more information on how to generate a default -# footer and what special commands can be used inside the footer. See also -# section "Doxygen usage" for information on how to generate the default footer -# that doxygen normally uses. -# This tag requires that the tag GENERATE_HTML is set to YES. - -HTML_FOOTER = - -# The HTML_STYLESHEET tag can be used to specify a user-defined cascading style -# sheet that is used by each HTML page. It can be used to fine-tune the look of -# the HTML output. If left blank doxygen will generate a default style sheet. -# See also section "Doxygen usage" for information on how to generate the style -# sheet that doxygen normally uses. -# Note: It is recommended to use HTML_EXTRA_STYLESHEET instead of this tag, as -# it is more robust and this tag (HTML_STYLESHEET) will in the future become -# obsolete. -# This tag requires that the tag GENERATE_HTML is set to YES. - -HTML_STYLESHEET = - -# The HTML_EXTRA_STYLESHEET tag can be used to specify additional user-defined -# cascading style sheets that are included after the standard style sheets -# created by doxygen. Using this option one can overrule certain style aspects. -# This is preferred over using HTML_STYLESHEET since it does not replace the -# standard style sheet and is therefore more robust against future updates. -# Doxygen will copy the style sheet files to the output directory. -# Note: The order of the extra style sheet files is of importance (e.g. the last -# style sheet in the list overrules the setting of the previous ones in the -# list). For an example see the documentation. -# This tag requires that the tag GENERATE_HTML is set to YES. - -HTML_EXTRA_STYLESHEET = - -# The HTML_EXTRA_FILES tag can be used to specify one or more extra images or -# other source files which should be copied to the HTML output directory. Note -# that these files will be copied to the base HTML output directory. Use the -# $relpath^ marker in the HTML_HEADER and/or HTML_FOOTER files to load these -# files. In the HTML_STYLESHEET file, use the file name only. Also note that the -# files will be copied as-is; there are no commands or markers available. -# This tag requires that the tag GENERATE_HTML is set to YES. - -HTML_EXTRA_FILES = - -# The HTML_COLORSTYLE_HUE tag controls the color of the HTML output. Doxygen -# will adjust the colors in the style sheet and background images according to -# this color. Hue is specified as an angle on a colorwheel, see -# http://en.wikipedia.org/wiki/Hue for more information. For instance the value -# 0 represents red, 60 is yellow, 120 is green, 180 is cyan, 240 is blue, 300 -# purple, and 360 is red again. -# Minimum value: 0, maximum value: 359, default value: 220. -# This tag requires that the tag GENERATE_HTML is set to YES. - -HTML_COLORSTYLE_HUE = 220 - -# The HTML_COLORSTYLE_SAT tag controls the purity (or saturation) of the colors -# in the HTML output. For a value of 0 the output will use grayscales only. A -# value of 255 will produce the most vivid colors. -# Minimum value: 0, maximum value: 255, default value: 100. -# This tag requires that the tag GENERATE_HTML is set to YES. - -HTML_COLORSTYLE_SAT = 100 - -# The HTML_COLORSTYLE_GAMMA tag controls the gamma correction applied to the -# luminance component of the colors in the HTML output. Values below 100 -# gradually make the output lighter, whereas values above 100 make the output -# darker. The value divided by 100 is the actual gamma applied, so 80 represents -# a gamma of 0.8, The value 220 represents a gamma of 2.2, and 100 does not -# change the gamma. -# Minimum value: 40, maximum value: 240, default value: 80. -# This tag requires that the tag GENERATE_HTML is set to YES. - -HTML_COLORSTYLE_GAMMA = 80 - -# If the HTML_TIMESTAMP tag is set to YES then the footer of each generated HTML -# page will contain the date and time when the page was generated. Setting this -# to YES can help to show when doxygen was last run and thus if the -# documentation is up to date. -# The default value is: NO. -# This tag requires that the tag GENERATE_HTML is set to YES. - -HTML_TIMESTAMP = NO - -# If the HTML_DYNAMIC_SECTIONS tag is set to YES then the generated HTML -# documentation will contain sections that can be hidden and shown after the -# page has loaded. -# The default value is: NO. -# This tag requires that the tag GENERATE_HTML is set to YES. - -HTML_DYNAMIC_SECTIONS = NO - -# With HTML_INDEX_NUM_ENTRIES one can control the preferred number of entries -# shown in the various tree structured indices initially; the user can expand -# and collapse entries dynamically later on. Doxygen will expand the tree to -# such a level that at most the specified number of entries are visible (unless -# a fully collapsed tree already exceeds this amount). So setting the number of -# entries 1 will produce a full collapsed tree by default. 0 is a special value -# representing an infinite number of entries and will result in a full expanded -# tree by default. -# Minimum value: 0, maximum value: 9999, default value: 100. -# This tag requires that the tag GENERATE_HTML is set to YES. - -HTML_INDEX_NUM_ENTRIES = 100 - -# If the GENERATE_DOCSET tag is set to YES, additional index files will be -# generated that can be used as input for Apple's Xcode 3 integrated development -# environment (see: http://developer.apple.com/tools/xcode/), introduced with -# OSX 10.5 (Leopard). To create a documentation set, doxygen will generate a -# Makefile in the HTML output directory. Running make will produce the docset in -# that directory and running make install will install the docset in -# ~/Library/Developer/Shared/Documentation/DocSets so that Xcode will find it at -# startup. See http://developer.apple.com/tools/creatingdocsetswithdoxygen.html -# for more information. -# The default value is: NO. -# This tag requires that the tag GENERATE_HTML is set to YES. - -GENERATE_DOCSET = NO - -# This tag determines the name of the docset feed. A documentation feed provides -# an umbrella under which multiple documentation sets from a single provider -# (such as a company or product suite) can be grouped. -# The default value is: Doxygen generated docs. -# This tag requires that the tag GENERATE_DOCSET is set to YES. - -DOCSET_FEEDNAME = "Doxygen generated docs" - -# This tag specifies a string that should uniquely identify the documentation -# set bundle. This should be a reverse domain-name style string, e.g. -# com.mycompany.MyDocSet. Doxygen will append .docset to the name. -# The default value is: org.doxygen.Project. -# This tag requires that the tag GENERATE_DOCSET is set to YES. - -DOCSET_BUNDLE_ID = org.doxygen.Project - -# The DOCSET_PUBLISHER_ID tag specifies a string that should uniquely identify -# the documentation publisher. This should be a reverse domain-name style -# string, e.g. com.mycompany.MyDocSet.documentation. -# The default value is: org.doxygen.Publisher. -# This tag requires that the tag GENERATE_DOCSET is set to YES. - -DOCSET_PUBLISHER_ID = org.doxygen.Publisher - -# The DOCSET_PUBLISHER_NAME tag identifies the documentation publisher. -# The default value is: Publisher. -# This tag requires that the tag GENERATE_DOCSET is set to YES. - -DOCSET_PUBLISHER_NAME = Publisher - -# If the GENERATE_HTMLHELP tag is set to YES then doxygen generates three -# additional HTML index files: index.hhp, index.hhc, and index.hhk. The -# index.hhp is a project file that can be read by Microsoft's HTML Help Workshop -# (see: http://www.microsoft.com/en-us/download/details.aspx?id=21138) on -# Windows. -# -# The HTML Help Workshop contains a compiler that can convert all HTML output -# generated by doxygen into a single compiled HTML file (.chm). Compiled HTML -# files are now used as the Windows 98 help format, and will replace the old -# Windows help format (.hlp) on all Windows platforms in the future. Compressed -# HTML files also contain an index, a table of contents, and you can search for -# words in the documentation. The HTML workshop also contains a viewer for -# compressed HTML files. -# The default value is: NO. -# This tag requires that the tag GENERATE_HTML is set to YES. - -GENERATE_HTMLHELP = NO - -# The CHM_FILE tag can be used to specify the file name of the resulting .chm -# file. You can add a path in front of the file if the result should not be -# written to the html output directory. -# This tag requires that the tag GENERATE_HTMLHELP is set to YES. - -CHM_FILE = - -# The HHC_LOCATION tag can be used to specify the location (absolute path -# including file name) of the HTML help compiler (hhc.exe). If non-empty, -# doxygen will try to run the HTML help compiler on the generated index.hhp. -# The file has to be specified with full path. -# This tag requires that the tag GENERATE_HTMLHELP is set to YES. - -HHC_LOCATION = - -# The GENERATE_CHI flag controls if a separate .chi index file is generated -# (YES) or that it should be included in the master .chm file (NO). -# The default value is: NO. -# This tag requires that the tag GENERATE_HTMLHELP is set to YES. - -GENERATE_CHI = NO - -# The CHM_INDEX_ENCODING is used to encode HtmlHelp index (hhk), content (hhc) -# and project file content. -# This tag requires that the tag GENERATE_HTMLHELP is set to YES. - -CHM_INDEX_ENCODING = - -# The BINARY_TOC flag controls whether a binary table of contents is generated -# (YES) or a normal table of contents (NO) in the .chm file. Furthermore it -# enables the Previous and Next buttons. -# The default value is: NO. -# This tag requires that the tag GENERATE_HTMLHELP is set to YES. - -BINARY_TOC = NO - -# The TOC_EXPAND flag can be set to YES to add extra items for group members to -# the table of contents of the HTML help documentation and to the tree view. -# The default value is: NO. -# This tag requires that the tag GENERATE_HTMLHELP is set to YES. - -TOC_EXPAND = NO - -# If the GENERATE_QHP tag is set to YES and both QHP_NAMESPACE and -# QHP_VIRTUAL_FOLDER are set, an additional index file will be generated that -# can be used as input for Qt's qhelpgenerator to generate a Qt Compressed Help -# (.qch) of the generated HTML documentation. -# The default value is: NO. -# This tag requires that the tag GENERATE_HTML is set to YES. - -GENERATE_QHP = NO - -# If the QHG_LOCATION tag is specified, the QCH_FILE tag can be used to specify -# the file name of the resulting .qch file. The path specified is relative to -# the HTML output folder. -# This tag requires that the tag GENERATE_QHP is set to YES. - -QCH_FILE = - -# The QHP_NAMESPACE tag specifies the namespace to use when generating Qt Help -# Project output. For more information please see Qt Help Project / Namespace -# (see: http://qt-project.org/doc/qt-4.8/qthelpproject.html#namespace). -# The default value is: org.doxygen.Project. -# This tag requires that the tag GENERATE_QHP is set to YES. - -QHP_NAMESPACE = org.doxygen.Project - -# The QHP_VIRTUAL_FOLDER tag specifies the namespace to use when generating Qt -# Help Project output. For more information please see Qt Help Project / Virtual -# Folders (see: http://qt-project.org/doc/qt-4.8/qthelpproject.html#virtual- -# folders). -# The default value is: doc. -# This tag requires that the tag GENERATE_QHP is set to YES. - -QHP_VIRTUAL_FOLDER = doc - -# If the QHP_CUST_FILTER_NAME tag is set, it specifies the name of a custom -# filter to add. For more information please see Qt Help Project / Custom -# Filters (see: http://qt-project.org/doc/qt-4.8/qthelpproject.html#custom- -# filters). -# This tag requires that the tag GENERATE_QHP is set to YES. - -QHP_CUST_FILTER_NAME = - -# The QHP_CUST_FILTER_ATTRS tag specifies the list of the attributes of the -# custom filter to add. For more information please see Qt Help Project / Custom -# Filters (see: http://qt-project.org/doc/qt-4.8/qthelpproject.html#custom- -# filters). -# This tag requires that the tag GENERATE_QHP is set to YES. - -QHP_CUST_FILTER_ATTRS = - -# The QHP_SECT_FILTER_ATTRS tag specifies the list of the attributes this -# project's filter section matches. Qt Help Project / Filter Attributes (see: -# http://qt-project.org/doc/qt-4.8/qthelpproject.html#filter-attributes). -# This tag requires that the tag GENERATE_QHP is set to YES. - -QHP_SECT_FILTER_ATTRS = - -# The QHG_LOCATION tag can be used to specify the location of Qt's -# qhelpgenerator. If non-empty doxygen will try to run qhelpgenerator on the -# generated .qhp file. -# This tag requires that the tag GENERATE_QHP is set to YES. - -QHG_LOCATION = - -# If the GENERATE_ECLIPSEHELP tag is set to YES, additional index files will be -# generated, together with the HTML files, they form an Eclipse help plugin. To -# install this plugin and make it available under the help contents menu in -# Eclipse, the contents of the directory containing the HTML and XML files needs -# to be copied into the plugins directory of eclipse. The name of the directory -# within the plugins directory should be the same as the ECLIPSE_DOC_ID value. -# After copying Eclipse needs to be restarted before the help appears. -# The default value is: NO. -# This tag requires that the tag GENERATE_HTML is set to YES. - -GENERATE_ECLIPSEHELP = NO - -# A unique identifier for the Eclipse help plugin. When installing the plugin -# the directory name containing the HTML and XML files should also have this -# name. Each documentation set should have its own identifier. -# The default value is: org.doxygen.Project. -# This tag requires that the tag GENERATE_ECLIPSEHELP is set to YES. - -ECLIPSE_DOC_ID = org.doxygen.Project - -# If you want full control over the layout of the generated HTML pages it might -# be necessary to disable the index and replace it with your own. The -# DISABLE_INDEX tag can be used to turn on/off the condensed index (tabs) at top -# of each HTML page. A value of NO enables the index and the value YES disables -# it. Since the tabs in the index contain the same information as the navigation -# tree, you can set this option to YES if you also set GENERATE_TREEVIEW to YES. -# The default value is: NO. -# This tag requires that the tag GENERATE_HTML is set to YES. - -DISABLE_INDEX = NO - -# The GENERATE_TREEVIEW tag is used to specify whether a tree-like index -# structure should be generated to display hierarchical information. If the tag -# value is set to YES, a side panel will be generated containing a tree-like -# index structure (just like the one that is generated for HTML Help). For this -# to work a browser that supports JavaScript, DHTML, CSS and frames is required -# (i.e. any modern browser). Windows users are probably better off using the -# HTML help feature. Via custom style sheets (see HTML_EXTRA_STYLESHEET) one can -# further fine-tune the look of the index. As an example, the default style -# sheet generated by doxygen has an example that shows how to put an image at -# the root of the tree instead of the PROJECT_NAME. Since the tree basically has -# the same information as the tab index, you could consider setting -# DISABLE_INDEX to YES when enabling this option. -# The default value is: NO. -# This tag requires that the tag GENERATE_HTML is set to YES. - -GENERATE_TREEVIEW = NO - -# The ENUM_VALUES_PER_LINE tag can be used to set the number of enum values that -# doxygen will group on one line in the generated HTML documentation. -# -# Note that a value of 0 will completely suppress the enum values from appearing -# in the overview section. -# Minimum value: 0, maximum value: 20, default value: 4. -# This tag requires that the tag GENERATE_HTML is set to YES. - -ENUM_VALUES_PER_LINE = 4 - -# If the treeview is enabled (see GENERATE_TREEVIEW) then this tag can be used -# to set the initial width (in pixels) of the frame in which the tree is shown. -# Minimum value: 0, maximum value: 1500, default value: 250. -# This tag requires that the tag GENERATE_HTML is set to YES. - -TREEVIEW_WIDTH = 250 - -# If the EXT_LINKS_IN_WINDOW option is set to YES, doxygen will open links to -# external symbols imported via tag files in a separate window. -# The default value is: NO. -# This tag requires that the tag GENERATE_HTML is set to YES. - -EXT_LINKS_IN_WINDOW = NO - -# Use this tag to change the font size of LaTeX formulas included as images in -# the HTML documentation. When you change the font size after a successful -# doxygen run you need to manually remove any form_*.png images from the HTML -# output directory to force them to be regenerated. -# Minimum value: 8, maximum value: 50, default value: 10. -# This tag requires that the tag GENERATE_HTML is set to YES. - -FORMULA_FONTSIZE = 10 - -# Use the FORMULA_TRANPARENT tag to determine whether or not the images -# generated for formulas are transparent PNGs. Transparent PNGs are not -# supported properly for IE 6.0, but are supported on all modern browsers. -# -# Note that when changing this option you need to delete any form_*.png files in -# the HTML output directory before the changes have effect. -# The default value is: YES. -# This tag requires that the tag GENERATE_HTML is set to YES. - -FORMULA_TRANSPARENT = YES - -# Enable the USE_MATHJAX option to render LaTeX formulas using MathJax (see -# http://www.mathjax.org) which uses client side Javascript for the rendering -# instead of using pre-rendered bitmaps. Use this if you do not have LaTeX -# installed or if you want to formulas look prettier in the HTML output. When -# enabled you may also need to install MathJax separately and configure the path -# to it using the MATHJAX_RELPATH option. -# The default value is: NO. -# This tag requires that the tag GENERATE_HTML is set to YES. - -USE_MATHJAX = NO - -# When MathJax is enabled you can set the default output format to be used for -# the MathJax output. See the MathJax site (see: -# http://docs.mathjax.org/en/latest/output.html) for more details. -# Possible values are: HTML-CSS (which is slower, but has the best -# compatibility), NativeMML (i.e. MathML) and SVG. -# The default value is: HTML-CSS. -# This tag requires that the tag USE_MATHJAX is set to YES. - -MATHJAX_FORMAT = HTML-CSS - -# When MathJax is enabled you need to specify the location relative to the HTML -# output directory using the MATHJAX_RELPATH option. The destination directory -# should contain the MathJax.js script. For instance, if the mathjax directory -# is located at the same level as the HTML output directory, then -# MATHJAX_RELPATH should be ../mathjax. The default value points to the MathJax -# Content Delivery Network so you can quickly see the result without installing -# MathJax. However, it is strongly recommended to install a local copy of -# MathJax from http://www.mathjax.org before deployment. -# The default value is: http://cdn.mathjax.org/mathjax/latest. -# This tag requires that the tag USE_MATHJAX is set to YES. - -MATHJAX_RELPATH = http://cdn.mathjax.org/mathjax/latest - -# The MATHJAX_EXTENSIONS tag can be used to specify one or more MathJax -# extension names that should be enabled during MathJax rendering. For example -# MATHJAX_EXTENSIONS = TeX/AMSmath TeX/AMSsymbols -# This tag requires that the tag USE_MATHJAX is set to YES. - -MATHJAX_EXTENSIONS = - -# The MATHJAX_CODEFILE tag can be used to specify a file with javascript pieces -# of code that will be used on startup of the MathJax code. See the MathJax site -# (see: http://docs.mathjax.org/en/latest/output.html) for more details. For an -# example see the documentation. -# This tag requires that the tag USE_MATHJAX is set to YES. - -MATHJAX_CODEFILE = - -# When the SEARCHENGINE tag is enabled doxygen will generate a search box for -# the HTML output. The underlying search engine uses javascript and DHTML and -# should work on any modern browser. Note that when using HTML help -# (GENERATE_HTMLHELP), Qt help (GENERATE_QHP), or docsets (GENERATE_DOCSET) -# there is already a search function so this one should typically be disabled. -# For large projects the javascript based search engine can be slow, then -# enabling SERVER_BASED_SEARCH may provide a better solution. It is possible to -# search using the keyboard; to jump to the search box use + S -# (what the is depends on the OS and browser, but it is typically -# , /