Commit Graph

59 Commits

Author SHA1 Message Date
Philip Withnall
5b002edf06 builder: Add collection ID support to the flatpak builder
Pass a --collection-id argument through to `flatpak build-export`.

Also add a ‘collection-id’ property to manifest files, which can be used
to set the collection ID on an exported repo (when using --repo) without
having to provide a command line option.

Signed-off-by: Philip Withnall <withnall@endlessm.com>
2017-08-18 14:23:03 +02:00
Alexander Larsson
f9a06a0fcd builder: Add --install-deps-from=REMOTE
This allows you to automatically install/update dependencies required
by the manifest. The dependencies include:
 Runtime, Sdk, Base, Sdk Extensions and Platform Extensions

There is also a --install-deps-only switch to make the build
stop after the dependencies are installed.

Fixes https://github.com/flatpak/flatpak/issues/955
2017-08-18 11:04:13 +02:00
Alexander Larsson
9c06cbb360 Add support for --extra-sources-url=URL
This makes flatpak-builder look in the specified URL for mirrored
sources before downloading the original URL.
2017-07-29 17:43:10 +02:00
Matthias Clasen
9ef22bed1d Catch some more errors in option parsing
flatpak-builder was running into an assertion when --run was
used after a non-option. Catch this and exit with a clear error
message. Same for --show-deps
2017-07-09 23:06:00 -04:00
Matthias Clasen
ed08538f92 Merge pull request #900 from pwithnall/lan-and-usb-preparation
LAN/USB preparation refactoring
2017-07-08 20:54:28 -04:00
Philip Withnall
81301f0ef4 general: Remove trailing \n from GLib log messages
The GLib logging framework automatically appends a \n to messages, so it
doesn’t need to be added by callers.

Signed-off-by: Philip Withnall <withnall@endlessm.com>
2017-07-07 17:18:24 +01:00
Philip Withnall
548f60ebc0 general: Add missing static modifier to various local methods
The compiler warning flag which was supposed to warn about this was not
being included in the CFLAGS for these targets. That will be fixed in an
upcoming commit.

Signed-off-by: Philip Withnall <withnall@endlessm.com>
2017-07-06 14:41:21 +01:00
Philip Withnall
15bc368188 builder: Rearrange args to do_export() to make mandatory ones obvious
Previously, the mandatory location, directory and branch positional
arguments were passed as part of the varargs. This meant the varargs
were conflating flags and positional arguments, making it harder to add
new positional arguments.

Move the mandatory positional arguments to separate parameters for
do_export(). This introduces no functional changes, but makes later
commits easier.

Signed-off-by: Philip Withnall <withnall@endlessm.com>
2017-06-30 13:41:38 +02:00
Alexander Larsson
1820ebdc72 builder: Better handling of default-branch
Instead of constantly looking at the option we change the
manifest value if its unset. This means we can access the
default branch outside of builder-main.c, and that we get
it into the serialized manifest in the build.
2017-06-19 21:16:46 +02:00
Alexander Larsson
05f9c8730c builder: Add --export-only feature
This allows you to export a previous build, and still get features
like debug and locale extensions.

Fixes https://github.com/flatpak/flatpak/issues/824
2017-05-31 14:08:36 +02:00
Alexander Larsson
e41cb789fa builder: Add support for screenshot mirroring
Specify --mirror-screenshots-url=URL and then copy the resulting
appdata/screenshots directory to the URL, then these
will be used instead of the upstream screenshots.
2017-05-22 16:18:45 +02:00
Alexander Larsson
fbde709739 builder: Convert bundle sources to cached stage
Instead of mixing the source bundling with the build we make
it a separate step at the end, with cache support just like the
other stages.

Being at the end means we can reuse the cached stages from the
build if we enable bundle-sources after an existing build.

Also, this changes how the json and local files/patches are stored.
Now they are in a subdirectory called "manifest" in the sources
directory, with proper handling of relative pathnames for included
modules, etc. This also means we don't look for these file in the
extra-sources directory, but rather next to the json like we
do normally.
2017-04-24 13:52:38 +02:00
Alexander Larsson
3058d413f0 builder: Use flatpak_mkdir_p instead of query + mkdir 2017-04-21 16:38:05 +02:00
Simon Schampijer
bb11ef3c37 Merge remote-tracking branch 'upstream/master' into bundle-sources 2017-04-21 00:13:01 +02:00
Alexander Larsson
d60bfcdf95 builder: handle module-relative paths for json includes
I.e. each include is relative to the location of the "parent"
json directory.
2017-04-19 17:46:28 +02:00
Simon Schampijer
5d52f93a25 Bundle sources: bundling has to happen before the extracting
In the case of the source-file we have code that handle
the file inline. When we extract the source we do set
the url with that inline data. In get_download_location
that is also used by the bundling code we do then
check against this url and fail.
2017-04-13 13:57:56 +02:00
Simon Schampijer
60f96e31d6 Bundle sources: bundle the manifest 2017-04-13 13:32:33 +02:00
Simon Schampijer
603801eeb8 Bundle sources: rename option to --extra-sources=DIR
The specified directory is added to the list of sources
and when downloading the sources it is looked up first
before the sources are tried to be downloaded.
2017-04-13 13:32:33 +02:00
Simon Schampijer
4357ba6844 Bundle sources: allow use case to mix local and online sources
Fix checksum check for archives and files when sources are from
a local directory.
2017-04-13 13:32:33 +02:00
Simon Schampijer
16ca129253 Bundle sources: the path is always sources 2017-04-13 13:32:33 +02:00
Simon Schampijer
e473ca3b46 Bundle sources: add flag --bundle-sources to control the bundling
By default the bundling feature is turned off.
2017-04-13 13:32:33 +02:00
Simon Schampijer
894311e243 flatpak-builder: bundle module sources as runtime
This adds a step to the build process to bundle
the module sources, used for building the flatpak,
as a runtime extension.

The sources can then be installed like the
debug or translation runtime.

This also adds an option to flatpak-builder
for specifying sources directories. One can specify
source dirctories with the use-sources argument. This
will skip the download part of the processing
and will extract the sources from the given sources
directory directly for further processing.

Those source directories do need the same
structure as the ones that flatpak-builder
creates during processing in .flatpak-builder
and which is also used in the exported sources
runtime.
2017-04-13 13:32:33 +02:00
Alexander Larsson
3459ef8655 builder: Add disable-fsckobjects to git sources
This lets you work around clone problems for broken
git repositories.

Fixes https://github.com/flatpak/flatpak/issues/661
2017-04-04 14:31:48 +02:00
Matthew Leeds
543c0735f1 builder: Add a hint about --force-clean 2017-03-27 12:52:46 +02:00
Alexander Larsson
b00b8b1601 builder: Add --default-branch=BRANCH 2017-03-21 13:16:29 +01:00
Alexander Larsson
72e36b90a0 builder: Keep cache per arch
This just adds the arch name to the cache branch names, which
means a build on a different arch will not invalidate the other
arch:s build.

Fixes https://github.com/flatpak/flatpak/issues/628
2017-03-15 09:43:50 +01:00
Alexander Larsson
ea6747fcb4 builder: Fix --from-git
We need to specify the full branch when checking out
2017-03-07 14:20:44 +01:00
Alexander Larsson
60cadead5c builder: Properly handle relative module include paths from git
We load the sub-modules from the base directory (i.e. where
the manifest is stored).
2017-03-06 09:20:34 +01:00
Alexander Larsson
0379c01981 builder: Add build-arches and skip-arches to modules
This lets you avoid modules on some arches.
2017-02-28 12:07:53 +01:00
Alexander Larsson
364499939d builder: Add --skip-if-unchanged
This lets you skip rebuilds unless the actual json changes.
This is useful for continuos builds that only run if the json
changes, not on any commit to any git source.
2017-02-22 15:12:59 +01:00
Alexander Larsson
1ca3c86552 builder: Add support for --from-git=URL
This allows you to build a json manifest that is stored in a git repo.

Closes: https://github.com/flatpak/flatpak/issues/511
2017-02-21 11:58:40 +01:00
Alexander Larsson
e71e4221a9 builder: Separate run-dir and base-dir
We always put the state in the .flatpak-builder subdirectory of where
you ran flatpak-builder, independent on the location of the json
manifest. This makes sense in general, but is also extra important
when we want to check out the json from git inside the .flatpak-buidler
directory.
2017-02-21 11:50:58 +01:00
Alexander Larsson
c4da82341b builder: Add --build-shell=MODULE support
This lets you prepare a build directory for a given module
and start a build shell inside it. Very useful for debugging.
2017-02-20 15:02:09 +01:00
Alexander Larsson
a5e1b2200a builder: Move python timestamp checkin to post-process
This changes what files we look at to only those in this module,
which is generally right, but to handle the base-layer sdk
case we also have to run the python fixup in the initial layer.
2017-02-16 10:51:00 +01:00
Alexander Larsson
18b8c67207 builder: Use rofiles-fuse for the cache
Instead of building directly into the app directory we build into a
rofiles-fuse mount of it, which allows us to safely check out the
cache into the app directory using hardlinks (because rofiles-fuse
will not let you modify hardlinked files).

Additionally, every time we commit to the cache we check out all
the new and modified files into the appdir so that we get hardlinks
to the repo for the new files too.

The advantage of having hardlinks to the repo is that we can commit
much more efficient since we don't have to do a full checksum of
the hardlinked files.

There are some issues here:
 eu-strip fails due to doing in-place editin
 rofiles-fuse is using lots of CPU, unclear if this is faster, needs
 measurements
 needs testing of how well the fallback works (ie. if fuse is not
 working).

rofuse: use kernel caches

rofiles: check out after commit

Use devino cache

Only check out new files from cache after commit
2017-02-15 16:09:48 +01:00
Alexander Larsson
f2fb0eaf55 By default, don't rebuild apps if the SDK changes
Rebuilding all apps because a minor change in the runtime is way too wasteful
and generated unnecessary app updates, especially since runtimes are supposed
to be API stable.

We add an a --rebuild-on-sdk-change option which you use to disable this feature,
for instance if you're building against an unstable SDK.
2017-02-10 16:37:33 +01:00
Alexander Larsson
b585e4008e builder: Store the context in the BuilderCache
This allows the cache to respect global options
2017-02-07 16:39:23 +01:00
Alexander Larsson
e38d980bca builder: Initial version of extension building support 2017-01-27 15:27:55 +01:00
Ikey Doherty
605c7ee87b Fix compiler warnings with uninitialised cleanup pointers
This change fixes GCC compiler warnings where a cleanup function was
specified on an uninitialised pointer, which in the view of GCC could
result in an errornous free of uninitialised memory, if the functions
that initialise them do not return NULL.

Signed-off-by: Ikey Doherty <ikey@solus-project.com>
2017-01-19 10:05:13 +01:00
Alexander Larsson
8737153ded builder: Fix --verbose logging
We need to pass the right log domain in.
2016-12-19 14:55:22 +01:00
Alexander Larsson
8ce04c6788 builder: Add support for --jobs=N to limit parallel builds
As requested in: https://github.com/flatpak/flatpak/issues/419
2016-11-30 09:57:55 +01:00
Alexander Larsson
fad8af2f36 builder: Break out name generation for Debug and Locale extensions
These helpers lets us tweak these names
2016-10-26 20:57:57 +02:00
Matt Watson
b9521e719b builder: add allow missing runtimes options
The --allow-missing-runtimes options will allow flatpak builder to
not abort immediately if the sdk or runtime for the app being built
are missing.

This option will be useless when building anything in the modules
section of the app manifest. The calls to flatpak-build will fail
because of the missing sdk.

However, it may be useful when an application does not require
building anything inside the sandbox, and the application files
will be installed via other means.
2016-10-17 09:49:34 +02:00
Matt Watson
e968b87f6a builder: add a build finish option
With the option you can resume an application build where it left
off when calling flatpak-buidler --build-only
2016-10-17 09:49:34 +02:00
Alexander Larsson
91c9d474dd flatpak-builder: Add support for --show-deps 2016-09-20 15:58:43 +02:00
Alexander Larsson
b0a7f8052b builder: Add --sandbox support
This disables build-args support, which is nice when building things
on e.g. a shared build-machine, where we don't want the build to
be able to break out of the sandbox (by specifying e.g. --share=network).
2016-08-22 17:23:32 +02:00
Alexander Larsson
6578d37984 Drop libgsystem dependency 2016-08-22 16:00:33 +02:00
Alexander Larsson
9c907c992d Replace gs_shutil_rm_rf with small helper wrapper over libglnx 2016-08-22 10:06:08 +02:00
Alexander Larsson
fb3bfae85b builder: Add --stop-at=module
Stop building at a specified module from the json file.
This is useful during development. For instance, you can do --stop-at=main-app
to build all the dependencies into the appdir, then you can do a build of an
already checked out git repo (possibly with changes).
2016-08-19 11:25:23 +02:00
Matthias Clasen
75218c4a65 Fix license headers
We were referring to the nonexisting "version 2" of the
Lesser GPL. It should be "version 2.1".
2016-07-29 14:27:49 -04:00