Commit Graph

108 Commits

Author SHA1 Message Date
Mathieu Bridon
8b9df0ddcf Automatically use a separate builddir with Meson
Meson does not support builddir == srcdir, so there is no reason to
require developers to set "builddir" to true in their manifests, when we
can just do the right thing.
2017-05-24 15:18:26 +02:00
Alexander Larsson
461084d1a9 builder: Add a install-rule to allow customized install
Fixes: https://github.com/flatpak/flatpak/issues/791
2017-05-15 09:38:22 +02:00
Alexander Larsson
4774b3c80f builder: Take "buildsystem" into consideration for cache freshness 2017-05-15 09:38:22 +02:00
Alexander Larsson
fbde709739 builder: Convert bundle sources to cached stage
Instead of mixing the source bundling with the build we make
it a separate step at the end, with cache support just like the
other stages.

Being at the end means we can reuse the cached stages from the
build if we enable bundle-sources after an existing build.

Also, this changes how the json and local files/patches are stored.
Now they are in a subdirectory called "manifest" in the sources
directory, with proper handling of relative pathnames for included
modules, etc. This also means we don't look for these file in the
extra-sources directory, but rather next to the json like we
do normally.
2017-04-24 13:52:38 +02:00
Simon Schampijer
bb11ef3c37 Merge remote-tracking branch 'upstream/master' into bundle-sources 2017-04-21 00:13:01 +02:00
Alexander Larsson
4c2c6af4ac builder: Take build-commands into consideration for rebuild 2017-04-20 16:10:04 +02:00
Alexander Larsson
eebcefa3b2 builder: Print warnings for unknown properties 2017-04-19 20:22:11 +02:00
Alexander Larsson
63978c299a builder: Load source files from the directory of the module
This means if you include a json snippet in a subdirectory, then
that module will load e.g. patches from the same subdirectory.
2017-04-19 18:54:33 +02:00
Alexander Larsson
d60bfcdf95 builder: handle module-relative paths for json includes
I.e. each include is relative to the location of the "parent"
json directory.
2017-04-19 17:46:28 +02:00
Chocobo1
32c022df5b builder-module: add "bootstrap.sh" to autogen_names
Projects like libtorrent-rasterbar, qBittorrent are using this name.
2017-04-18 12:35:16 +02:00
Simon Schampijer
5d52f93a25 Bundle sources: bundling has to happen before the extracting
In the case of the source-file we have code that handle
the file inline. When we extract the source we do set
the url with that inline data. In get_download_location
that is also used by the bundling code we do then
check against this url and fail.
2017-04-13 13:57:56 +02:00
Simon Schampijer
e473ca3b46 Bundle sources: add flag --bundle-sources to control the bundling
By default the bundling feature is turned off.
2017-04-13 13:32:33 +02:00
Simon Schampijer
894311e243 flatpak-builder: bundle module sources as runtime
This adds a step to the build process to bundle
the module sources, used for building the flatpak,
as a runtime extension.

The sources can then be installed like the
debug or translation runtime.

This also adds an option to flatpak-builder
for specifying sources directories. One can specify
source dirctories with the use-sources argument. This
will skip the download part of the processing
and will extract the sources from the given sources
directory directly for further processing.

Those source directories do need the same
structure as the ones that flatpak-builder
creates during processing in .flatpak-builder
and which is also used in the exported sources
runtime.
2017-04-13 13:32:33 +02:00
André Klitzing
567802749c Fix some issues with musl
* Use realpath instead of canonicalize_file_name
* Add missing include
2017-03-15 10:14:39 +01:00
Alexander Larsson
b6d06ba8ef Bump manifest cache version to rebuild everything
This is due to the append changes, etc
Also, we drop all _compat_ cache checksums
2017-03-14 12:02:35 +01:00
Alexander Larsson
44399bcf5f builder: Add only-arches and skip-arches for sources 2017-02-28 17:41:45 +01:00
Alexander Larsson
0379c01981 builder: Add build-arches and skip-arches to modules
This lets you avoid modules on some arches.
2017-02-28 12:07:53 +01:00
Alexander Larsson
c3322fd9e9 builder: Add no-make-install to modules
This is useful for modules that don't have a make install rule.
You can use the new build-commands which is run after make to
create your own custom installation phase.

Fixes https://github.com/flatpak/flatpak/issues/458
2017-02-28 11:30:59 +01:00
Alexander Larsson
0b448c5f68 builder: Handle chdir failures when spawning shell 2017-02-28 11:11:28 +01:00
Alexander Larsson
aaf923bd37 builder: Use separate rofiles-fuse mounts for each module
We're doing a post-commit checkout directly to the app dir, which
means the fuse filesystem cache may get out of sync with the backing
directory. So, to ensure this doesn't happen we mount a fresh rofiles
fs for each build.
2017-02-24 14:16:12 +01:00
Alexander Larsson
61db31bb00 builder: Add ensure-writable to module 2017-02-22 09:48:00 +01:00
Alexander Larsson
608374e57a builder: Checksum cleanup-platform for platform 2017-02-21 15:09:50 +01:00
Alexander Larsson
1cdd9e796d builder: Break out build-dir allocation code 2017-02-21 11:50:41 +01:00
Mathieu Bridon
b996a636ff builder: Add a "simple" buildsystem
This just runs the specified "build-commands" one after the other,
ignoring makefiles, configure scripts, and all the rest.

Relates to #134
2017-02-20 16:08:35 +01:00
Alexander Larsson
c4da82341b builder: Add --build-shell=MODULE support
This lets you prepare a build directory for a given module
and start a build shell inside it. Very useful for debugging.
2017-02-20 15:02:09 +01:00
Alexander Larsson
a5e1b2200a builder: Move python timestamp checkin to post-process
This changes what files we look at to only those in this module,
which is generally right, but to handle the base-layer sdk
case we also have to run the python fixup in the initial layer.
2017-02-16 10:51:00 +01:00
Alexander Larsson
05676536a8 builder: Split out debuginfo handling to a post-process step
We want to call this on the initial stage too, so move
it out of BuilderModule.
2017-02-16 09:40:08 +01:00
Alexander Larsson
5a3c7feb15 builder: Unbreak hardlinks when eu-stripping
Otherwise this breaks with rofiles-fuse if the build produces
hardlinked installed files. For instance, as done by mesa.
2017-02-15 19:22:14 +01:00
Alexander Larsson
ee83364a8e builder: Work around g_file_replace truncating hardlinked dests
If the target is the rofiles-fuse and a file is hardlinked, then
copy will fall back to truncation, which will fail with READONLY.
We work around this sometimes by deleting the destination first
and sometimes by using g_file_set_contents instead.
2017-02-15 18:04:00 +01:00
Alexander Larsson
1e2e718fd4 builder: Reuse devino cache for builder_cache_get_outstanding_changes
This makes finding debuginfo files to strip much faster.
2017-02-15 17:36:26 +01:00
Alexander Larsson
de2f222474 builder: Don't modify .pyc files in-place
If we need to edit the files, make a copy and edit that.
Otherwise we can't handle the file being hardlinked into
the cache.
2017-02-07 16:39:23 +01:00
Alexander Larsson
e38d980bca builder: Initial version of extension building support 2017-01-27 15:27:55 +01:00
Bartłomiej Piotrowski
751cae4639 builder: add cmake-ninja support
The commit adds another buildsystem, cmake-ninja, that tells
CMake to generate Ninja rules instead of Makefile.

Fixes #461.
2017-01-20 15:28:27 +01:00
Patrick Griffis
21b65947e8 builder: Add buildsystem option and meson support
Closes #400
2016-11-30 16:59:20 +01:00
Alexander Larsson
8ce04c6788 builder: Add support for --jobs=N to limit parallel builds
As requested in: https://github.com/flatpak/flatpak/issues/419
2016-11-30 09:57:55 +01:00
Alexander Larsson
0f05143227 builder: Handle the case where the source directory is in a symlink
It can happen, like in https://github.com/flatpak/flatpak/issues/353
that there are references in the source to the absolute source path.
We need to ensure that is visible in the build. We mostly do this,
but for technical reasons it didn't get the same pathname when
the source was inside a symlinked directory. We fix this with an
extra bind-mount to the symlinked directory name too.

This fixes https://github.com/flatpak/flatpak/issues/353
2016-10-18 11:06:43 +02:00
Alexander Larsson
91c9d474dd flatpak-builder: Add support for --show-deps 2016-09-20 15:58:43 +02:00
Alexander Larsson
fba645a999 Remove unused local variables reported by clang 2016-09-12 11:11:35 +02:00
Alexander Larsson
6689c5c7f1 Switch back to mtime==0 for ostree checkouts
OSTree upstream changed back from mtime 1
2016-09-09 09:22:56 +02:00
Alexander Larsson
e792686892 builder: Propagate build-args to shell sources
This way you can e.g. do network i/o in them.
Thats not a recommended way to do it though, as you work around
a lot of the feature in flatpak-builder like the caching and
verification.
2016-09-05 22:01:56 +02:00
Alexander Larsson
c66e394266 builder: Use HostCommand calls if in sandbox 2016-09-05 21:41:14 +02:00
Alexander Larsson
ddf05ef063 builder: Use flatpak_spawnv helper in a few more places
This is in preparation for it calling the new host-command.
2016-09-05 21:41:14 +02:00
Alexander Larsson
509bb33a2b Merge pull request #237 from mwleeds/add-trace-msg
utils: Print a debug message when launching subprocesses
2016-08-30 09:22:02 +02:00
Alexander Larsson
b0a7f8052b builder: Add --sandbox support
This disables build-args support, which is nice when building things
on e.g. a shared build-machine, where we don't want the build to
be able to break out of the sandbox (by specifying e.g. --share=network).
2016-08-22 17:23:32 +02:00
Alexander Larsson
6578d37984 Drop libgsystem dependency 2016-08-22 16:00:33 +02:00
Alexander Larsson
65f1cf993d Create and use flatpak_file_get_path_cached
This is simpler that the one in libgs, as well as lockless. Also, it
removes one more use of libgs.
2016-08-22 10:22:42 +02:00
Alexander Larsson
9c907c992d Replace gs_shutil_rm_rf with small helper wrapper over libglnx 2016-08-22 10:06:08 +02:00
Alexander Larsson
e0bd22bfe8 Add flatpak_mkdir_p helper and use it
This replaces all current callers of gs_file_ensure_directory with
equivalent code.

Actually, two instances were calling gs_file_ensure_directory with
FALSE, i.e. error out on EEXIST, but those cases seem fine with the
do-nothing-if-exists semantics.
2016-08-22 09:29:24 +02:00
Alexander Larsson
7431fff588 Fix up pyo stale file handling
We were not doing what the comments said we should, and we also
want to change what happens in the case of an old py file from
a previous layer (such as a base layer in a runtime) has a non-rewritten
.pyc file. In that case we want to assume its correct and fix it up.
2016-08-11 07:54:48 +02:00
Alexander Larsson
348b7f9ab4 builder: Add some printouts if we remove stale .pyc files
This makes it easier to figure out why it disappeared
2016-08-11 07:54:25 +02:00