Commit Graph

95 Commits

Author SHA1 Message Date
André Klitzing
567802749c Fix some issues with musl
* Use realpath instead of canonicalize_file_name
* Add missing include
2017-03-15 10:14:39 +01:00
Alexander Larsson
b6d06ba8ef Bump manifest cache version to rebuild everything
This is due to the append changes, etc
Also, we drop all _compat_ cache checksums
2017-03-14 12:02:35 +01:00
Alexander Larsson
44399bcf5f builder: Add only-arches and skip-arches for sources 2017-02-28 17:41:45 +01:00
Alexander Larsson
0379c01981 builder: Add build-arches and skip-arches to modules
This lets you avoid modules on some arches.
2017-02-28 12:07:53 +01:00
Alexander Larsson
c3322fd9e9 builder: Add no-make-install to modules
This is useful for modules that don't have a make install rule.
You can use the new build-commands which is run after make to
create your own custom installation phase.

Fixes https://github.com/flatpak/flatpak/issues/458
2017-02-28 11:30:59 +01:00
Alexander Larsson
0b448c5f68 builder: Handle chdir failures when spawning shell 2017-02-28 11:11:28 +01:00
Alexander Larsson
aaf923bd37 builder: Use separate rofiles-fuse mounts for each module
We're doing a post-commit checkout directly to the app dir, which
means the fuse filesystem cache may get out of sync with the backing
directory. So, to ensure this doesn't happen we mount a fresh rofiles
fs for each build.
2017-02-24 14:16:12 +01:00
Alexander Larsson
61db31bb00 builder: Add ensure-writable to module 2017-02-22 09:48:00 +01:00
Alexander Larsson
608374e57a builder: Checksum cleanup-platform for platform 2017-02-21 15:09:50 +01:00
Alexander Larsson
1cdd9e796d builder: Break out build-dir allocation code 2017-02-21 11:50:41 +01:00
Mathieu Bridon
b996a636ff builder: Add a "simple" buildsystem
This just runs the specified "build-commands" one after the other,
ignoring makefiles, configure scripts, and all the rest.

Relates to #134
2017-02-20 16:08:35 +01:00
Alexander Larsson
c4da82341b builder: Add --build-shell=MODULE support
This lets you prepare a build directory for a given module
and start a build shell inside it. Very useful for debugging.
2017-02-20 15:02:09 +01:00
Alexander Larsson
a5e1b2200a builder: Move python timestamp checkin to post-process
This changes what files we look at to only those in this module,
which is generally right, but to handle the base-layer sdk
case we also have to run the python fixup in the initial layer.
2017-02-16 10:51:00 +01:00
Alexander Larsson
05676536a8 builder: Split out debuginfo handling to a post-process step
We want to call this on the initial stage too, so move
it out of BuilderModule.
2017-02-16 09:40:08 +01:00
Alexander Larsson
5a3c7feb15 builder: Unbreak hardlinks when eu-stripping
Otherwise this breaks with rofiles-fuse if the build produces
hardlinked installed files. For instance, as done by mesa.
2017-02-15 19:22:14 +01:00
Alexander Larsson
ee83364a8e builder: Work around g_file_replace truncating hardlinked dests
If the target is the rofiles-fuse and a file is hardlinked, then
copy will fall back to truncation, which will fail with READONLY.
We work around this sometimes by deleting the destination first
and sometimes by using g_file_set_contents instead.
2017-02-15 18:04:00 +01:00
Alexander Larsson
1e2e718fd4 builder: Reuse devino cache for builder_cache_get_outstanding_changes
This makes finding debuginfo files to strip much faster.
2017-02-15 17:36:26 +01:00
Alexander Larsson
de2f222474 builder: Don't modify .pyc files in-place
If we need to edit the files, make a copy and edit that.
Otherwise we can't handle the file being hardlinked into
the cache.
2017-02-07 16:39:23 +01:00
Alexander Larsson
e38d980bca builder: Initial version of extension building support 2017-01-27 15:27:55 +01:00
Bartłomiej Piotrowski
751cae4639 builder: add cmake-ninja support
The commit adds another buildsystem, cmake-ninja, that tells
CMake to generate Ninja rules instead of Makefile.

Fixes #461.
2017-01-20 15:28:27 +01:00
Patrick Griffis
21b65947e8 builder: Add buildsystem option and meson support
Closes #400
2016-11-30 16:59:20 +01:00
Alexander Larsson
8ce04c6788 builder: Add support for --jobs=N to limit parallel builds
As requested in: https://github.com/flatpak/flatpak/issues/419
2016-11-30 09:57:55 +01:00
Alexander Larsson
0f05143227 builder: Handle the case where the source directory is in a symlink
It can happen, like in https://github.com/flatpak/flatpak/issues/353
that there are references in the source to the absolute source path.
We need to ensure that is visible in the build. We mostly do this,
but for technical reasons it didn't get the same pathname when
the source was inside a symlinked directory. We fix this with an
extra bind-mount to the symlinked directory name too.

This fixes https://github.com/flatpak/flatpak/issues/353
2016-10-18 11:06:43 +02:00
Alexander Larsson
91c9d474dd flatpak-builder: Add support for --show-deps 2016-09-20 15:58:43 +02:00
Alexander Larsson
fba645a999 Remove unused local variables reported by clang 2016-09-12 11:11:35 +02:00
Alexander Larsson
6689c5c7f1 Switch back to mtime==0 for ostree checkouts
OSTree upstream changed back from mtime 1
2016-09-09 09:22:56 +02:00
Alexander Larsson
e792686892 builder: Propagate build-args to shell sources
This way you can e.g. do network i/o in them.
Thats not a recommended way to do it though, as you work around
a lot of the feature in flatpak-builder like the caching and
verification.
2016-09-05 22:01:56 +02:00
Alexander Larsson
c66e394266 builder: Use HostCommand calls if in sandbox 2016-09-05 21:41:14 +02:00
Alexander Larsson
ddf05ef063 builder: Use flatpak_spawnv helper in a few more places
This is in preparation for it calling the new host-command.
2016-09-05 21:41:14 +02:00
Alexander Larsson
509bb33a2b Merge pull request #237 from mwleeds/add-trace-msg
utils: Print a debug message when launching subprocesses
2016-08-30 09:22:02 +02:00
Alexander Larsson
b0a7f8052b builder: Add --sandbox support
This disables build-args support, which is nice when building things
on e.g. a shared build-machine, where we don't want the build to
be able to break out of the sandbox (by specifying e.g. --share=network).
2016-08-22 17:23:32 +02:00
Alexander Larsson
6578d37984 Drop libgsystem dependency 2016-08-22 16:00:33 +02:00
Alexander Larsson
65f1cf993d Create and use flatpak_file_get_path_cached
This is simpler that the one in libgs, as well as lockless. Also, it
removes one more use of libgs.
2016-08-22 10:22:42 +02:00
Alexander Larsson
9c907c992d Replace gs_shutil_rm_rf with small helper wrapper over libglnx 2016-08-22 10:06:08 +02:00
Alexander Larsson
e0bd22bfe8 Add flatpak_mkdir_p helper and use it
This replaces all current callers of gs_file_ensure_directory with
equivalent code.

Actually, two instances were calling gs_file_ensure_directory with
FALSE, i.e. error out on EEXIST, but those cases seem fine with the
do-nothing-if-exists semantics.
2016-08-22 09:29:24 +02:00
Alexander Larsson
7431fff588 Fix up pyo stale file handling
We were not doing what the comments said we should, and we also
want to change what happens in the case of an old py file from
a previous layer (such as a base layer in a runtime) has a non-rewritten
.pyc file. In that case we want to assume its correct and fix it up.
2016-08-11 07:54:48 +02:00
Alexander Larsson
348b7f9ab4 builder: Add some printouts if we remove stale .pyc files
This makes it easier to figure out why it disappeared
2016-08-11 07:54:25 +02:00
Alexander Larsson
006d9a1927 Correctly handle .pyc mtimes for .py files changing multiple times
If a .py file changes multiple times we can end up in a situation
where there is an .py file with corresponding .pyc file that we
rewrote, so both are now mtime==1. Then a new version of the .py file
is added, but the corresponding .pyc file is not updated. This means
that the .pyc file is stale, and python would not normally use it.
However, we will later change the mtime on the .py file to 1, causing
the old .pyo file to look up-to-date even though its stale.

We fix this by detecting the case where the is a new mtime on a .py file
where the .pyc file doesn't match, and remove the stale .pyc file.
2016-08-10 16:24:11 +02:00
Matthew Leeds
392c597c5f utils: Print a debug message when launching subprocesses
This commit makes flatpak print a debug message (which only appears to
the user if the -v option is used) whenever a subprocess is launched.
This should make debugging easier, both for flatpak users and
developers.
2016-08-05 14:15:19 -04:00
Alexander Larsson
b5204c908d builder: Clear mtime to 1, not 0, to match what new ostree does 2016-07-01 14:25:22 +02:00
Matthias Clasen
99e4f83e33 Allow inclusions in the module list
When we see a string in the modules array, parse it as a json
file and use the resulting BuilderModule object.
2016-06-09 01:01:37 -04:00
Matthias Clasen
2757c63fea Add a modules property to BuilderModule
This will let us load modules recursively.
2016-06-08 23:56:38 -04:00
Matthias Clasen
914b841481 module: Be a bit more robust
When we call builder_module_cleanup_collect on a module that
has no changes, we shouldn't crash.
2016-06-08 23:55:57 -04:00
Alexander Larsson
bd66da0dad builder: Add support for patching mtime in python bytecode headers
This makes them work at runtime (as the mtime will then be 0) and
makes builds more repeatable.
2016-06-08 13:28:27 +02:00
Alexander Larsson
8362d9cafe Merge pull request #36 from matthiasclasen/module-errors
flatpak-builder: Print name of failed module on error
2016-05-27 09:02:44 +02:00
Matthias Clasen
5f928b30ac builder: Prefix module errors
Sprinkle g_prefix_error calls in builder-module.c, to make
sure that module-specific errors get a "module foo:" prefix.
This is not quite perfect, since callers tend to impose their
own prefix as well, but at least it clearly identifies the
module in which the error ocurred.
2016-05-27 00:25:28 -04:00
Matthias Clasen
6d1d9724c5 builder: Warn about bad module names
Putting spaces or slashes in module names is a recipe for
cryptic error from one of the many constructed flatpak
commandlines. Better warn early on, as soon as we see such
a name.
2016-05-27 00:05:55 -04:00
Alexander Larsson
28c1c65d8f Builder: Support disabled=true to not build a module
This is a simple way to "comment out" a module.
2016-05-09 17:24:42 +02:00
Alexander Larsson
8abbc0186e Rename everything but the on-disk location to flatpak 2016-05-09 11:11:55 +02:00
Alexander Larsson
c24528d369 Rename source files to flatpak 2016-05-09 09:00:20 +02:00