Commit Graph

237 Commits

Author SHA1 Message Date
Patrick Griffis
a410e86fff builder: Error when failing to include modules
Currently it can silently fail if an included file is invalid
or missing. This makes that fatal.
2016-12-22 14:39:41 +01:00
Alexander Larsson
8737153ded builder: Fix --verbose logging
We need to pass the right log domain in.
2016-12-19 14:55:22 +01:00
Alexander Larsson
364def2135 builder: Add some spew when applying patches
This makes it easier to figure out what patch is broken
2016-12-12 17:20:36 +01:00
Alexander Larsson
e514dc2e14 Add flatpak_create_soup_session 2016-12-08 16:41:20 +01:00
Patrick Griffis
21b65947e8 builder: Add buildsystem option and meson support
Closes #400
2016-11-30 16:59:20 +01:00
Alexander Larsson
632f714509 builder: Add "options" property to patch sources
This lets you specify custom arguments to the patch command.

Fixes https://github.com/flatpak/flatpak/issues/417
2016-11-30 10:17:15 +01:00
Alexander Larsson
8ce04c6788 builder: Add support for --jobs=N to limit parallel builds
As requested in: https://github.com/flatpak/flatpak/issues/419
2016-11-30 09:57:55 +01:00
Alexander Larsson
462178bcd3 builder-options: Fix leaks 2016-11-28 17:23:38 +01:00
Alexander Larsson
100038513d builder: Fix coverity warning
If type is NULL we should never look at it.
2016-11-21 10:28:13 +01:00
Matthew Leeds
7b8becd1b2 builder: Don't download modules that won't be built
Currently flatpak-builder downloads the sources for all modules, even if
the --stop-at option prevents them from being built. This commit changes
the behavior so that we don't bother to download the ignored modules.
2016-11-15 15:01:05 +01:00
Alexander Larsson
7c0ef5dccb builder-source: Check for type == NULL before dereferencing 2016-11-11 10:48:37 +01:00
Matthew Leeds
8b50648ec0 builder: Check if .gitmodules exists before showing it. Fixes #366
This commit checks if <revision>:.gitmodules exists in the repo before
calling git-show on it, so an error message doesn't show up in the
program's output.
2016-11-01 09:30:31 +01:00
Alexander Larsson
6e31932221 OCI: Generate an oci image dir structure instead of a runtime 2016-10-31 20:11:14 +01:00
Alexander Larsson
1b09719298 builder: Handle app ids with dashes in them
An app id like org.domain.App-name is now allowed (dashes in the
last element), but blindly appending e.g. ".Locale" to these
for the locale extension will cause an invalid id.

In order to handle this we replace "-" with "_" in this case, which
should be fine as both are under the org.domain control which can
make sure there are no conflicts, and there is no requirement for
the id prefixes to exactly match.

This fixes https://github.com/flatpak/flatpak/issues/370
2016-10-26 21:40:49 +02:00
Alexander Larsson
fad8af2f36 builder: Break out name generation for Debug and Locale extensions
These helpers lets us tweak these names
2016-10-26 20:57:57 +02:00
Alexander Larsson
0f05143227 builder: Handle the case where the source directory is in a symlink
It can happen, like in https://github.com/flatpak/flatpak/issues/353
that there are references in the source to the absolute source path.
We need to ensure that is visible in the build. We mostly do this,
but for technical reasons it didn't get the same pathname when
the source was inside a symlinked directory. We fix this with an
extra bind-mount to the symlinked directory name too.

This fixes https://github.com/flatpak/flatpak/issues/353
2016-10-18 11:06:43 +02:00
Alexander Larsson
4cb3f7fd79 builder: Support application inheritance
When initializing a build you can initialize the application directory
with the files from another directory, and optionally a set of its
extensions. This allows one to create a "base" application that
can be derived from in multiple forms. A typical example is an application
framework like Electron. The base electron app has all the dependencies
needed to write Electron app, and then your manifest can add the specific
files needed for your application.

An example snipped could look something like:

    "base": "org.base.App",
    "base-version": "stable",
    "base-extensions": ["org.base.app.Locale", "org.base.app.Debug" ],
2016-10-17 11:40:57 +02:00
Alexander Larsson
a9c924928c builder: Add runtime-commit to resolved manifest
Apparently this must have been forgotten when sdk-commit was added.
2016-10-17 10:19:36 +02:00
Matt Watson
b9521e719b builder: add allow missing runtimes options
The --allow-missing-runtimes options will allow flatpak builder to
not abort immediately if the sdk or runtime for the app being built
are missing.

This option will be useless when building anything in the modules
section of the app manifest. The calls to flatpak-build will fail
because of the missing sdk.

However, it may be useful when an application does not require
building anything inside the sandbox, and the application files
will be installed via other means.
2016-10-17 09:49:34 +02:00
Matt Watson
e968b87f6a builder: add a build finish option
With the option you can resume an application build where it left
off when calling flatpak-buidler --build-only
2016-10-17 09:49:34 +02:00
Alexander Larsson
c0e0fe31ef utils: Remove unused variable 2016-10-06 12:31:54 +02:00
Alexander Larsson
ea17c27b37 builder: Add missing debug sprew when running an app 2016-10-06 12:31:54 +02:00
Niv Sardi
7f036570fd flatpak-builder: don't strip zip files
It is allowed to make a self-executable zip, it will then be an ELF file
on Linux. The problem is that the current stripping code somehow breaks
that file (i.e. ./file.zip doesn't work anymore after stripping).

Signed-off-by: Niv Sardi <xaiki@endlessm.com>
2016-10-06 12:05:30 +02:00
Alexander Larsson
91c9d474dd flatpak-builder: Add support for --show-deps 2016-09-20 15:58:43 +02:00
Tobias Mueller
72de257ec0 builder: check for name being defined in expand_modules
If this is not checked, flatpak-builder might crash like this:

Starting program: /var/tmp/flatpak/bin/flatpak-builder -v --force-clean
-v --ccache --repo=/tmp/fb.repo --gpg-sign=tobiasmue@gnome.org
/tmp/fpbuilder org.gnome.Sdk.ASan.json
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-
gnu/libthread_db.so.1".
[New Thread 0x7ffff139c700 (LWP 20407)]

Thread 1 "flatpak-builder" received signal SIGSEGV, Segmentation fault.
0x00007ffff697c770 in g_str_hash () from /lib/x86_64-linux-
gnu/libglib-2.0.so.0
(gdb) bt
   from /lib/x86_64-linux-gnu/libglib-2.0.so.0
   from /lib/x86_64-linux-gnu/libglib-2.0.so.0
    expanded=expanded@entry=0x7fffffffdae0, names=names@entry=0x690a40,
    error=error@entry=0x7fffffffdc08) at builder/builder-manifest.c:175
    expanded=expanded@entry=0x696158, names=names@entry=0x690a40,
    error=error@entry=0x7fffffffdc08) at builder/builder-manifest.c:169
(self=self@entry=0x696050,
    context=context@entry=0x697970, error=error@entry=0x7fffffffdc08)
    at builder/builder-manifest.c:933
    at builder/builder-main.c:349
(gdb) up
   from /lib/x86_64-linux-gnu/libglib-2.0.so.0
(gdb)
    expanded=expanded@entry=0x7fffffffdae0, names=names@entry=0x690a40,
    error=error@entry=0x7fffffffdc08) at builder/builder-manifest.c:175
175	      if (g_hash_table_lookup (names, name) != NULL)
(gdb) l
170	        return FALSE;
171
172	      *expanded = g_list_concat (*expanded, submodules);
173
174	      name = builder_module_get_name (m);
175	      if (g_hash_table_lookup (names, name) != NULL)
176	        {
177	          g_set_error (error, G_IO_ERROR, G_IO_ERROR_FAILED,
178	                       "Duplicate modules named '%s'", name);
179	          return FALSE;
(gdb) p names
$1 = (GHashTable *) 0x690a40
(gdb) p name
$2 = 0x0
(gdb) p m
$3 = (BuilderModule *) 0x6978d0
(gdb) p *m
$4 = {parent = {g_type_instance = {g_class = 0x6959d0}, ref_count = 1,
    qdata = 0x0}, name = 0x0, subdir = 0x0, post_install = 0x0,
config_opts = 0x0, make_args = 0x0, make_install_args = 0x0, disabled
= 0,
rm_configure = 0, no_autogen = 0, no_parallel_make = 0,
no_python_timestamp_fix = 0, cmake = 0, builddir = 0, build_options =
0x0,
changes = 0x0, cleanup = 0x0, cleanup_platform = 0x0, sources = 0x0,
modules = 0x0}
(gdb)

This happens when a definition does not include a name, e.g.

            "modules": [
                {
                    "type": "archive",
                    "url": ""foo",
                    "sha256": "123"
                }
2016-09-14 15:52:14 +02:00
Mathieu Bridon
d165475e17 builder: Don't fail when submodule path isn't a gitlink
This is what git itself does, so we just should do the same.

cda1bbd474/builtin/submodule--helper.c (L247-L248)
2016-09-13 00:24:42 +02:00
Mathieu Bridon
5430d4d40e builder: Properly check for gitlink trees
Instead of expecting the second word to be "commit", we should do like
git does and check for the file mode, which is 160000 for gitlink trees.

cda1bbd474/cache.h (L96)
2016-09-13 00:24:42 +02:00
Alexander Larsson
fba645a999 Remove unused local variables reported by clang 2016-09-12 11:11:35 +02:00
Alexander Larsson
5ef43f6ee3 builder: Bump manifest version to trigger rebuild with new mtimes
To make sure we're not using old caches with the previous mtime
timestamps encoded we bump the manifest version to force a rebuild.
2016-09-09 09:44:54 +02:00
Alexander Larsson
6689c5c7f1 Switch back to mtime==0 for ostree checkouts
OSTree upstream changed back from mtime 1
2016-09-09 09:22:56 +02:00
Alexander Larsson
41443313ca session-helper: Move HostCommand to its own interface 2016-09-06 15:30:38 +02:00
Alexander Larsson
e792686892 builder: Propagate build-args to shell sources
This way you can e.g. do network i/o in them.
Thats not a recommended way to do it though, as you work around
a lot of the feature in flatpak-builder like the caching and
verification.
2016-09-05 22:01:56 +02:00
Alexander Larsson
c66e394266 builder: Use HostCommand calls if in sandbox 2016-09-05 21:41:14 +02:00
Alexander Larsson
ddf05ef063 builder: Use flatpak_spawnv helper in a few more places
This is in preparation for it calling the new host-command.
2016-09-05 21:41:14 +02:00
Alexander Larsson
509bb33a2b Merge pull request #237 from mwleeds/add-trace-msg
utils: Print a debug message when launching subprocesses
2016-08-30 09:22:02 +02:00
Alexander Larsson
07fa8138a2 Merge pull request #194 from smcv/shell-injection
unrpm: prevent shell injection
2016-08-26 11:19:34 +02:00
Alexander Larsson
90d4f8d1b4 Don't fail if appstream-compose fails, instead warn.
This means we stop breaking existing builds e.g. when an older
appstream-compose in the runtime can't handle a new appdata field.
2016-08-25 11:34:10 +02:00
Mathieu Bridon
63cddfa80e builder: Handle submodules using with relative URLs
Some projects declare their Git submodules with URLs relative to their
own.

For example, gedit is hosted at git://git.gnome.org/gedit, and has its
libgd submodule declared with the ../libgd URL. We should treat the
latter as git://git.gnome.org/libgd.

This was in fact properly done in the past, but I broke it in commit
1171e1d202.

This fixes it again.
2016-08-24 21:06:00 +02:00
Alexander Larsson
b0a7f8052b builder: Add --sandbox support
This disables build-args support, which is nice when building things
on e.g. a shared build-machine, where we don't want the build to
be able to break out of the sandbox (by specifying e.g. --share=network).
2016-08-22 17:23:32 +02:00
Alexander Larsson
b03b21f416 Merge branch 'builder-git-submodules' of git://github.com/bochecha/flatpak into bochecha-builder-git-submodules 2016-08-22 16:08:39 +02:00
Alexander Larsson
6578d37984 Drop libgsystem dependency 2016-08-22 16:00:33 +02:00
Alexander Larsson
65f1cf993d Create and use flatpak_file_get_path_cached
This is simpler that the one in libgs, as well as lockless. Also, it
removes one more use of libgs.
2016-08-22 10:22:42 +02:00
Alexander Larsson
9c907c992d Replace gs_shutil_rm_rf with small helper wrapper over libglnx 2016-08-22 10:06:08 +02:00
Alexander Larsson
e0bd22bfe8 Add flatpak_mkdir_p helper and use it
This replaces all current callers of gs_file_ensure_directory with
equivalent code.

Actually, two instances were calling gs_file_ensure_directory with
FALSE, i.e. error out on EEXIST, but those cases seem fine with the
do-nothing-if-exists semantics.
2016-08-22 09:29:24 +02:00
Mathieu Bridon
1171e1d202 builder: Redo git submodules handling
flatpak-builder was mixing the path and the name of a submodule, which
would lead to download failures when they were different.

Fixes #216
2016-08-20 12:44:27 +02:00
Alexander Larsson
fb3bfae85b builder: Add --stop-at=module
Stop building at a specified module from the json file.
This is useful during development. For instance, you can do --stop-at=main-app
to build all the dependencies into the appdir, then you can do a build of an
already checked out git repo (possibly with changes).
2016-08-19 11:25:23 +02:00
Alexander Larsson
f622b014a7 builder: Handle symbolic icons in rename-icons 2016-08-19 11:05:18 +02:00
Alexander Larsson
0d115fb2e7 builder: Save a copy of the base manifest 2016-08-17 14:24:32 +02:00
Alexander Larsson
7431fff588 Fix up pyo stale file handling
We were not doing what the comments said we should, and we also
want to change what happens in the case of an old py file from
a previous layer (such as a base layer in a runtime) has a non-rewritten
.pyc file. In that case we want to assume its correct and fix it up.
2016-08-11 07:54:48 +02:00
Alexander Larsson
348b7f9ab4 builder: Add some printouts if we remove stale .pyc files
This makes it easier to figure out why it disappeared
2016-08-11 07:54:25 +02:00