Commit Graph

2369 Commits

Author SHA1 Message Date
Alexander Larsson
8983834f92 builder: Fix up .py[co] invalidation
Since we now only look at changed files we need to handle
the case where a .py file changed and the .pyc didn't, and
remove the now stale .pyc files.

Previously we looked at all files, so we detected this via
the stale .pyc file.
2017-02-24 14:16:12 +01:00
Thibault Saunier
75388609b3 scripts: Add a script to allow bisecting failures
The CLI is pretty similare to the `git bisect` one and works as follow:

  $ flatpak-bisect org.app.App start # Update application and get the history
  $ flatpak-bisect org.app.App bad # Sets current commit as first bad commit
  $ flatpak-bisect org.app.App checkout GoodHash # Checkout the first known good commit
  $ flatpak-bisect org.app.App good # Sets GoodHash as first good commit

... Here it starts bisection and checkouts a commit on the way, the user should launch the
app to check if the commit is good or bad and run:

  $ flatpak-bisect org.app.App good # if commit is good
  $ flatpak-bisect org.app.App bad # if commit is bad

flatpak-bisect will tell when the first bad commit is found.

Fixes https://github.com/flatpak/flatpak/issues/530
2017-02-23 17:41:06 +01:00
Richard Hughes
6125dabf58 Return the container from flatpak_get_system_installations()
This matches what other functions in libflatpak do, and also fixes a memory
leak in the error path.
2017-02-23 17:26:45 +01:00
Matthias Clasen
a95dbc8e9b Document --from-git and --from-git-branch 2017-02-23 17:15:35 +01:00
Alexander Larsson
3556272094 CI: require ostree package too
The libs were split out into its own package
2017-02-23 17:11:51 +01:00
Alexander Larsson
4fdac9e5d4 document portal: Disable spice_read as it seems broken
We're getting "fuse: copy from pipe: short read" from
libfuse and a broken filesystem with this. See
https://github.com/flatpak/flatpak/issues/570

I've tried to look into if this is fixable, but I can't
really find any way we could do things differently that
may help, so the fix is to disable spice_read.
2017-02-23 16:55:49 +01:00
Mario Sanchez Prada
d0b5b51076 Initialize g_autofree string to NULL, not to crash when early returning
Otherwise, anytime we fail in ostree_repo_write_metadata() will cause
an invalid free to happen, and flatpak to crash.
2017-02-23 14:27:22 +00:00
Alexander Larsson
6793d90b82 update: Don't update related refs from different remote
In case you happen to have a reference A with a related reference
B (say a runtime and a GL extension), and they come from different
remotes, then updating A should not cause B to update from the same
remote as A, but rather the current remote.
2017-02-23 14:42:43 +01:00
Alexander Larsson
51e14fe33b Fix repeated download of locates on update
When we update an app and add updates for all the related
operations, such as locates, and it is already installed,
make sure we inherit the current subpaths rather than
use the default ones.

Fixes https://github.com/flatpak/flatpak/issues/587
2017-02-23 14:26:41 +01:00
Alexander Larsson
cc03a2b94b Don't remove origin remotes if some other ref uses it
This fixes https://github.com/flatpak/flatpak/issues/586
2017-02-23 13:52:35 +01:00
Alexander Larsson
c07bc50779 Import NEWS entries from the 0.8 branch to master 2017-02-23 11:55:43 +01:00
Alexander Larsson
364499939d builder: Add --skip-if-unchanged
This lets you skip rebuilds unless the actual json changes.
This is useful for continuos builds that only run if the json
changes, not on any commit to any git source.
2017-02-22 15:12:59 +01:00
Alexander Larsson
90674fafa0 builder: Don't mirror submodules if told not to 2017-02-22 15:12:59 +01:00
Ray Strode
a82708cb10 run: propagate wildcard xauth entries to app bundle
At the moment, flatpak applications are only given FamilyLocal family
xauth cookies from the Xauthority file.  This is so, the sandboxed
application doesn't inadvertently get access to displays on other
computers.

But FamilyLocal isn't the only xauth family that's local.  FamilyWild
entries can be local as well.

Furthermore, FamilyWild entries are preferable to FamilyLocal entries
when found, because they don't break if the system hostname is changed.

This commit makes FamilyWild xauth entries get propagated in the same
way as their FamilyLocal counterparts.
2017-02-22 14:47:14 +01:00
Alexander Larsson
61db31bb00 builder: Add ensure-writable to module 2017-02-22 09:48:00 +01:00
Alexander Larsson
2c71298d53 builder: Its break_hardlink, not unbreak_hardlink 2017-02-22 09:43:57 +01:00
Alexander Larsson
67dc4a153f builder: Add builder_cache_checksum_compat_strv
This lets us add strv:s without invalidating old caches
if the empty strv means same as before
2017-02-22 09:42:13 +01:00
Alexander Larsson
359d4fa96b builder: Add builder_cache_get_files() to get all files in cache 2017-02-22 09:41:40 +01:00
Alexander Larsson
082efe8951 Properly quote all commandlines we print
Fixes https://github.com/flatpak/flatpak/issues/236
2017-02-21 18:56:11 +01:00
Alexander Larsson
d27799518d Fix no-systemd-user warning (it doesn't affect sandboxing anymore) 2017-02-21 15:41:43 +01:00
Alexander Larsson
608374e57a builder: Checksum cleanup-platform for platform 2017-02-21 15:09:50 +01:00
Alexander Larsson
1ca3c86552 builder: Add support for --from-git=URL
This allows you to build a json manifest that is stored in a git repo.

Closes: https://github.com/flatpak/flatpak/issues/511
2017-02-21 11:58:40 +01:00
Alexander Larsson
b136e7fdc6 builder: extract git mirroring code to builder-git.c
This way we can resuse it outside the git source
2017-02-21 11:58:35 +01:00
Alexander Larsson
e71e4221a9 builder: Separate run-dir and base-dir
We always put the state in the .flatpak-builder subdirectory of where
you ran flatpak-builder, independent on the location of the json
manifest. This makes sense in general, but is also extra important
when we want to check out the json from git inside the .flatpak-buidler
directory.
2017-02-21 11:50:58 +01:00
Alexander Larsson
1cdd9e796d builder: Break out build-dir allocation code 2017-02-21 11:50:41 +01:00
Alexander Larsson
50c9812a56 utils: Remove unused/broken autorm_rf 2017-02-21 11:50:41 +01:00
Mathieu Bridon
b996a636ff builder: Add a "simple" buildsystem
This just runs the specified "build-commands" one after the other,
ignoring makefiles, configure scripts, and all the rest.

Relates to #134
2017-02-20 16:08:35 +01:00
Alexander Larsson
c4da82341b builder: Add --build-shell=MODULE support
This lets you prepare a build directory for a given module
and start a build shell inside it. Very useful for debugging.
2017-02-20 15:02:09 +01:00
Alexander Larsson
75a16555e7 builder: Fix spelling 2017-02-17 14:10:12 +01:00
Alexander Larsson
be9e8c6eeb builder: Fix unused variable warning 2017-02-16 15:59:01 +01:00
Alexander Larsson
bcbd92f774 builder: Bump checksum version to invalidate all caches
Since we changed how the post process handles python
files in the init stage we need to rebuild everything.
2017-02-16 15:52:25 +01:00
Alexander Larsson
a5e1b2200a builder: Move python timestamp checkin to post-process
This changes what files we look at to only those in this module,
which is generally right, but to handle the base-layer sdk
case we also have to run the python fixup in the initial layer.
2017-02-16 10:51:00 +01:00
Alexander Larsson
ec1589e9ca builder: Fix up get_outstanding changes for first cache stage 2017-02-16 10:27:23 +01:00
Alexander Larsson
55e5ab272b tests: Fix python test regression from rofiles-fuse
We need to delete the old file when copying over it.
2017-02-16 09:52:14 +01:00
Alexander Larsson
05676536a8 builder: Split out debuginfo handling to a post-process step
We want to call this on the initial stage too, so move
it out of BuilderModule.
2017-02-16 09:40:08 +01:00
Alexander Larsson
690908eade builder: Spawn rofiles-fuse with maximum nr of open fds allowed 2017-02-15 20:18:13 +01:00
Alexander Larsson
5a3c7feb15 builder: Unbreak hardlinks when eu-stripping
Otherwise this breaks with rofiles-fuse if the build produces
hardlinked installed files. For instance, as done by mesa.
2017-02-15 19:22:14 +01:00
Alexander Larsson
ee83364a8e builder: Work around g_file_replace truncating hardlinked dests
If the target is the rofiles-fuse and a file is hardlinked, then
copy will fall back to truncation, which will fail with READONLY.
We work around this sometimes by deleting the destination first
and sometimes by using g_file_set_contents instead.
2017-02-15 18:04:00 +01:00
Alexander Larsson
1e2e718fd4 builder: Reuse devino cache for builder_cache_get_outstanding_changes
This makes finding debuginfo files to strip much faster.
2017-02-15 17:36:26 +01:00
Alexander Larsson
18b8c67207 builder: Use rofiles-fuse for the cache
Instead of building directly into the app directory we build into a
rofiles-fuse mount of it, which allows us to safely check out the
cache into the app directory using hardlinks (because rofiles-fuse
will not let you modify hardlinked files).

Additionally, every time we commit to the cache we check out all
the new and modified files into the appdir so that we get hardlinks
to the repo for the new files too.

The advantage of having hardlinks to the repo is that we can commit
much more efficient since we don't have to do a full checksum of
the hardlinked files.

There are some issues here:
 eu-strip fails due to doing in-place editin
 rofiles-fuse is using lots of CPU, unclear if this is faster, needs
 measurements
 needs testing of how well the fallback works (ie. if fuse is not
 working).

rofuse: use kernel caches

rofiles: check out after commit

Use devino cache

Only check out new files from cache after commit
2017-02-15 16:09:48 +01:00
Alexander Larsson
8c131be4f6 utils: Add missing Ostree autoptr types 2017-02-15 16:09:48 +01:00
Matthew Leeds
9067c8322d doc: Fix a typo 2017-02-14 10:22:39 +01:00
Roman Kharin
721ddb43d1 Update russian translation 2017-02-14 10:22:16 +01:00
Alexander Larsson
3287fdbbbb builder: Add config-opts to the docs for build-options
Reported in https://github.com/flatpak/flatpak/issues/560
2017-02-14 10:11:49 +01:00
Alexander Larsson
84246b6ec5 document-portal: Fix ownership of basic directories
We now report these as the user uid+gid instead of root.
2017-02-14 09:07:15 +01:00
Mario Sanchez Prada
86bf88d89f Allow cancelling the downloading process for extra data
So far, the installation of external apps can only be cancelled
before flatpak starts downloading the extra data, as there's no
cancellable being passed to g_input_stream_read_async().

This fixes that problem, making it possible to cancel installs
from GNOME Software regardless of the installation stage.
2017-02-14 09:01:16 +01:00
Mario Sanchez Prada
2e1740297c Report progress for extra data downloads, as we do for OSTree pulls
Otherwise, clients such as GNOME Software won't be able to report
any progress once the flatpak application has been downloaded and
we enter the stage to download the extra data.
2017-02-14 09:00:57 +01:00
Mario Sanchez Prada
ca952b0f21 Use the thread's default context to download extra data, not a new one
Otherwise, all the progress reporting for the extra data being downloaded
won't work, as the main context used by OstreeAsyncProgress will not be
the same than the one from the nested main loop used to download this.
2017-02-14 09:00:57 +01:00
Mario Sanchez Prada
df680dc0a6 Fix clang build: remove unused variables 2017-02-14 09:00:57 +01:00
Alexander Larsson
521e7e6a37 Don't user user namespaces when bubblewrap is setuid
It turns out that it is impossible for to get ptrace capabilities
for child user namespaces in the current kernel if the user
namespace is created as root, which is what happens when bwrap
is setuid root (see https://github.com/flatpak/flatpak/issues/557
for details).

This is very problematic, as ptrace rights controls access to
/proc/$pid/root which is what we base the detection of peer
app id and rights on for portals.

For now, we disable user namespaces (except for the case of
unprivileged user namespaces, where it is necessary and works).
2017-02-13 15:03:00 +01:00