Commit Graph

779 Commits

Author SHA1 Message Date
Urvashi Mohnani
f5e4ffb5e4 Add init containers to generate and play kube
Kubernetes has a concept of init containers that run and exit before
the regular containers in a pod are started. We added init containers
to podman pods as well. This patch adds support for generating init
containers in the kube yaml when a pod we are converting had init
containers. When playing a kube yaml, it detects an init container
and creates such a container in podman accordingly.
Note, only init containers created with the init type set to "always"
will be generated as the "once" option deletes the init container after
it has run and exited. Play kube will always creates init containers
with the "always" init container type.

Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2021-09-10 09:37:46 -04:00
OpenShift Merge Robot
d477fe5cae Merge pull request #11447 from chenzhiwei/respect-config
fix play kube can't use infra_image in config file
2021-09-09 10:12:22 -04:00
OpenShift Merge Robot
d68e429859 Merge pull request #11476 from vrothberg/fix-11392
container inspect: improve error handling
2021-09-08 16:57:23 +02:00
Chen Zhiwei
a1cab358cc fix play kube can't use infra_image in config file
Signed-off-by: Chen Zhiwei <zhiweik@gmail.com>
2021-09-08 19:55:45 +08:00
OpenShift Merge Robot
558ba1b99e Merge pull request #11468 from Luap99/play-kube-slirp
fix play kube --network options
2021-09-08 13:55:21 +02:00
Valentin Rothberg
6aa666a27c container inspect: improve error handling
Improve the error handling of `container inspect` to properly handle
when the container has been removed _between_ the lookup and the
inspect.  That will yield the correct "no such object" error message in
`inspect`.

[NO TESTS NEEDED] since I do not know have a reliable and cheap
reproducer.  It's fixing a CI flake, so there's already an indicator.

Fixes: #11392
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-09-08 13:26:29 +02:00
Paul Holzinger
f18ccbcc0f fix play kube --network options
Commit 092902b455 introduced advanced network options for podman play
kube. However this never worked because it unconditionally set the
network mode to bridge after it parsed the network option.

Added a test to ensure the correct mode is set.

Truly fixes #10807

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-09-08 10:09:28 +02:00
OpenShift Merge Robot
536f23c0b7 Merge pull request #11431 from jmguzik/secrets-ls-filters
Add filtering functionality to http api secrets list
2021-09-07 19:24:20 +02:00
OpenShift Merge Robot
30d0cc3cca Merge pull request #11427 from flouthoc/kube-pod-logs
kube: Add support for `podman pod logs`.
2021-09-07 18:39:45 +02:00
Valentin Rothberg
d1573b95e3 generate systemd: handle --restart
Handle custom restart policies of containers when generating the unit
files; those should be set on the unit level and removed from ExecStart
flags.

Fixes: #11438
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-09-07 13:58:11 +02:00
Aditya Rajan
11fc0e5540 kube: Add support for podman pod logs
Following PR adds support for `kubectl` like `pod logs` to podman.
Usage `podman pod logs <podIDorName` gives a stream of logs for all
the containers within the pod with **containername** as a field.

Just like **`kubectl`** also supports `podman pod logs -c ctrIDorName podIDorName`
to limit the log stream to any of the specificied container which belongs to pod.

Signed-off-by: Aditya Rajan <arajan@redhat.com>
2021-09-05 16:24:49 +05:30
Jakub Guzik
d346e6e734 Add filtering functionality to http api secrets list
Filtering is missing in both compat API and libpod API, while docker
has filtering functinality. This commit enables filtering option using
name and id in both libpod and http API.

Signed-off-by: Jakub Guzik <jakubmguzik@gmail.com>
2021-09-03 10:29:31 +02:00
Aditya Rajan
cba114dd36 manifest: rm should not remove referenced images.
Following PR makes sure that `podman manifest rm <list>` only removes
the named manifest list and not referenced images.

Bumping and squashing c/common to v0.43.3-0.20210902095222-a7acc160fb25
in same commit in order to make sure build commit test passes.

Signed-off-by: Aditya Rajan <arajan@redhat.com>
2021-09-02 17:30:02 +05:30
Paul Holzinger
a55f595fe1 podman stop always cleanup
When a container is configured for auto removal podman stop should still
do cleanup, there is no guarantee the the cleanup process spawned by
conmon will be successful. Also a user expects after podman stop that
the network/mounts are cleaned up. Therefore podman stop should not return
early and instead do the cleanup and ignore errors if the container was
already removed.

[NO TESTS NEEDED] I don't know how to test this.

Fixes #11384

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-09-01 12:56:38 +02:00
cdoern
d28e85741f InfraContainer Rework
InfraContainer should go through the same creation process as regular containers. This change was from the cmd level
down, involving new container CLI opts and specgen creating functions. What now happens is that both container and pod
cli options are populated in cmd and used to create a podSpecgen and a containerSpecgen. The process then goes as follows

FillOutSpecGen (infra) -> MapSpec (podOpts -> infraOpts) -> PodCreate -> MakePod -> createPodOptions -> NewPod -> CompleteSpec (infra) -> MakeContainer -> NewContainer -> newContainer -> AddInfra (to pod state)

Signed-off-by: cdoern <cdoern@redhat.com>
2021-08-26 16:05:16 -04:00
OpenShift Merge Robot
94c37d7d47 Merge pull request #11298 from baude/kubeupdown
teardown play kube
2021-08-26 13:58:44 -04:00
OpenShift Merge Robot
49cfed756f Merge pull request #11103 from jwhonce/wip/bindings
Fix file descriptor leaks in bindings and add test
2021-08-25 14:42:12 -04:00
Jhon Honce
1dc6d14735 Fix file descriptor leaks and add test
* Add response.Body.Close() where needed to release HTTP
  connections to API server.
* Add tests to ensure no general leaks occur. 100% coverage would be
  required to ensure no leaks on any call.
* Update code comments to be godoc correct

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-08-24 16:36:10 -07:00
Brent Baude
1e176923b1 teardown play kube
add the ability for play kube to tear down based on the yaml used to
play it.  it is indicated by --down in the play kube command.  volumes
are NOT deleted during the teardown.  pods and their containers are
stopped and removed.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2021-08-24 14:26:14 -05:00
OpenShift Merge Robot
23f9565547 Merge pull request #11263 from nalind/journal-read
libpod/Container.readFromJournal(): don't skip the first entry
2021-08-24 14:16:22 -04:00
Paul Holzinger
c0b1edd6a4 Network interface
Implement a new network interface to abstract CNI from libpod. The
interface is implemented for the CNI backend but in the future we can
add more backends.

The code is structured in three new packages:
- `libpod/network/types`: contains the interface definition
  and the necessary types for it.
- `libpod/network/cni` contains the interface implementation for the CNI
  backend.
- `libpod/network/util` a set of utility functions related to
  networking.

The CNI package uses ginkgo style unit tests. To test Setup/Teardown the
test must be run as root. Each test will run in their own namespace to
make the test independent from the host environment.

New features with the CNI backend:
- The default network will be created in memory if it does not exists on
  disk.
- It can set more than one static IP per container network.
- Networks are loaded once from disk and only if this interface is
  used, e.g. for commands such as `podman info` networks are not loaded.
  This reduces unnecessary disk IO.

This commit only adds the interface it is not wired into libpod. This
requires a lot of breaking changes which will be done in a followup
commit.

Once this is integrated into libpod the current network code under
`libpod/network` should be removed. Also the dependency on OCICNI
should be dropped.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-08-24 12:28:03 +02:00
Nalin Dahyabhai
21f396de6f logs: adjust handling around partial log messages
In libpod/logs.LogLine.Write(), don't write a newline to stdout/stderr
when the log message is only part of a line.

In libpod.ConmonOCIRuntime.HTTPAttach(), don't send a newline over the
HTTP connection when the log message is only part of a line.

In pkg/api/handlers/compat.LogsFromContainer(), don't send a newline
over the HTTP connection when the log message is only part of a line,
and don't make doing so conditional on whether or not the client used
the docker or podman endpoint.

In pkg/domain/infra/tunnel.ContainerEngine.ContainerLogs(), don't add
our own newline to log messages, since they already come through from
the server when they need to.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2021-08-23 18:00:18 -04:00
OpenShift Merge Robot
90cf78b199 Merge pull request #11290 from flouthoc/volume-export
volumes: Add support for `volume export` which allows exporting content to  external path.
2021-08-23 14:01:20 -04:00
flouthoc
edddfe8c4f volumes: Add support for exporting volumes to external tar
Adds support for transferring data between systems and backing up systems.
Use cases: recover from disasters or move data between machines.

Signed-off-by: flouthoc <flouthoc.git@gmail.com>
2021-08-23 20:42:41 +05:30
Valentin Rothberg
33fee83dea add flag to record memory profiles
Add a new flag `--memory-profile=$path` which creates a memory profile.
The generated profile can later be analyzed via `go tool pprof`.

[NO TESTS NEEDED] since it's a hidden flag, devs-only.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-08-20 15:35:58 +02:00
Brent Baude
cd40c875ac Add ability to build images in play kube
When playing a kube YAML file, it can be desirable to be able to build
an image on the fly.  This is good for development of an image and YAML
files and somewhat mocks what compose does.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2021-08-18 09:55:24 -05:00
Valentin Rothberg
fe2be7f886 make sure that signal buffers are sufficiently big
Dealing with os.Signal channels seems more like an art than science
since signals may get lost.  os.Notify doesn't block on an unbuffered
channel, so users are expected to know what they're doing or hope for
the best.

In the recent past, I've seen a number of flakes and BZs on non-amd64
architectures where I was under the impression that signals may got
lost, for instance, during stop and exec.

[NO TESTS NEEDED] since this is art.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-08-17 12:10:20 +02:00
Daniel J Walsh
404488a087 Run codespell to fix spelling
[NO TESTS NEEDED] Just fixing spelling.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-08-11 16:41:45 -04:00
openshift-ci[bot]
1968fdc874 Merge pull request #11173 from jmguzik/pod-ps-until-filter
Add until filter to podman pod ps
2021-08-11 08:38:20 +00:00
Jakub Guzik
ed30ae4a8a Add until filter to podman pod ps
This commit adds additional until filter to podman pod ps (ls/list).
Additionally, it also adds descriptions for podman pod ps filters available
via http api.

Signed-off-by: Jakub Guzik <jakubmguzik@gmail.com>
2021-08-10 22:10:40 +02:00
Daniel J Walsh
221b1add74 Add support for pod inside of user namespace.
Add the --userns flag to podman pod create and keep
track of the userns setting that pod was created with
so that all containers created within the pod will inherit
that userns setting.

Specifically we need to be able to launch a pod with
--userns=keep-id

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2021-08-09 15:17:22 -04:00
openshift-ci[bot]
8409817733 Merge pull request #11074 from vrothberg/auto-update-rollback
auto-update: simple rollback
2021-08-06 12:26:31 +00:00
openshift-ci[bot]
1f0a24437d Merge pull request #11011 from baude/initcontainers
implement init containers in podman
2021-08-05 13:52:15 +00:00
Valentin Rothberg
30df551bde auto-update: simple rollback
Add support for simple rollbacks during `podman auto-update`.  Rollbacks
are enabled by default.  If a systemd unit cannot be restarted after an
update, the previous image will be retagged and the unit will be
restarted a second time.

Add system tests for rollbacks.  Also fix a bug in the restart sequence;
we have to use the channel to actually know whether the restart was
successful or not.

NOTE: To make rollbacks really useful, users must run their containers
with `--sdnotify=container` such that the containers send the ready
message over the (mounted) socket.  This way, restarting the systemd
units during auto update will block until the message has been received
(or a timeout kicked in).

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-08-05 15:20:38 +02:00
Brent Baude
3c3fa6fac4 implement init containers in podman
this is the first pass at implementing init containers for podman pods.
init containersare made popular by k8s as a way to run setup for pods
before the pods standard containers run.

unlike k8s, we support two styles of init containers: always and
oneshot.  always means the container stays in the pod and starts
whenever a pod is started.  this does not apply to pods restarting.
oneshot means the container runs onetime when the pod starts and then is
removed.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2021-08-04 14:14:36 -05:00
Daniel J Walsh
41f94a4dc1 Fix podman unpause,pause,kill --all to work like podman stop --all
Currently if you execute podman unpause --all, podman pause --all
Podman shows attempts to unpause containers that are not paused
and prints an error.  This PR catches this error and only prints errors if
a paused container was not able to be unpaused.

Currently if you execute podman pause --all or podman kill --all, Podman
Podman shows attempts to pause or kill containers that are not running
and prints an error.  This PR catches this error and only prints errors if
a running container was not able to be paused or killed.

Also change printing of multiple errors to go to stderr and to prefix
"Error: " in front to match the output of the last error.

Fixes: https://github.com/containers/podman/issues/11098

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-08-04 12:50:00 -04:00
openshift-ci[bot]
8aa869e628 Merge pull request #10973 from rhatdan/quota
Support size options on builtin volumes
2021-08-04 11:59:58 +00:00
openshift-ci[bot]
3a922cbc99 Merge pull request #11003 from pascomnet/f_stats
stats: add a interval parameter to cli and api stats streaming
2021-08-04 09:56:57 +00:00
openshift-ci[bot]
bdbc21095a Merge pull request #10828 from cdoern/scp
Created image scp feature
2021-08-02 17:15:54 +00:00
Daniel J Walsh
c0952c7334 Support size and inode options on builtin volumes
[NO TESTS NEEDED] Since it is difficult to setup xfs quota

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1982164

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-08-02 10:32:45 -04:00
cdoern
1d10ca739f Created scp.go image_scp_test.go and podman-image-scp.1.md
added functionality for image secure copying from local to remote.
Also moved system connection add code around a bit so functions within that file
can be used by scp.

Signed-off-by: cdoern <cdoern@redhat.com>
2021-07-30 17:19:24 -04:00
flouthoc
2a484e782a ps: support the container notation for ps --filter network=...
Signed-off-by: flouthoc <flouthoc.git@gmail.com>
2021-07-30 19:31:05 +05:30
OpenShift Merge Robot
f9395ddc5a Merge pull request #11067 from vrothberg/fix-10154-2
remote build: fix streaming and error handling
2021-07-28 16:17:45 +02:00
OpenShift Merge Robot
6c5966cf3c Merge pull request #10910 from adrianreber/2021-07-12-checkpoint-restore-into-pod
Add support for checkpoint/restore into and out of pods
2021-07-28 14:48:28 +02:00
Valentin Rothberg
4df6e31ccb remote build: fix streaming and error handling
Address a number of issues in the streaming logic in remote build, most
importantly an error in using buffered channels on the server side.

The pattern below does not guarantee that the channel is entirely read
before the context fires.

for {
	select {
		case <- bufferedChannel:
		...
		case <- ctx.Done():
		...
	}
}

Fixes: #10154
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-28 14:46:15 +02:00
OpenShift Merge Robot
a5de831418 Merge pull request #11049 from vrothberg/fix-7370
support container to container copy
2021-07-27 22:20:45 +02:00
OpenShift Merge Robot
a6a0f6c192 Merge pull request #11061 from giuseppe/migrate-no-move-to-cgroup
command: migrate doesn't move process to cgroup
2021-07-27 20:46:46 +02:00
Adrian Reber
eb94467780 Support checkpoint/restore with pods
This adds support to checkpoint containers out of pods and restore
container into pods.

It is only possible to restore a container into a pod if it has been
checkpointed out of pod. It is also not possible to restore a non pod
container into a pod.

The main reason this does not work is the PID namespace. If a non pod
container is being restored in a pod with a shared PID namespace, at
least one process in the restored container uses PID 1 which is already
in use by the infrastructure container. If someone tries to restore
container from a pod with a shared PID namespace without a shared PID
namespace it will also fail because the resulting PID namespace will not
have a PID 1.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-07-27 16:10:44 +02:00
Mehul Arora
6fe03b25ab support container to container copy
Implement container to container copy.  Previously data could only be
copied from/to the host.

Fixes: #7370
Co-authored-by: Mehul Arora <aroram18@mcmaster.ca>
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-27 15:32:23 +02:00
Giuseppe Scrivano
63ef5576ed command: migrate doesn't move process to cgroup
add a new annotation for the "system migrate" command to not move the
pause process to a separate cgroup.

The operation is not needed since "system migrate" destroys the pause
process, so there won't be any process left to move to a cgroup.

[NO TESTS NEEDED]

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-07-27 14:26:23 +02:00