Commit Graph

2939 Commits

Author SHA1 Message Date
Paul Holzinger
fe90a45e0d Add flag to overwrite network backend from config
To make testing easier we can overwrite the network backend with the
global `--network-backend` option.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-11 17:30:27 +01:00
Urvashi Mohnani
a55fdbb49c Print headers for system connection ls
Print out the headers even if the system connection list
is empty to match the behavior of other list commands.

Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2021-11-10 14:48:08 -05:00
OpenShift Merge Robot
4bf0146c29 Merge pull request #12255 from vrothberg/fix-11970
podman load: support downloading files
2021-11-10 18:27:39 +01:00
Valentin Rothberg
1ef66d6d7f podman load: support downloading files
Support downloading files, for instance via
`podman load -i server.com/image.tar`.  The specified URL is downloaded
in the frontend and stored as a temp file that gets passed down to the
backend.

Also vendor in c/common@main to use the new `pkg/download`.

Fixes: #11970
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-11-10 15:43:16 +01:00
Valentin Rothberg
df06664363 pod create: read infra image from containers.conf
Fix a bug where pods would be created with the hard-coded default infra
image instead of the custom one from containers.conf.  Add a simple
regression test.

Fixes: #12245
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-11-10 12:59:42 +01:00
OpenShift Merge Robot
6d56ac7e0f Merge pull request #12222 from Luap99/pod-create-net
pod create: read network mode from config
2021-11-09 02:08:56 +01:00
Paul Holzinger
8b7b0b7caf pod create: read network mode from config
When we create a pod we have to parse the network mode form the config
file. This is a regression in commit d28e85741f.

Fixes #12207

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-08 18:59:28 +01:00
OpenShift Merge Robot
d0a44755c7 Merge pull request #12036 from jwhonce/issues/11984
test connection add
2021-11-08 18:53:29 +01:00
OpenShift Merge Robot
abfec8144a Merge pull request #11953 from markusthoemmes/help-default
Display help text on empty subcommand by default
2021-11-08 18:23:17 +01:00
Jhon Honce
e907f095b2 test connection add
* Fix connection JSON encoding
* Add custom ginkgo matchers for connection testing
* Cleanup code

Fixes #11984

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-11-08 09:20:58 -07:00
OpenShift Merge Robot
22ef488d24 Merge pull request #12213 from flouthoc/system-connection-rm-all
system: Adds support for removing all named destination via `--all`
2021-11-08 16:50:59 +01:00
OpenShift Merge Robot
c9ba1fb7d4 Merge pull request #11958 from cdoern/scp
Podman Image SCP rootful to rootless transfer
2021-11-08 16:19:57 +01:00
Aditya Rajan
338eb9d75e system: Adds support for removing all named destination via --all
Adds support of dropping all named destination from system connections via `--all`.

Closes: https://github.com/containers/podman/issues/12018

Signed-off-by: Aditya Rajan <arajan@redhat.com>
2021-11-08 19:42:45 +05:30
OpenShift Merge Robot
abbd6c167e Merge pull request #11890 from Luap99/ports
libpod: deduplicate ports in db
2021-11-06 10:39:16 +01:00
OpenShift Merge Robot
e826158b4b Merge pull request #12183 from Luap99/rootless-netns-cleanup
Fix rootless cni netns cleanup logic and rename to rootless netns
2021-11-05 18:59:55 +01:00
cdoern
ffa5ed0e0e Podman Image SCP rootful to rootless transfer
Added functionality for users to transfer images from root storage to rootless storage without using sshd. This is
done through rootful podman by running `sudo podman image scp root@localhost::image user@localhost:: the user is needed
in order to find and use their uid/gid to exec a new process.

added necessary tests, and functions for this implementation. Created new image function Transfer so that
the underlying code is majorly removed from CLI

Signed-off-by: cdoern <cdoern@redhat.com>
2021-11-05 12:04:20 -04:00
Paul Holzinger
7f433df7e7 rename rootless cni ns to rootless netns
Since we want to use the rootless cni ns also for netavark we should
pick a more generic name. The name is now "rootless network namespace"
or short "rootless netns".

The rename might cause some issues after the update but when the
all containers are restarted or the host is rebooted it should work
correctly.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-05 15:44:37 +01:00
Markus Thömmes
756dda298c Keep error semantics intact
Signed-off-by: Markus Thömmes <markusthoemmes@me.com>
2021-11-05 09:34:22 +01:00
Nalin Dahyabhai
b85e3764ac tweak a couple of flag descriptions in help output
Descriptions of flags don't need to start with whitespace of their own.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2021-11-04 16:06:24 -04:00
Anders F Björklund
6e6388eac4 Make stop message more similar to start
[NO TESTS NEEDED]

Signed-off-by: Anders F Björklund <anders.f.bjorklund@gmail.com>
2021-11-02 22:26:04 +01:00
OpenShift Merge Robot
7cd317e9c3 Merge pull request #12141 from kprav33n/help-doc
Fix help message case for `podman version`
2021-10-30 08:56:45 +00:00
OpenShift Merge Robot
09efcd48a8 Merge pull request #12090 from afbjorklund/image-stream
Record the image stream along with the path
2021-10-29 20:22:29 +00:00
Praveen Kumar
e69eae6458 Fix help message case for podman version
This is a cosmetic change. The help message for `podman version` is in
title case whereas all other command help messages are not in title
case. This stands out as inconsistent when looking at the output of
`podman help`.

Signed-off-by: Praveen Kumar <praveen+git@kumar.in>
2021-10-29 11:49:49 -07:00
Paul Holzinger
0136a66a83 libpod: deduplicate ports in db
The OCICNI port format has one big problem: It does not support ranges.
So if a users forwards a range of 1k ports with podman run -p 1001-2000
we have to store each of the thousand ports individually as array element.
This bloats the db and makes the JSON encoding and decoding much slower.
In many places we already use a better port struct type which supports
ranges, e.g. `pkg/specgen` or the new network interface.

Because of this we have to do many runtime conversions between the two
port formats. If everything uses the new format we can skip the runtime
conversions.

This commit adds logic to replace all occurrences of the old format
with the new one. The database will automatically migrate the ports
to new format when the container config is read for the first time
after the update.

The `ParsePortMapping` function is `pkg/specgen/generate` has been
reworked to better work with the new format. The new logic is able
to deduplicate the given ports. This is necessary the ensure we
store them efficiently in the DB. The new code should also be more
performant than the old one.

To prove that the code is fast enough I added go benchmarks. Parsing
1 million ports took less than 0.5 seconds on my laptop.

Benchmark normalize PortMappings in specgen:
Please note that the 1 million ports are actually 20x 50k ranges
because we cannot have bigger ranges than 65535 ports.
```
$ go test -bench=. -benchmem  ./pkg/specgen/generate/
goos: linux
goarch: amd64
pkg: github.com/containers/podman/v3/pkg/specgen/generate
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
BenchmarkParsePortMappingNoPorts-12             480821532                2.230 ns/op           0 B/op          0 allocs/op
BenchmarkParsePortMapping1-12                      38972             30183 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMapping100-12                    18752             60688 ns/op          141088 B/op        315 allocs/op
BenchmarkParsePortMapping1k-12                      3104            331719 ns/op          223840 B/op       3018 allocs/op
BenchmarkParsePortMapping10k-12                      376           3122930 ns/op         1223650 B/op      30027 allocs/op
BenchmarkParsePortMapping1m-12                         3         390869926 ns/op        124593840 B/op   4000624 allocs/op
BenchmarkParsePortMappingReverse100-12             18940             63414 ns/op          141088 B/op        315 allocs/op
BenchmarkParsePortMappingReverse1k-12               3015            362500 ns/op          223841 B/op       3018 allocs/op
BenchmarkParsePortMappingReverse10k-12               343           3318135 ns/op         1223650 B/op      30027 allocs/op
BenchmarkParsePortMappingReverse1m-12                  3         403392469 ns/op        124593840 B/op   4000624 allocs/op
BenchmarkParsePortMappingRange1-12                 37635             28756 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange100-12               39604             28935 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange1k-12                38384             29921 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange10k-12               29479             40381 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange1m-12                  927           1279369 ns/op          143022 B/op        164 allocs/op
PASS
ok      github.com/containers/podman/v3/pkg/specgen/generate    25.492s
```

Benchmark convert old port format to new one:
```
go test -bench=. -benchmem  ./libpod/
goos: linux
goarch: amd64
pkg: github.com/containers/podman/v3/libpod
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
Benchmark_ocicniPortsToNetTypesPortsNoPorts-12          663526126                1.663 ns/op           0 B/op          0 allocs/op
Benchmark_ocicniPortsToNetTypesPorts1-12                 7858082               141.9 ns/op            72 B/op          2 allocs/op
Benchmark_ocicniPortsToNetTypesPorts10-12                2065347               571.0 ns/op           536 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts100-12                138478              8641 ns/op            4216 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts1k-12                   9414            120964 ns/op           41080 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts10k-12                   781           1490526 ns/op          401528 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts1m-12                      4         250579010 ns/op        40001656 B/op          4 allocs/op
PASS
ok      github.com/containers/podman/v3/libpod  11.727s
```

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-10-27 18:59:56 +02:00
Valentin Rothberg
ff31f2264d container create: fix --tls-verify parsing
Make sure that the value is only set if specified on the CLI.  c/image
already defaults to true but if set in the system context, we'd skip
settings in the registries.conf.

Fixes: #11933
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-10-27 14:36:25 +02:00
Anders F Björklund
864748df5b Record the image stream along with the path
[NO TESTS NEEDED]

Signed-off-by: Anders F Björklund <anders.f.bjorklund@gmail.com>
2021-10-26 21:58:39 +02:00
Valentin Rothberg
75f478c08b pod create: remove need for pause image
So far, the infra containers of pods required pulling down an image
rendering pods not usable in disconnected environments.  Instead, build
an image locally which uses local pause binary.

Fixes: #10354
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-10-26 13:51:45 +02:00
Daniel J Walsh
acd8b49000 Add support to play kube for --log-opt
Fixes: https://github.com/containers/podman/issues/11727

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-25 11:40:42 -04:00
Stefan Weil
24c83980e5 Fix some typos in documentation and comments (found by codespell)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2021-10-25 07:41:14 +02:00
OpenShift Merge Robot
5dd211f91b Merge pull request #11991 from rhatdan/size
Allow API to specify size and inode quota
2021-10-22 14:18:45 +00:00
Daniel J Walsh
8e3760c29f Remove --kernel-memory options
Kernel memory option has been depracated in runtime-spec,  It is
believed that it will not work properly on certain kernels.  runc
ignores it.

This PR removes documentation of the flag and also prints a warning if
a user uses it.

[NO NEW TESTS NEEDED]

Helps Fix: https://github.com/containers/podman/issues/12045

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-21 13:29:44 -04:00
Ed Santiago
70a5d8cd1d System tests: confirm that -a and -l clash
...and fix one instance where there was no check

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-10-20 11:09:32 -06:00
Daniel J Walsh
34dcbc9491 Change podman connection list to use default field
Stop using "*" to indicate default.  Add default field to make
it more obvios and the json field more machine usable.

Fixes: https://github.com/containers/podman/issues/12019

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-18 16:52:52 -04:00
Daniel J Walsh
087f8fc73b Allow API to specify size and inode quota
Fixes: https://github.com/containers/podman/issues/11016

[NO NEW TESTS NEEDED] We have no easy way to tests this in
CI/CD systems.  Requires quota to be setup on directories to work.

Fixes: https://github.com/containers/podman/issues/11016

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-18 14:44:10 -04:00
Daniel J Walsh
468e7c689b Move CONTAINER_HOST and _CONNECTION to IsRemote Function
Current code does not check early enough.

Follow up to https://github.com/containers/podman/pull/11978

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-15 14:40:02 -04:00
Daniel J Walsh
4f857bc106 If CONTAINER_HOST env variable is set default podman --remote=true
Users enabling CONTAINER_HOST==PATH is indicating to podman they intend
to use remote functionality.

Fixes: https://github.com/containers/podman/issues/11196

Update man pages to document all of the environment variables.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-14 17:08:48 -04:00
OpenShift Merge Robot
77ae43af61 Merge pull request #11971 from rhatdan/archive
Don't use docker/pkg/archive, use containers/storage/pkg/archive
2021-10-14 19:53:55 +02:00
Daniel J Walsh
50b9d82f2e Don't use docker/pkg/archive, use containers/storage/pkg/archive
[NO NEW TESTS NEEDED]

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-14 10:28:42 -04:00
Paul Holzinger
8cae2978ef Fix panic in container create compat api
The bind and tmpfs options can be nil, we have to check that before we
try to use it.

Fixes #11961

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-10-14 12:56:44 +02:00
OpenShift Merge Robot
1c156f2267 Merge pull request #11948 from rhatdan/codespell
codespell code
2021-10-13 14:46:00 +02:00
OpenShift Merge Robot
e3cf9b685a Merge pull request #11892 from vrothberg/search
podman search: display only name and description by default
2021-10-13 14:29:59 +02:00
Markus Thömmes
cc875fd268 Display help text on empty subcommand by default
Signed-off-by: Markus Thömmes <markusthoemmes@me.com>
2021-10-13 11:48:45 +02:00
OpenShift Merge Robot
f12dc28bbc Merge pull request #11565 from Luap99/rootlessport-bin
rootlessport: reduce memory usage of the process
2021-10-13 10:41:59 +02:00
Valentin Rothberg
14bfee31f7 podman search: display only name and description by default
Change the default format of `podman search` to only display the name
and the description of each image.  The index is redundant to the name
and consumes a lot of space, and other descriptors (i.e., stars,
official, automated) are specific to Docker Hub and also consume a lot
space.  Users can still use `--format` for displaying the descriptors
they want to.

Add a `--compatible` flag to offer an easy way to get them back.

Also update the man page to account for the behavior and get some fresh
data in the examples.

Motivated by a recent conversation in libimage:
https://github.com/containers/common/pull/802#issuecomment-937108734

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-10-13 10:13:24 +02:00
Daniel J Walsh
8600bce53a codespell code
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-12 16:44:25 -04:00
OpenShift Merge Robot
72e87c0ca8 Merge pull request #11924 from jwhonce/issues/11894
Refactor podman search to be more code friendly
2021-10-12 22:17:09 +02:00
Paul Holzinger
3ba69dccf7 rootlessport: reduce memory usage of the process
Don't use reexec for the rootlessport process, instead make it a
separate binary to reduce the memory usage. The problem with reexec is
that it will import all packages that podman uses and therefore loads a
lot of stuff into the heap. The rootlessport process however only needs
the rootlesskit library.
The memory usage is a concern since the rootlessport process will spawn
two process per container which has ports forwarded. The processes stay
until the container dies. On my laptop the current reexec version uses
47800 KB RSS. The new separate binary only uses 4540 KB RSS. This is
more than a 90% improvement.

The Makefile has been updated to compile the new binary and install it
to the libexec directory.

Fixes #10790

[NO TESTS NEEDED]

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-10-12 21:43:11 +02:00
OpenShift Merge Robot
03095bd090 Merge pull request #11927 from jwhonce/issues/11921
Fix CI flake on time of shutdown for API service
2021-10-12 21:08:10 +02:00
Jhon Honce
0459484bdf Fix CI flake on time of shutdown for API service
* Increase timeout for tests to 10s
* To aid in debugging add PID to shutdown package logging
* Added new message for forced service shutdown
* Always wait for HTTP server to shutdown, duration of 0 not friendly
  to clients

Note: The log event

"IdleTracker: StateClosed transition by connection marked un-managed"

denotes a TCP connection has been initiated but no HTTP request was sent.
And is expected during these tests.

Fixes #11921

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-10-12 09:53:19 -07:00
Jhon Honce
b28a8bc198 Refactor podman search to be more code friendly
* JSON and API description fields are no longer truncated. Formatting
  moved to client, better support of MVP.
* --no-trunc now defaults to true
* Updated tests for changes

Closes #11894

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-10-12 09:30:40 -07:00