mtd-utils.git
6 days agoubiupdatevol: Prevent null pointer dereference master
Bastian Germann [Sun, 8 Sep 2019 19:59:22 +0000 (21:59 +0200)]
ubiupdatevol: Prevent null pointer dereference

libubi_close(libubi) is called in the error handler if libubi is null.
Prevent that by handling the error case similar to the other ubi
executables.

Signed-off-by: Bastian Germann <bastiangermann@fishpost.de>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
3 weeks agoAdd ubihealthd to gitignore file
David Oberhollenzer [Sun, 25 Aug 2019 11:51:21 +0000 (13:51 +0200)]
Add ubihealthd to gitignore file

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
3 weeks agoubi-utils: Implement a ubihealthd
Richard Weinberger [Thu, 25 Jul 2019 20:34:42 +0000 (22:34 +0200)]
ubi-utils: Implement a ubihealthd

ubihealthd is a simple daemon which scans every PEB
of an UBI device in random order.
It helps to deal with read disturb on systems which either
reboot seldom, use fastmap or read few data.

To use this daemon you need Linux >= v5.1.

Signed-off-by: Richard Weinberger <richard@nod.at>
4 weeks agomkfs.ubifs: Add authentication support
Sascha Hauer [Tue, 6 Aug 2019 10:49:28 +0000 (12:49 +0200)]
mkfs.ubifs: Add authentication support

This adds support for authenticated UBIFS images. In authenticated
images all UBIFS nodes are hashed as described in the UBIFS
authentication whitepaper. Additionally the superblock node contains a
hash of the master node and itself is cryptographically signed in a node
following the superblock node. The signature is in PKCS #7 CMS format.

To generate an authenticated image these options are necessary:

--hash-algo=NAME     hash algorithm to use for signed images
                     (Valid options include sha1, sha256, sha512)
--auth-key=FILE      filename or PKCS #11 uri containing the authentication key
                     for signing
--auth-cert=FILE     Authentication certificate filename for signing. Unused
                     when certificate is provided via PKCS #11

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
4 weeks agoubifs-media: Update to Linux-5.3-rc3
Sascha Hauer [Tue, 6 Aug 2019 10:49:27 +0000 (12:49 +0200)]
ubifs-media: Update to Linux-5.3-rc3

This updates ubifs-media.h to Linux-5.3-rc3 which brings us the bits
and pieces necessary for UBIFS authentication and offline signing.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
8 weeks agoRelease mtd-utils-2.1.1 v2.1.1
David Oberhollenzer [Sun, 21 Jul 2019 20:40:28 +0000 (22:40 +0200)]
Release mtd-utils-2.1.1

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
8 weeks agomkfs.ubifs: fix description of favor_lzo
Uwe Kleine-König [Thu, 18 Jul 2019 09:42:58 +0000 (11:42 +0200)]
mkfs.ubifs: fix description of favor_lzo

"favor_lzo" uses "lzo" unless the space savings when using "zlib" are
big. The current wording got this wrong.

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2 months agomkfs.ubifs: fix compilation without ZSTD
Martin Kaiser [Mon, 15 Jul 2019 13:38:58 +0000 (15:38 +0200)]
mkfs.ubifs: fix compilation without ZSTD

Fix a trivial typo to make sure that zstd.h is included only if
zstd is not disabled.

Signed-off-by: Martin Kaiser <martin@kaiser.cx>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2 months agomkfs.ubifs: remove ZSTD_CLEVEL_DEFAULT for backwards compatibillity
David Oberhollenzer [Tue, 9 Jul 2019 10:22:30 +0000 (12:22 +0200)]
mkfs.ubifs: remove ZSTD_CLEVEL_DEFAULT for backwards compatibillity

Support for ZSTD compression has been added recently through the ZSTD
library, which is famously known for its incredibly well designed and
stable API.

This patch removes usage of ZSTD_CLEVEL_DEFAULT, which isn't exposed
in older versions of the ZSTD library, and replaces it with with the
constant parameter 0. According to the documentation this should then
use a reasonable default (which is defined internally).

Other possible approachs include defining ZSTD_CLEVEL_DEFAULT to 3
(the value it _currently_ has) if it isn't defined. This patch chooses
the approach of passing 0 since this seems to be encouraged by the
existing documentation.

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2 months agoubinize: Exit with non-zero exit code on error.
Patrick Doyle [Mon, 1 Jul 2019 18:25:28 +0000 (14:25 -0400)]
ubinize: Exit with non-zero exit code on error.

...specifically -1 in all of the new cases.

Signed-off-by: Patrick Doyle <pdoyle@irobot.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
3 months agoubi-tests: fm_param: Replace 'fm_auto' with 'fm_autoconvert'
Zhihao Cheng [Fri, 14 Jun 2019 12:19:50 +0000 (20:19 +0800)]
ubi-tests: fm_param: Replace 'fm_auto' with 'fm_autoconvert'

The value of fm_param should be 'fm_autoconvert' rather than 'fm_auto' when
fastmap is supported by kernel. Currently, following verbose will appear in
dmesg when fm_param is set to 'fm_auto':

  ubi: unknown parameter 'fm_auto' ignored

This patch replace 'fm_auto' with 'fm_autoconvert' for fm_param, so ubi
kernel module can receive correct parameters.

----------------------------------------

Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
3 months agoubi-tests: mkvol test: Checks return value 'ENOSPC' for 'ubi_mkvol'
Zhihao Cheng [Fri, 14 Jun 2019 12:18:48 +0000 (20:18 +0800)]
ubi-tests: mkvol test: Checks return value 'ENOSPC' for 'ubi_mkvol'

UBI tests try to create too many volumes in mkvol_bad and mkvol_basic.
Currently mtd-utils allows return value 'ENFILE' from 'ubi_mkvol', that
works fine in most situations. But what if the number of PEBs equals to
the maximum count of volumes? For example, mkvol_basic test will fail in
a 64MiB flash with 512KiB PEB size.
Following is the output of mkvol_basic test:

  ======================================================================
  ======================================================================
  ======================================================================
  Test on mtdram, fastmap enabled, VID header offset factor 1
  ======================================================================
  ======================================================================
  ======================================================================
  mtdram: 64MiB, PEB size 512KiB, fastmap enabled
  Running mkvol_basic /dev/ubi0
  [mkvol_basic] mkvol_multiple():182: function ubi_mkvol() failed with
  error 28 (No space left on device)
  [mkvol_basic] mkvol_multiple():183: vol_id 122
  Error: mkvol_basic failed
  FAILURE

The reason is that there is no available PEB to support a new volume. We
can see following verbose in dmesg:

  ubi0: attached mtd0 (name "mtdram test device", size 64 MiB)
  ubi0: user volume: 0, internal volumes: 1, max. volumes count: 128
  ubi0: available PEBs: 122, total reserved PEBs: 6, PEBs reserved for
  bad PEB handling: 0

The maximum count of volumes is 128, so we can create 128 volumes
theoretically. But there are 122 available PEBs becauese of existence of
reserved PEBs. In addition, a volume occupies at least one PEB. Actually,
we can only create 122 volumes, Therefore, 'ubi_mkvol' returns 'ENOSPC'
when mkvol_basic tries to create 123rd volume. And we can see
corresponding error message in dmesg:

  ubi0 error: ubi_create_volume [ubi]: not enough PEBs, only 0 available
  ubi0 error: ubi_create_volume [ubi]: cannot create volume 122, error -28

So, 'ENOSPC' can happen before 'ENFILE' in flash with a small amount of
PEBs. This patch checks return value 'ENOSPC' for 'ubi_mkvol' when mkvol
test is trying to create too many volumes.

----------------------------------------

Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
3 months agoubi-tests: io_read: Filter invalid offset value before 'lseek' in io_read test
Zhihao Cheng [Fri, 14 Jun 2019 12:18:16 +0000 (20:18 +0800)]
ubi-tests: io_read: Filter invalid offset value before 'lseek' in io_read test

There are many different offset values passed in 'lseek' during io_read
testing of ubi test. The offset value maybe a negative number or a big
number that exceeds the volume data size, which can lead to ubi tests
failure by passing invalid offset value to 'lseek'. For example:

Example 1: The data size of volume is 39525 bytes, offset = (sz) -
MAX_NAND_PAGE_SIZE - 1, where MAX_NAND_PAGE_SIZE is 65536. Here, offset
is a negative value passed to 'lseek', which leads to fail in io_read.

  ======================================================================
  ======================================================================
  ======================================================================
  Test on mtdram, fastmap enabled, VID header offset factor 1
  ======================================================================
  ======================================================================
  ======================================================================
  mtdram: 16MiB, PEB size 16KiB, fastmap enabled
  Running mkvol_basic /dev/ubi0
  Running mkvol_bad /dev/ubi0
  Running mkvol_paral /dev/ubi0
  Running rsvol /dev/ubi0
  Running io_basic /dev/ubi0
  Running io_read /dev/ubi0
  [io_basic] test_read3():189: function seek() failed with error 22
  (Invalid argument)
  [io_basic] test_read3():190: len = 1
  [io_basic] test_read2():237: offset = -26012
  [io_basic] test_read1():303: length = 1
  [io_basic] test_read():362: alignment = 7905
  Error: io_read failed
  FAILURE

Example 2: The data size of volume is 79035 bytes, offset = 2 *
MAX_NAND_PAGE_SIZE, where MAX_NAND_PAGE_SIZE is 65536. Here, offset is a
value exceeds volume size, which leads to fail in io_read.

  ======================================================================
  ======================================================================
  ======================================================================
  Test on mtdram, fastmap enabled, VID header offset factor 1
  ======================================================================
  ======================================================================
  ======================================================================
  mtdram: 16MiB, PEB size 16KiB, fastmap enabled
  Running mkvol_basic /dev/ubi0
  Running mkvol_bad /dev/ubi0
  Running mkvol_paral /dev/ubi0
  Running rsvol /dev/ubi0
  Running io_basic /dev/ubi0
  Running io_read /dev/ubi0
  [io_basic] test_read3():185: function seek() failed with error 22
  (Invalid argument)
  [io_basic] test_read3():186: len = 1
  [io_basic] test_read2():233: offset = 131072
  [io_basic] test_read1():299: length = 1
  [io_basic] test_read():358: alignment = 3
  Error: io_read failed
  FAILURE

This patch checks offset value before executing 'lseek', invalid offset
values are filtered.

----------------------------------------

Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
3 months agoubi-tests: ubi_mkvol_request: Fully initialize 'struct ubi_mkvol_request req'
Zhihao Cheng [Fri, 14 Jun 2019 12:14:38 +0000 (20:14 +0800)]
ubi-tests: ubi_mkvol_request: Fully initialize 'struct ubi_mkvol_request req'

'struct ubi_mkvol_request req' is one parameter of the function 'ubi_mkvol'
, this parameter will be passed to kernel and then be checked. It acts as a
local variable in many ubi tests, such as io_basic, io_read, mkvol_bad,
mkvol_basic, etc.

After commit c355aa465fce ("ubi: expose the volume CRC check skip flag") in
linux-stable, 'struct ubi_mkvol_request' supports a new configuration named
'flags', and req.flags will be checked in kernel function
'verify_mkvol_req'. Currently, there is no initialization for req.flags
before 'ubi_mkvol' invoked. So, req.flags can be an arbitrary number passed
to kernel. When we run ubi tests in qemu (x86_64, kernel image: 5.2.0-rc4),
the following errors may occur:

  ======================================================================
  ======================================================================
  ======================================================================
  Test on mtdram, fastmap enabled, VID header offset factor 1
  ======================================================================
  ======================================================================
  ======================================================================
  mtdram: 16MiB, PEB size 16KiB, fastmap enabled
  Running mkvol_basic /dev/ubi0
  Running mkvol_bad /dev/ubi0
  [mkvol_bad] test_mkvol():105: ubi_mkvol failed with error 22
  (Invalid argument), expected 28 (No space left on device)
  [mkvol_bad] test_mkvol():105: bytes = 16060929
  Error: mkvol_bad failed
  FAILURE

This patch fully initializes every 'struct ubi_mkvol_request req' passed to
'ubi_mkvol', which can fix the bug that the ubi test failed caused by that
req.flags was not initialized. And it is still compatible with old kernel
before kernel commit c355aa465fce ("ubi: expose the volume CRC check skip
flag").

----------------------------------------

Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
3 months agomkfs.ubifs: Add ZSTD compression
Sebastian Andrzej Siewior [Sat, 1 Jun 2019 10:43:23 +0000 (12:43 +0200)]
mkfs.ubifs: Add ZSTD compression

I added ZSTD support to mkfs.ubifs and compared the ZSTD results with
zlib/lzo and the available ZSTD compression levels. The results are in
the following table:

Comp    image MiB        time  image2 MiB        time
none          271   0m 0,723s         223   0m 0,589s
lzo           164   0m13,705s         116   0m11,636s
zlib          150   0m 7,654s         103   0m 6,347s
favor-lzo     158   0m21,137s         110   0m17,764s
zstd-01       154   0m 1,607s         106   0m 1,429s
zstd-02       153   0m 1,704s         105   0m 1,479s
zstd-03*      152   0m 1,888s         104   0m 1,668s
zstd-04       151   0m 2,741s         103   0m 2,391s
zstd-05       150   0m 3,257s         102   0m 2,916s
zstd-06       150   0m 3,735s         102   0m 3,356s
zstd-07       150   0m 4,066s         102   0m 3,705s
zstd-08       152   0m 1,857s         104   0m 1,644s
zstd-09       152   0m 1,855s         104   0m 1,639s
zstd-10       150   0m 6,654s         102   0m 6,195s
zstd-11       150   0m10,027s         102   0m 9,130s
zstd-12       149   0m14,724s         101   0m13,415s
zstd-13       148   0m18,232s         100   0m16,719s
zstd-14       148   0m20,859s         100   0m19,554s
zstd-15       148   0m25,033s         100   0m23,186s
zstd-16       148   0m38,837s         100   0m36,543s
zstd-17       148   0m46,051s         100   0m43,120s
zstd-18       148   0m49,157s         100   0m45,807s
zstd-19       148   0m49,421s         100   0m45,951s
zstd-20       148   0m51,271s         100   0m48,030s
zstd-21       148   0m51,015s         100   0m48,676s
zstd-22       148   0m52,575s         100   0m50,013s

The UBIFS image was created via
  mkfs.ubifs -x $Comp -m 512 -e 128KiB -c 2200 -r $image $out

I used "debootstrap sid" to create a basic RFS and the results are in
the `image' column. The image2 column denotes the results for the same
image but with .deb files removed.
The time column contains the output of the run time of the command.

ZSTD's compression level three is currently default. Based on the
compression results (for the default level) it outperforms LZO in
run time and compression and is almost as good as ZLIB in terms of
compression but quicker.
The higher compression levels make almost no difference in compression
but take a lot of time.

The compression level used is the default offered by ZSTD. It does not
make sense the higher levels.

Signed-off-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
4 months agoubiformat: Dont ignore sequence number CLI option
Amol Vengurlekar [Mon, 13 May 2019 00:59:19 +0000 (17:59 -0700)]
ubiformat: Dont ignore sequence number CLI option

Image sequence number for the UBI header can be specified for the
ubiformat tool according to the documentation and the help message for
ubiformat. The CLI option --image-seq for image sequence number is not
supported. -Q option for image sequence number is silently ignored.
This patch adds the CLI support for image sequence number.

Signed-off-by: Amol Vengurlekar <amol.sven@gmail.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
4 months agomtd-tests: nandbiterrs: Fix issue that just insert error at bit 7
Xiaolei Li [Mon, 29 Apr 2019 03:28:34 +0000 (11:28 +0800)]
mtd-tests: nandbiterrs: Fix issue that just insert error at bit 7

The function insert_biterror should be designed to insert error at
the first '1' bit starting at offset byte.

But now, only bit 7 of each byte is checked, because checking mask
is always 0x80.

So, do right shift for checking mask after each checking to check
the whole 8 bits of each bytes.

Signed-off-by: Xiaolei Li <xiaolei.li@mediatek.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
4 months agounittests: Define the use of _GNU_SOURCE
Olliver Schinagl [Mon, 15 Apr 2019 08:20:31 +0000 (10:20 +0200)]
unittests: Define the use of _GNU_SOURCE

The unittest suite actually makes use of some _GNU extensions during the
build (loff_t for example). So lets enable this in the makefile.

Signed-off-by: Olliver Schinagl <oliver@schinagl.nl>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
4 months agounittests/libmtd_test: Include fcntl header
Olliver Schinagl [Mon, 15 Apr 2019 08:20:30 +0000 (10:20 +0200)]
unittests/libmtd_test: Include fcntl header

The test library for the mtd unit tests include various type's and
macro's that officially live in fcntl. Each file and header should
always properly include what they use, so lets add the fcntl headers.

Signed-off-by: Olliver Schinagl <oliver@schinagl.nl>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
4 months agounittests/test_lib: Include proper header for _IOC_SIZE
Olliver Schinagl [Mon, 15 Apr 2019 08:20:29 +0000 (10:20 +0200)]
unittests/test_lib: Include proper header for _IOC_SIZE

The macro _IOC_SIZE is not part of sys/ioctl.h but lives in asm/ioctl.h
so we should include the proper header. If we do not, some systems
complain during linking that they cannot find the symbol _IOC_SIZE()
which was not expanded by the pre-compiler.

Signed-off-by: Olliver Schinagl <oliver@schinagl.nl>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
5 months agomkfs.ubifs: fix regression when trying to store device special files
David Oberhollenzer [Wed, 3 Apr 2019 10:54:16 +0000 (12:54 +0200)]
mkfs.ubifs: fix regression when trying to store device special files

Commit a767dd30 added a check to add_inode that bails when trying to
store extra data in anything other than a symlink. The symlink
encryption support added by that commit relies on the assumption.

Unfortionately it was overlooked that device special files also store
the device number as additional data in the inode. The check added in
commit a767dd30 broke support for device files in mkfs.ubifs.

This commit adds a quick and dirty fix, moving the check into the
fscrypt branch, breaking only the fscrypt version but restoring old
functionality for unencrypted file systems.

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
5 months agomkfs.ubifs: fix build without openssl
Baruch Siach [Fri, 29 Mar 2019 10:32:24 +0000 (13:32 +0300)]
mkfs.ubifs: fix build without openssl

Exclude openssl headers when WITH_CRYPTO is not defined.

Fixes this build failure:

In file included from ubifs-utils/mkfs.ubifs/mkfs.ubifs.c:25:0:
ubifs-utils/mkfs.ubifs/mkfs.ubifs.h:49:10: fatal error: openssl/rand.h: No such file or directory
 #include <openssl/rand.h>
          ^~~~~~~~~~~~~~~~

Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
6 months agoRelease mtd-utils-2.1.0 v2.1.0
David Oberhollenzer [Wed, 13 Mar 2019 16:42:12 +0000 (17:42 +0100)]
Release mtd-utils-2.1.0

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
6 months agomtd-utils: fixes verification percent display in flashcp
Andrij Abyzov [Mon, 11 Feb 2019 16:13:40 +0000 (17:13 +0100)]
mtd-utils: fixes verification percent display in flashcp

flashcp was always showing 0% progress due to incorrect
printf format specifier.

Signed-off-by: Andrij Abyzov <drolevar@gmail.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 months agomtd-utils: fixes double free in mkfs.ubifs
Yufen Yu [Thu, 24 Jan 2019 09:06:29 +0000 (17:06 +0800)]
mtd-utils: fixes double free in mkfs.ubifs

In inode_add_xattr(), it malloc a buffer for name, and then passes
the bufffer ptr to add_xattr(). The ptr will be used to create a new
idx_entry in add_to_index().

However, inode_add_xattr() will free the buffer before return.
which can cause double free in write_index(): free(idx_ptr[i]->name)

*** Error in `./mkfs.ubifs': double free or corruption (fasttop): 0x0000000000aae220 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7cbac)[0x7f4881ff5bac]
/lib64/libc.so.6(+0x87a59)[0x7f4882000a59]
/lib64/libc.so.6(cfree+0x16e)[0x7f48820063be]
./mkfs.ubifs[0x402fbf]
/lib64/libc.so.6(__libc_start_main+0xea)[0x7f4881f9988a]
./mkfs.ubifs[0x40356a]

Signed-off-by: Yufen Yu <yuyufen@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 months agomtd-utils: avoid to create two UBI_LAYOUT_VOLUME_ID volume
Yufen Yu [Fri, 25 Jan 2019 07:01:24 +0000 (15:01 +0800)]
mtd-utils: avoid to create two UBI_LAYOUT_VOLUME_ID volume

When we create a ubi image by ubinize, a UBI_LAYOUT_VOLUME_ID
volume will be created by ubigen_write_layout_vol().
However, after the commit 4c00cf2c5816 (ubiformat: remove
no-volume-table option), ubiformat remove novtbl args in format().
As a result, it will also create a layout volume.

When we attempt to do ubiattach, it will fail for ubi_compare_lebs error:
ubi0 error: ubi_compare_lebs: unsupported on-flash UBI format
ubi0 error: ubi_attach_mtd_dev: failed to attach mtd1, error -22

Signed-off-by: Yufen Yu <yuyufen@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
8 months agomkfs.ubifs: fix default cipher in help output
David Gstir [Mon, 3 Dec 2018 13:40:46 +0000 (14:40 +0100)]
mkfs.ubifs: fix default cipher in help output

AES-256-XTS is the default since dd0d9c623e22 ("mkfs.ubifs: Use AES-256-XTS as default"),
we want that to be correctly reflected in the help output as well.

Signed-off-by: David Gstir <david@sigma-star.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Don't compress by default if filesystem is encrypted
Richard Weinberger [Wed, 7 Nov 2018 20:56:35 +0000 (21:56 +0100)]
mkfs.ubifs: Don't compress by default if filesystem is encrypted

Encryption and compression are not friends.
Enable compression in encryption mode only if the user explicitly
sets a compressor.

Signed-off-by: Richard Weinberger <richard@nod.at>
Tested-by: Heiko Schocher <hsdenx.de>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Store compr_size in correct endianness.
Richard Weinberger [Wed, 7 Nov 2018 20:21:08 +0000 (21:21 +0100)]
mkfs.ubifs: Store compr_size in correct endianness.

compr_size has to be in LE16.

Signed-off-by: Richard Weinberger <richard@nod.at>
Tested-by: Heiko Schocher <hsdenx.de>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Fix IV selection
Richard Weinberger [Wed, 7 Nov 2018 20:21:07 +0000 (21:21 +0100)]
mkfs.ubifs: Fix IV selection

We need to check for AES being in 128-cbc mode and not 256-cbc.
fscrypt supports only 128-cbc and 256-xts so far.

Signed-off-by: Richard Weinberger <richard@nod.at>
Tested-by: Heiko Schocher <hsdenx.de>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Use AES-256-XTS as default
Richard Weinberger [Thu, 18 Oct 2018 14:37:18 +0000 (16:37 +0200)]
mkfs.ubifs: Use AES-256-XTS as default

AES-128-CBC should only being used when 256-XTS is too slow
on low end hardware.

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Print key descriptor only when generated
Richard Weinberger [Thu, 18 Oct 2018 14:37:17 +0000 (16:37 +0200)]
mkfs.ubifs: Print key descriptor only when generated

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Enable support for building without crypto
Richard Weinberger [Thu, 18 Oct 2018 14:37:16 +0000 (16:37 +0200)]
mkfs.ubifs: Enable support for building without crypto

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Move RAND_poll to crypto.c
Richard Weinberger [Thu, 18 Oct 2018 14:37:15 +0000 (16:37 +0200)]
mkfs.ubifs: Move RAND_poll to crypto.c

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: More fscryptctl compatibility
Richard Weinberger [Thu, 18 Oct 2018 14:37:14 +0000 (16:37 +0200)]
mkfs.ubifs: More fscryptctl compatibility

fscryptctl reads up to FS_MAX_KEY_SIZE bytes from the source key
to compute the descriptor.

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Fix key descriptor printing
Richard Weinberger [Thu, 18 Oct 2018 14:37:13 +0000 (16:37 +0200)]
mkfs.ubifs: Fix key descriptor printing

normsg() sucks.

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Compute encryption key descriptor automatically
Richard Weinberger [Thu, 18 Oct 2018 14:37:12 +0000 (16:37 +0200)]
mkfs.ubifs: Compute encryption key descriptor automatically

...if none is given. To be compatible with fscryptctl.

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Fixup AES-XTS mode
Richard Weinberger [Thu, 18 Oct 2018 14:37:11 +0000 (16:37 +0200)]
mkfs.ubifs: Fixup AES-XTS mode

In XTS mode we don't need ESSIV, just use the block number
as tweak.
Also apply EVP_EncryptFinal().

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Use correct sizes for keys and hash lengths
Richard Weinberger [Thu, 18 Oct 2018 14:37:10 +0000 (16:37 +0200)]
mkfs.ubifs: Use correct sizes for keys and hash lengths

This works currently by chance since the sizes match, but
that might change with different cipher setups.

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Enable Cipher selection
Richard Weinberger [Thu, 18 Oct 2018 14:37:09 +0000 (16:37 +0200)]
mkfs.ubifs: Enable Cipher selection

No longer hard code AES-128-CBC, we support AES-256-XTS too.

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Correctly use iv lengths in aes-cts mode
Richard Weinberger [Thu, 18 Oct 2018 14:37:08 +0000 (16:37 +0200)]
mkfs.ubifs: Correctly use iv lengths in aes-cts mode

The key length can be very long, for example in xts mode.
So we have to use the right sizes for block and iv lengths.

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Accept 0x prefix for key descriptor
David Oberhollenzer [Thu, 18 Oct 2018 14:37:07 +0000 (16:37 +0200)]
mkfs.ubifs: Accept 0x prefix for key descriptor

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
10 months agomkfs.ubifs: Check length of master key
David Oberhollenzer [Thu, 18 Oct 2018 14:37:06 +0000 (16:37 +0200)]
mkfs.ubifs: Check length of master key

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
10 months agomkfs.ubifs: Cleanup over-long lines
David Oberhollenzer [Thu, 18 Oct 2018 14:37:05 +0000 (16:37 +0200)]
mkfs.ubifs: Cleanup over-long lines

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
10 months agomkfs.ubifs: Move fscrypt definitions and functions out of mkfs.ubifs.c
David Oberhollenzer [Thu, 18 Oct 2018 14:37:04 +0000 (16:37 +0200)]
mkfs.ubifs: Move fscrypt definitions and functions out of mkfs.ubifs.c

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
10 months agomkfs.ubifs: Remove cipher implementations from public header
David Oberhollenzer [Thu, 18 Oct 2018 14:37:03 +0000 (16:37 +0200)]
mkfs.ubifs: Remove cipher implementations from public header

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
10 months agomkfs.ubifs: Initial support for encryption command lines
David Oberhollenzer [Thu, 18 Oct 2018 14:37:02 +0000 (16:37 +0200)]
mkfs.ubifs: Initial support for encryption command lines

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
10 months agomkfs.ubifs: Specify padding policy via command line
David Oberhollenzer [Thu, 18 Oct 2018 14:37:01 +0000 (16:37 +0200)]
mkfs.ubifs: Specify padding policy via command line

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
10 months agomkfs.ubifs: Get key descriptor from command line and master key from file
David Oberhollenzer [Thu, 18 Oct 2018 14:37:00 +0000 (16:37 +0200)]
mkfs.ubifs: Get key descriptor from command line and master key from file

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
10 months agomkfs.ubifs: Make encryption dependend on (not-yet-existant) command line options
David Oberhollenzer [Thu, 18 Oct 2018 14:36:59 +0000 (16:36 +0200)]
mkfs.ubifs: Make encryption dependend on (not-yet-existant) command line options

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
10 months agomkfs.ubifs: Replace constant values with parameters in init_fscrypt_context
David Oberhollenzer [Thu, 18 Oct 2018 14:36:58 +0000 (16:36 +0200)]
mkfs.ubifs: Replace constant values with parameters in init_fscrypt_context

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
10 months agomkfs.ubifs: Cleanup add_dent_node, user path encryption helper
David Oberhollenzer [Thu, 18 Oct 2018 14:36:57 +0000 (16:36 +0200)]
mkfs.ubifs: Cleanup add_dent_node, user path encryption helper

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
10 months agomkfs.ubifs: Seperate path encryption from symlink encryption helper
David Oberhollenzer [Thu, 18 Oct 2018 14:36:56 +0000 (16:36 +0200)]
mkfs.ubifs: Seperate path encryption from symlink encryption helper

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
10 months agomkfs.ubifs: Free all index entry names
Richard Weinberger [Thu, 18 Oct 2018 14:36:55 +0000 (16:36 +0200)]
mkfs.ubifs: Free all index entry names

...and make valgrind memcheck happy

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Make sure we catch nodes that should or should not have name
Richard Weinberger [Thu, 18 Oct 2018 14:36:54 +0000 (16:36 +0200)]
mkfs.ubifs: Make sure we catch nodes that should or should not have name

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Move symlink data encryption to helper function
David Oberhollenzer [Thu, 18 Oct 2018 14:36:53 +0000 (16:36 +0200)]
mkfs.ubifs: Move symlink data encryption to helper function

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
10 months agomkfs.ubifs: Implement file contents encryption
Richard Weinberger [Thu, 18 Oct 2018 14:36:52 +0000 (16:36 +0200)]
mkfs.ubifs: Implement file contents encryption

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Add encrypted symlink support
Richard Weinberger [Thu, 18 Oct 2018 14:36:51 +0000 (16:36 +0200)]
mkfs.ubifs: Add encrypted symlink support

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Pass source/dest key len to key derive function
Richard Weinberger [Thu, 18 Oct 2018 14:36:50 +0000 (16:36 +0200)]
mkfs.ubifs: Pass source/dest key len to key derive function

fscrypto is using the max key lenth (64), so we cannot use the
AES-128-ECB len.

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Add dummy setup for crypto
Richard Weinberger [Thu, 18 Oct 2018 14:36:49 +0000 (16:36 +0200)]
mkfs.ubifs: Add dummy setup for crypto

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Implement filename encryption
Richard Weinberger [Thu, 18 Oct 2018 14:36:48 +0000 (16:36 +0200)]
mkfs.ubifs: Implement filename encryption

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Store directory name len in the temporary index
Richard Weinberger [Thu, 18 Oct 2018 14:36:47 +0000 (16:36 +0200)]
mkfs.ubifs: Store directory name len in the temporary index

since names are no longer strings, we need to know the length.

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Implement fscrypto context store as xattr
Richard Weinberger [Thu, 18 Oct 2018 14:36:46 +0000 (16:36 +0200)]
mkfs.ubifs: Implement fscrypto context store as xattr

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Implement basic fscrypto context passing
Richard Weinberger [Thu, 18 Oct 2018 14:36:45 +0000 (16:36 +0200)]
mkfs.ubifs: Implement basic fscrypto context passing

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Implement UBIFS_FLG_ENCRYPTION
Richard Weinberger [Thu, 18 Oct 2018 14:36:44 +0000 (16:36 +0200)]
mkfs.ubifs: Implement UBIFS_FLG_ENCRYPTION

...and set UBIFS format version

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Add basic fscrypto functions
Richard Weinberger [Thu, 18 Oct 2018 14:36:43 +0000 (16:36 +0200)]
mkfs.ubifs: Add basic fscrypto functions

...maybe we should add them to crypto.c?

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Add fscrypto defines
Richard Weinberger [Thu, 18 Oct 2018 14:36:42 +0000 (16:36 +0200)]
mkfs.ubifs: Add fscrypto defines

most of them should be UAPI, therefore check using #ifndef

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Make r5 hash binary string aware
Richard Weinberger [Thu, 18 Oct 2018 14:36:41 +0000 (16:36 +0200)]
mkfs.ubifs: Make r5 hash binary string aware

As of now all filenames known by UBIFS are strings with a NUL
terminator. With encrypted filenames a filename can be any binary
string and the r5 function cannot search for the NUL terminator.
UBIFS always knows how long a filename is, therefore we can change
the hash function to iterate over the filename length to work
correctly with binary strings.

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Implement UBIFS_FLG_DOUBLE_HASH
David Oberhollenzer [Thu, 18 Oct 2018 14:36:40 +0000 (16:36 +0200)]
mkfs.ubifs: Implement UBIFS_FLG_DOUBLE_HASH

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agomkfs.ubifs: Add crypto helper functions
David Oberhollenzer [Thu, 18 Oct 2018 14:36:39 +0000 (16:36 +0200)]
mkfs.ubifs: Add crypto helper functions

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
10 months agocommon: Add round functions
Richard Weinberger [Thu, 18 Oct 2018 14:36:38 +0000 (16:36 +0200)]
common: Add round functions

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
10 months agoImport latest ubifs-media.h
Richard Weinberger [Thu, 18 Oct 2018 14:36:37 +0000 (16:36 +0200)]
Import latest ubifs-media.h

Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
11 months agomkfs.ubifs: Fix xattr nlink value
Sascha Hauer [Thu, 27 Sep 2018 09:36:43 +0000 (11:36 +0200)]
mkfs.ubifs: Fix xattr nlink value

The nlink value of a xattr node must always be one, it has nothing to do
with the nlink value of the inode the attribute belongs to.

The bug can be observed when mounting a mkfs.ubifs generated image which
contains xattrs on directories or files with hardlinks to them. When
mounting such an image with chk_fs = 1 it fails with:

UBIFS error (ubi0:0 pid 1833): dbg_check_filesystem: inode 3308 nlink is 3, but calculated nlink is 1

Another bug that can be triggered is an assertion in
ubifs_xattr_remove() which assures that the xattr i_nlink count should
be one when the xattr is removed.

Fixes: 50044ef ("mkfs.ubifs: Add extended attribute support")

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
11 months agomtd-utils: Instead of doing preprocessor magic, just output off_t as long long
Thorsten Glaser [Thu, 20 Sep 2018 23:22:11 +0000 (01:22 +0200)]
mtd-utils: Instead of doing preprocessor magic, just output off_t as long long

Fix warnings abot PRIdoff_t in libmtd.c, in mtd_read (and mtd_write):

In file included from ../git/lib/libmtd.c:40:0:
../git/lib/libmtd.c: In function 'mtd_read':
../git/include/common.h:110:18: warning: format '%ld' expects argument of
 type 'long int', but argument 5 has type 'off_t {aka long long int}'
 [-Wformat=]

../git/include/common.h:120:2: note: in expansion of macro 'errmsg'
  errmsg(fmt, ##__VA_ARGS__);                                         \
  ^~~~~~
../git/lib/libmtd.c:1082:10: note: in expansion of macro 'sys_errmsg'
   return sys_errmsg("cannot seek mtd%d to offset %"PRIdoff_t,
          ^~~~~~~~~~

/usr/lib/klibc/include/inttypes.h:28:17: note: format string is defined here
 #define PRId32 "d"

Signed-off-by: Thorsten Glaser <tg@mirbsd.org>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
11 months agomtd-utils: common.h: fix prompt function
David Oberhollenzer [Wed, 19 Sep 2018 08:57:34 +0000 (10:57 +0200)]
mtd-utils: common.h: fix prompt function

The prompt() function is intended to query a yes/no reply from a command
line user by reading in an entire line of text using getline() and checking
the first character. If the line is empty, a default value is returned.

First of all, this patch replaces the usage of getline() with fgets() to
avoid compilation problems on some smaller C libraries, like klibc, that
do not have a getline() implementation. Since we now have a static line
length, this may break some build setups that input lengthy giberish
instead of 'y' or 'n'.

Second, this patch fixes a more severe bug in prompt(), replacing a 'while'
keyword with the 'if' that was most likely intended. In the old version, if
getline() reported an error, it would print an error message inside a while
loop, immediately followed by a break and then march on and process the
erroneous input instead of using the default value as printed to stdout.

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
11 months agolibmtd: don't print an error message for devices without ecc support
David Oberhollenzer [Wed, 12 Sep 2018 07:40:34 +0000 (09:40 +0200)]
libmtd: don't print an error message for devices without ecc support

The libmtd library tries to obtain the available OOB size via the sysfs
with a fallback to the ECCGETLAYOUT ioctl. For some devices (e.g. plat-ram),
the fallback path is always taken and prints an error message to stderr
since the ioctl fails.

This patch fixes the fallback path by suppressing the error message if
errno is set to EOPNOTSUPP (i.e. the device simply doesn't support that).

Fixes: a10353584f93 ("libmtd: Add support to access OOB available size")
Reported-by: Chris Packham <Chris.Packham@alliedtelesis.co.nz>
Reviewed-by: Xiaolei Li <xiaolei.li@mediatek.com>
Tested-by: Chris Packham <Chris.Packham@alliedtelesis.co.nz>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
12 months agoubi-tests: io_paral: Fix error handling of update_volume()
Martin Lund [Wed, 5 Sep 2018 13:47:38 +0000 (15:47 +0200)]
ubi-tests: io_paral: Fix error handling of update_volume()

The io_paral test returns success even in case it throws e.g. the
following error message:

[io_paral] update_volume():125: written and read data are different

This patch fixes so that the io_paral application returns a non-zero
error code when an error is detected.

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
13 months agoubi-utils: ubimkvol: Initialize req to zero to make sure no flags are set by default
Boris Brezillon [Thu, 26 Jul 2018 14:05:29 +0000 (16:05 +0200)]
ubi-utils: ubimkvol: Initialize req to zero to make sure no flags are set by default

The program expects req.flags to be zero-initialized, but it's not
the case. Let's explicitly initialize req to zero at declaration time.

Fixes: 7b4a65a27d26 ("ubi-utils: ubimkvol: add support for skipping CRC check of a static volume when opening")
Reviewed-by: Quentin Schulz <quentin.schulz@bootlin.com>
Signed-off-by: Boris Brezillon <boris.brezillon@bootlin.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
14 months agoubi-utils: ubinize: add support for skipping CRC check of a static volume when opening
Quentin Schulz [Thu, 28 Jun 2018 07:43:44 +0000 (09:43 +0200)]
ubi-utils: ubinize: add support for skipping CRC check of a static volume when opening

Let's let the user configure static UBI volume with CRC checking at
opening disabled if desired.

Introduce the skip-check setting for vol_flags configuration of a
volume.

There is no point in having both autoresize and skip-check set as
skip-check is reserved for static volumes only and it's useless to have
a static volume's size set to autoresize.

Suggested-by: Boris Brezillon <boris.brezillon@bootlin.com>
Reviewed-by: Boris Brezillon <boris.brezillon@bootlin.com>
Signed-off-by: Quentin Schulz <quentin.schulz@bootlin.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
14 months agoubi-utils: ubimkvol: add support for skipping CRC check of a static volume when opening
Quentin Schulz [Thu, 28 Jun 2018 07:43:43 +0000 (09:43 +0200)]
ubi-utils: ubimkvol: add support for skipping CRC check of a static volume when opening

Let's let the user create static UBI volume with CRC checking at opening
disabled if desired.

Introduce the `--skipcheck` or `-k` option for such feature.

Suggested-by: Boris Brezillon <boris.brezillon@bootlin.com>
Reviewed-by: Boris Brezillon <boris.brezillon@bootlin.com>
Signed-off-by: Quentin Schulz <quentin.schulz@bootlin.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
14 months agolibubi: add volume flags to ubi_mkvol_request
Quentin Schulz [Thu, 28 Jun 2018 07:43:42 +0000 (09:43 +0200)]
libubi: add volume flags to ubi_mkvol_request

Now that we have per-UBI volume flags (for instance for skipping CRC
check when opening it) from the Linux header, let's add it to the
ubi_mkvol_request in libubi and assign the flags to ubi_mkvol_req from
the Linux header from ubi_mkvol.

Suggested-by: Boris Brezillon <boris.brezillon@bootlin.com>
Reviewed-by: Boris Brezillon <boris.brezillon@bootlin.com>
Signed-off-by: Quentin Schulz <quentin.schulz@bootlin.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
14 months agoUBI: update ubi-user.h and ubi-media.h
Quentin Schulz [Thu, 28 Jun 2018 07:43:41 +0000 (09:43 +0200)]
UBI: update ubi-user.h and ubi-media.h

Update both header files to add support for flag specifying whether to
skip the CRC check for static UBI volumes.

Taken from the kernel headers.

Some users of static UBI volumes implement their own integrity check,
thus making the volume CRC check done at open time useless. For
instance, this is the case when one use the ubiblock + dm-verity +
squashfs combination, where dm-verity already checks integrity of the
block device but this time at the block granularity instead of verifying
the whole volume.

Skipping this test drastically improves the boot-time.

Suggested-by: Boris Brezillon <boris.brezillon@bootlin.com>
Reviewed-by: Boris Brezillon <boris.brezillon@bootlin.com>
Signed-off-by: Quentin Schulz <quentin.schulz@bootlin.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
14 months agoAdd configure switch to disable compiling/installing lsmtd
David Oberhollenzer [Mon, 18 Jun 2018 09:35:59 +0000 (11:35 +0200)]
Add configure switch to disable compiling/installing lsmtd

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
14 months agoAdd manpage for lsmtd
David Oberhollenzer [Wed, 19 Apr 2017 13:45:15 +0000 (15:45 +0200)]
Add manpage for lsmtd

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
14 months agoAdd lsmtd program
David Oberhollenzer [Wed, 15 Mar 2017 11:12:12 +0000 (12:12 +0100)]
Add lsmtd program

This patch adds a program called "lsmtd". The program produces a pretty
printed list of the hierarchy of UBI and MTD devices on a system. It
tries to imitate the lsblk program from util-linux as closely as
possible.

A number of command line switches are available to fine tune what information
should be exposed and in what output format.

The goal is to have a simple way of displaying the complete MTD stack on
a system in a human readable form instead of piecing details together
from proc files and various UBI utilities.

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
15 months agomkfs.ubifs: Implement selinux labelling support in mkfs.ubifs.
Ritesh Harjani [Wed, 6 Jun 2018 09:39:00 +0000 (15:09 +0530)]
mkfs.ubifs: Implement selinux labelling support in mkfs.ubifs.

This implements/adds selinux labelling support to mkfs.ubifs
utility. It adds an extra option in configure to enable
selinux labelling support and then finally in mkfs.ubifs adds
an extra option to pass the file_contexts which is looked up
for filesystem file labels.

- Default behavior is kept without selinux so as to not break existing
support where selinux library/headers may not be present.

- If this is configured with --with-selinux then XATTR from the
file_contexts(passed with --selinux option while mkfs.ubifs)
will be taken and not from the host file's xattr.
This is done to avoid the problem where the host OS may have
selinux enabled and hence same xattr names will be present in both
host filesystem files and from the --selinux=file passed.
So the existing behavior is kept mutually exclusive and preference
is given to selinux xattrs (if configured with --with-selinux).

Signed-off-by: Ritesh Harjani <riteshh@codeaurora.org>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
15 months agomkfs.ubifs: add_xattr is not depending on host XATTR support
Ritesh Harjani [Wed, 6 Jun 2018 09:38:59 +0000 (15:08 +0530)]
mkfs.ubifs: add_xattr is not depending on host XATTR support

add_xattr adds the xattr to the ubifs image and has nothing
to do with host XATTR support.

Now that we are adding support where selinux interfaces
may use this API even when host OS(where ubi/ubifs image
is being created) does not support XATTR -so remove it
from WITHOUT_XATTR #ifdef.

Signed-off-by: Ritesh Harjani <riteshh@codeaurora.org>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
15 months agoRevert "Return correct error number in ubi_get_vol_info1"
David Oberhollenzer [Sat, 9 Jun 2018 14:45:22 +0000 (16:45 +0200)]
Revert "Return correct error number in ubi_get_vol_info1"

This reverts commit dede98ffb706676309488d7cc660f569548d5930.

The original commit tried to fix a descrepancy between the implementation
and the documentation by making the implementation comply.

When making the change, it was overlooked, that ubinfo and ubirename were
written against the implementation instead of the behaviour specified by
the documentation. So were further internal functions like
ubi_get_vol_info1_nm which further breaks ubirmvol.

A report with an outline of a resulting problem can be read on
the mailing list:

http://lists.infradead.org/pipermail/linux-mtd/2018-June/081562.html

From the report:

steps to reproduce: have mtd-utils 2.0.1 or 2.0.2

0. make a bunch of ubi volumes in sequential order

ubimkvol /dev/ubi0 -s 64KiB -N test1
ubimkvol /dev/ubi0 -s 64KiB -N test2
ubimkvol /dev/ubi0 -s 64KiB -N test3
ubimkvol /dev/ubi0 -s 64KiB -N test4
..

1. delete the test1 volume, making a hole in the volume table

ubirmvol /dev/ubi0 -N test1

2. try an affected tool (i.e. "ubirmvol /dev/ubi0 -N test4" )

 |root at mr24:/# ubirmvol /dev/ubi0 -N test4
 |ubirmvol: error!: cannot find UBI volume "test4"
 |         error 19 (No such device)

or "ubinfo -a"

 | root at mr24:/# ubinfo -a
 | UBI version:                    1
 | Count of UBI devices:           1
 | UBI control device major/minor: 10:59
 | Present UBI devices:            ubi0
 |
 | ubi0
 | Volumes count:                           11
 | Logical eraseblock size:                 15872 bytes, 15.5 KiB
 | Total amount of logical eraseblocks:     1952 (30982144 bytes, 29.5 MiB)
 | Amount of available logical eraseblocks: 75 (1190400 bytes, 1.1 MiB)
 | Maximum count of volumes                 92
 | Count of bad physical eraseblocks:       0
 | Count of reserved physical eraseblocks:  40
 | Current maximum erase counter value:     984
 | Minimum input/output unit size:          512 bytes
 | Character device major/minor:            251:0
 | ubinfo: error!: libubi failed to probe volume 5 on ubi0
 |        error 19 (No such device)
 | Present volumes:                         0, 1, 2, 3, 4root at mr24:/#

Reported-by: Christian Lamparter <chunkeey@gmail.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
16 months agomtd-utils: make sure pkg-config is installed in configure script
David Oberhollenzer [Sat, 12 May 2018 21:28:13 +0000 (23:28 +0200)]
mtd-utils: make sure pkg-config is installed in configure script

This patch adds a check to configure.ac that tests if pkg-config
is available on the system.

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
16 months agoubiformat: remove no-volume-table option
David Oberhollenzer [Sat, 12 May 2018 20:17:16 +0000 (22:17 +0200)]
ubiformat: remove no-volume-table option

Using the -n or --no-volume-table flags, ubiformat can format an mtd device
to a broken UBI that does not attach on recent kernel. Only very old UBIs
had no volume table.

This patch removes the option entirely from ubiformat.

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
16 months agoubiformat: process command line arguments first
David Oberhollenzer [Sat, 12 May 2018 19:51:56 +0000 (21:51 +0200)]
ubiformat: process command line arguments first

If libmtd_open fails, the program always exists with failure status
and prints "MTD subsystem is not present".

Even `ubiformat --help` produces the same result, which is definitely
undesired.

This patch moves command line option processing first to get the desired
behavior.

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
17 months agoRelease mtd-utils-2.0.2 v2.0.2
David Oberhollenzer [Mon, 16 Apr 2018 15:54:33 +0000 (17:54 +0200)]
Release mtd-utils-2.0.2

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
17 months agoFix unit-test header and file paths for out of tree builds
David Oberhollenzer [Mon, 16 Apr 2018 15:41:52 +0000 (17:41 +0200)]
Fix unit-test header and file paths for out of tree builds

If we build mtd-utils outside the source path, we cannot use relative
paths to refere to headers in the source tree. We have to specify
absoulte paths using the top_srcdir variable.

This was done right for the utility binaries, but overlooked for the
unit test porgrams.

This patch fixes the header paths and SYSROOT variable for the unit
tests, so they build and run propperly outside the source tree.

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
17 months agoFix unit test mockup for oobavail sysfs file
David Oberhollenzer [Mon, 16 Apr 2018 15:37:00 +0000 (17:37 +0200)]
Fix unit test mockup for oobavail sysfs file

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
17 months agomisc-utils: flash_erase: Fix Jffs2 type flash erase problem
Xiaolei Li [Tue, 10 Apr 2018 07:36:57 +0000 (15:36 +0800)]
misc-utils: flash_erase: Fix Jffs2 type flash erase problem

Currently, Jffs2 clean marker is not written actually, because the oob
write length is set to 0 when do mtd_write().

So, get OOB available size at first, and set the correct clean marker
length, then program clean marker to free OOB area.

Fixes: d7e86124d55b ("mtd-utils: Support jffs2 flash-erase for large OOB (>32b)")
Signed-off-by: Xiaolei Li <xiaolei.li@mediatek.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
17 months agolibmtd_legacy: Fix some function description mismatches
Xiaolei Li [Tue, 10 Apr 2018 07:36:56 +0000 (15:36 +0800)]
libmtd_legacy: Fix some function description mismatches

Signed-off-by: Xiaolei Li <xiaolei.li@mediatek.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
17 months agolibmtd: Add support to access OOB available size
Xiaolei Li [Tue, 10 Apr 2018 07:36:55 +0000 (15:36 +0800)]
libmtd: Add support to access OOB available size

This patch exposes OOB available size to user. Then user can use
OOB free area according to OOB available size.

Steps to get OOB available size:
First, access /sys/class/mtd/mtdX/oobavail. If not exist, then
try to get ecc layout by ioctl "ECCGETLAYOUT". If none of them
work, set OOB available size to 0.

Signed-off-by: Xiaolei Li <xiaolei.li@mediatek.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
17 months agomtd-utils: ubifs: fix typo in without_lzo definition
Koen Vandeputte [Thu, 5 Apr 2018 07:51:43 +0000 (09:51 +0200)]
mtd-utils: ubifs: fix typo in without_lzo definition

Fixes: 1d04b4d5361a ("fix build when WITHOUT_LZO is set")

Signed-off-by: Koen Vandeputte <koen.vandeputte@ncentric.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
18 months agomkfs.ubifs: Allow root entry in device table
David Engraf [Tue, 6 Mar 2018 08:13:42 +0000 (09:13 +0100)]
mkfs.ubifs: Allow root entry in device table

When using a local root folder the permissions, user and group settings
are taken from the local folder. These permissions might be incorrect if
the folder has been created for the local user. Creating an UBIFS image
on my local system resulted in the following output on the target:

drwx------   17 1000     1000   1264 Jan  1 00:00 .
drwx------   17 1000     1000   1264 Jan  1 00:00 ..
drwxr-xr-x    2 root     root   9104 May 30  2017 bin
drwxr-xr-x    7 root     root   2760 Jan  1 00:00 dev
...

mkfs.ubifs aborts with an error message when the device table contains
a root entry. This patch allows setting the root folder permissions,
user and group to overwrite local configurations.

Signed-off-by: David Engraf <david.engraf@sysgo.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
18 months agomtd: tests: check erase block count in page test
Stefan Agner [Sat, 3 Mar 2018 22:39:46 +0000 (23:39 +0100)]
mtd: tests: check erase block count in page test

When there is only a single erase block, the cross erase test
does not report sensible errors. Warn in case there is only
a single erase block instead of executing the test.

Signed-off-by: Stefan Agner <stefan@agner.ch>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>