restoreconfig: fix skipping of targets [re]loading

Problem:
'targetcli restoreconfig [savefile] [target=...]' works for first target only
in the saveconfig.json and fails/skips for others silently.

Solution:
Just an indentation fix, yet very severe.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

    should fix in most situations (Closes: #858533).

  * Also remove the doc package, because epydoc is going away.

[rtslib] restoreconfig: add ability to restore/reload single target or storage_object

Problem:
-------
Right now, if any target or storage_object is failed to load as part of
target service bringup or manual restoreconfig, may be because of an issue from
the backend, then there is no way to reload that single storage_object or target,
we need to reload/restart all the targets/storage_objects present in the node
just to load one single target/storage_object, this interrupts on going
I/O (via this path) for all the volumes hosted within the node.

Solution:
--------
Add a capability to reload single target or storage_object without touching other
targets or storage_objects hosted in the node.

usage:

$ targetcli
targetcli shell version 2.1.fb49
Copyright 2011-2013 by Datera, Inc and others
For help on commands, type 'help'.

/> help
[...]

AVAILABLE COMMANDS
  The following commands are available in the current path:

[...]
    - restoreconfig [savefile] [clear_existing] [target] [storage_object]
[...]

Ex 1. reload single storage_object
$ targetcli restoreconfig /etc/target/saveconfig.json clear_existing \
    storage_object=blockx

Ex 2. reload single target
$ targetcli restoreconfig /etc/target/saveconfig.json clear_existing \
    target=iqn.2016-12.org.gluster-block:xxx

Ex 3. reload a given target and storage_object
$ targetcli restoreconfig /etc/target/saveconfig.json clear_existing \
    target=iqn.2016-12.org.gluster-block:xxx storage_object=blockx

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rtslib: fix __version__

Before:
$ python -c "import rtslib_fb; print(rtslib_fb.__version__)"
GIT_VERSION

After:
$ python -c "import rtslib_fb; print(rtslib_fb.__version__)"
2.1.69

Also rectified few other public exported variables

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

saveconfig: add hw_block_size support in control string

$ targetcli /backstores/user:glfs create name=block size=1048576 \
  cfgstring=hosting-volume@192.168.195.164/block-store/d85244f7-88e9-4b28-922d-373fa9f59f24 \
  control=max_data_area_mb=64,hw_block_size=1024 wwn=d85244f7-88e9-4b28-922d-373fa9f59f24

$ targetcli / saveconfig

$ cat /etc/target/saveconfig.json
{
  "storage_objects": [
    {
      "alua_tpgs": [],
      "attributes": {
        ...
      },
      "config": "glfs/hosting-volume@192.168.195.164/block-store/d85244f7-88e9-4b28-922d-373fa9f59f24",
      "control": "max_data_area_mb=64,hw_block_size=1024",
      "hw_max_sectors": 128,
      "name": "block",
      "plugin": "user",
      "size": 1048576,
      "wwn": "d85244f7-88e9-4b28-922d-373fa9f59f24"
    }
  ],
  "targets": []
}

Signed-off-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

remove extra semicolons in _get_saveconf

Signed-off-by: tang.wenji <tang.wenji@zte.com.cn>

save_to_file() function breaks symbolic link when saving configuration

Handle write-only attributes.

Extend use of the new 'readable' optional param
for the Node class into the Group class, so they
also return writable *and* readable entries.

Did NOT update the calls to list_attributes() and
list_parameters() in NodeACLGroup class, since
we don't need ACLs to be readable to write them
out.

Commit 03c8c159 added handling of write-only
attributes, but we could also see write-only parameters,
since they are also from sysfs. To be safe, ensure
the parameter list returned is writable in addition
to being readable.

Currently the location of the targetcli configuration file is not
configurable (saveconfig.json). The location is : /etc/target
Some users use stateless VMs, with just a persistent device
mounted under /var/lib/diskless.

In order to circumvent the non configurable location of
saveconfig.json, they create a symbolic link in /etc/target
pointing to /var/lib/diskless

Unfortunately, the rtslib save_to_file() function breaks
this symbolic link and creates a new regular file in place of it,
due to the creation of a saveconfig.json.temp file which is
then renamed to saveconfig.json using the os.rename() function.

This patch replace os.rename() with shutil.copyfile() + os.remove(),
using copyfile() instead of rename() will preserve the symbolic link

Signed-off-by: Maurizio Lombardi <mlombard@redhat.com>

A recent kernel change (see commit 6baca7601bdee2e5) makes
the pi_prot_format protection information attribute write-only,
since it always returned 0 when being read. This commit is being
reverted, but it still brought up the prospect of write-only
attributes.

Currently, when doing a dump(), rtslib iterates through all
readable attributes to decide which ones to save as part of
our current state, but saving write-only attributes
makes no sense, since we cannot read them to capture their
value. Towards this end, enhande the _list_files() internal
method to allow filtering on whether the file is writable
as well as whether or not it is readable. Then modify
list_attributes() to allow this new parameter and modify
dump() to request only R/W attributes.

Allow bs_cache to be bypassed

The ceph tools might have multiple daemons making and querying the system.
To avoid invalidating the cache for every creation and lookup this allows
tools like ours to avoid the cache.

If the caller passed in the index during creations or lookups then use
that value instead of building/quering the cache.

Signed-off-by: Mike Christie <mchristi@redhat.com>

report the correct size for partitions

If you use a partition instead of the whole device as the backstore device,
rtslib will always report its size as 0 bytes.

This happens because rtslib tries to read the "queue/logical_block_size"
attribute, which can only be found in the parent's node.
This patch fixes the bug by loading the parent node's attributes if
it detects that the device is a partition.

Signed-off-by: Maurizio Lombardi <mlombard@redhat.com>