Merge branch 'carbon' into 'develop'

Conflicts:
  - tests/unit/utils/schema_test.py

CONDUCT.md

@@ -0,0 +1,74 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, gender identity and expression, level of experience,
+nationality, personal appearance, race, religion, or sexual identity and
+orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at [email protected]. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

doc/topics/releases/2016.3.0.rst

@@ -37,6 +37,8 @@ Backwards-incompatible Changes
   fileserver under ``salt://_runners``, ``salt://_output``, etc. and sync them
   using the functions in the new :mod:`saltutil runner
   <salt.runners.saltutil>`.
+- The :py:func:`pkg.check_db <salt.modules.yumpkg.check_db>` function has been
+  removed for yum/dnf.
 
 
 Core Changes

doc/topics/thorium/index.rst

@@ -38,17 +38,63 @@ section of your Salt Master or Minion configuration file and restart the daemon:
     engines:
       - thorium: {}
 
+
+Thorium Modules
+===============
+Because of its specialized nature, Thorium uses its own set of modules. However,
+many of these modules are designed to wrap the more commonly-used Salt
+subsystems. These modules are:
+
+* local: Execution modules
+* runner: Runner modules
+* wheel: Wheel modules
+
+There are other modules that ship with Thorium as well. Some of these will be
+highlighted later in this document.
+
+
 Writing Thorium Formulas
 ========================
+Like some other Salt subsystems, Thorium uses its own directory structure. The
+default location for this structure is ``/srv/thorium/``, but it can be changed
+using the ``thorium_roots_dir`` setting in the ``master`` configuration file.
+
+
+The Thorium top.sls File
+------------------------
+Thorium uses its own ``top.sls`` file, which follows the same convention as is
+found in ``/srv/salt/``:
+
+.. code-block:: yaml
 
-To start with Thorium create the Thorium state tree, the state tree for Thorium
-looks exactly like the Salt Configuration Management state tree, it is just
-located in the `thorium_roots_dir` instead of the `file_roots_dir`. The default
-location for the `thorium_roots_dir` is `/srv/thorium`.
+    <srv>:
+      <target>:
+        - <formula 1>
+        - <formula 2>
+        - <etc...>
 
-This example uses thorium to detect when a minion has disappeared and then
-deletes the key from the master when the minion has been gone for 60 seconds:
+For instance, a ``top.sls`` using a standard ``base`` environment and a single
+Thorium formula called ``key_clean``, would look like:
 
+.. code-block:: yaml
+
+    base:
+      '*':
+        - key_clean
+
+Take note that the target in a Thorium ``top.sls`` is not used; it only exists
+to follow the same convention as other ``top.sls`` files. Leave this set to
+``'*'`` in your own Thorium ``top.sls``.
+
+Thorium Formula Files
+---------------------
+Thorium SLS files are processed by the same state compiler that processes Salt
+state files. This means that features like requisites, templates, and so on are
+available.
+
+Let's take a look at an example, and then discuss each component of it. This
+formula uses Thorium to detect when a minion has disappeared and then deletes
+the key from the master when the minion has been gone for 60 seconds:
 
 .. code-block:: yaml
 
@@ -57,27 +103,183 @@ deletes the key from the master when the minion has been gone for 60 seconds:
 
     keydel:
       key.timeout:
+        - delete: 60
         - require:
           - status: statreg
-        - delete: 60
 
-Remember to set up a top file so Thorium knows which sls files to use!!
+There are two stanzas in this formula, whose IDs are ``startreg`` and
+``keydel``. The first stanza, ``startreg``, tells Thorium to keep track of
+minion status beacons in its *register*. We'll talk more about the register in
+a moment.
 
-.. code-block:: yaml
+The second stanza, ``keydel``, is the one that does the real work. It uses the
+``key`` module to apply an expiration (using the ``timeout`` function) to a
+minion. Because ``delete`` is set to ``60``, this is a 60 second expiration. If
+a minion does not check in at least once every 60 seconds, its key will be
+deleted from the master. This particular function also allows you to use
+``reject`` instead of ``delete``, allowing for a minion to be rejected instead
+of deleted if it does not check in within the specified time period.
 
-    base:
-      '*':
-        - key_clean
+There is also a ``require`` requisite in this stanza. It states that the
+``key.timeout`` function will not be called unless the ``status.reg`` function
+in the ``startreg`` codeblock has been successfully called first.
 
-Thorium Links to Beacons
-========================
 
+Thorium Links to Beacons
+------------------------
 The above example was added in the Carbon release of Salt and makes use of the
-`status` beacon also added in the Carbon release. For the above Thorium state
-to function properly you will also need to enable the `status` beacon:
+``status`` beacon also added in the Carbon release. For the above Thorium state
+to function properly you will also need to enable the ``status`` beacon in the
+``minion`` configuration file:
 
 .. code-block:: yaml
 
     beacons:
       status:
         interval: 10
+
+This will cause the minion to use the status beacon to check in with the master
+every 10 seconds.
+
+
+The Thorium Register
+====================
+In order to keep track of information, Thorium uses an in-memory register (or
+rather, collection of registers) on the master. These registers are only
+populated when told to by a formula, and they normally will be erased when the
+master is restarted. It is possible to persist the registers to disk, but we'll
+get to that in a moment.
+
+The example above uses ``status.reg`` to populate a register for you, which is
+automatically used by the ``key.timeout`` function. However, you can set your
+own register values as well, using the ``reg`` module.
+
+Because Thorium watches the event bus, the ``reg`` module is designed to look
+for user-specified tags, and then extract data from the payload of events that
+match those tags. For instance, the following stanza will look for an event
+with a tag of ``my/custom/event``:
+
+.. code-block:: yaml
+
+    foo:
+      reg.list:
+        - add: bar
+        - match: my/custom/event
+
+When such an event is found, the data found in the payload dictionary key of
+``bar`` will be stored in a register called ``foo``. This register will store
+that data in a ``list``. You may also use ``reg.set`` to add data to a ``set()``
+instead.
+
+If you would like to see a copy of the register as it is stored in memory, you
+can use the ``file.save`` function:
+
+.. code-block:: yaml
+
+    myreg:
+      file.save
+
+In this case, each time the register is updated, a copy will be saved in JSON
+format at ``/var/cache/salt/master/thorium/saves/myreg``. If you would like to
+see when particular events are added to a list-type register, you may add a
+``stamp`` option to ``reg.list`` (but not ``reg.set``). With the above two
+stanzas put together, this would look like:
+
+.. code-block:: yaml
+
+    foo:
+      reg.list:
+        - add: bar
+        - match: my/custom/event
+        - stamp: True
+
+    myreg:
+      file.save
+
+If you would like to only keep a certain number of the most recent register
+entries, you may also add a ``prune`` option to ``reg.list`` (but not
+``reg.set``):
+
+.. code-block:: yaml
+
+    foo:
+      reg.list:
+        - add: bar
+        - match: my/custom/event
+        - stamp: True
+        - prune: 50
+
+This example will only keep the 50 most recent entries in the ``foo`` register.
+
+Using Register Data
+-------------------
+Putting data in a register is useless if you don't do anything with it. The
+``check`` module is designed to examine register data and determine whether it
+matches the given parameters. For instance, the ``check.contains`` function
+will return ``True`` if the given ``value`` is contained in the specified
+register:
+
+.. code-block:: yaml
+
+    foo:
+      reg.list:
+        - add: bar
+        - match: my/custom/event
+        - stamp: True
+        - prune: 50
+      check.contains:
+        - value: somedata
+
+Used with a ``require`` requisite, we can call one of the wrapper modules and
+perform an operation. For example:
+
+.. code-block:: yaml
+
+    shell_test:
+      local.cmd:
+        - tgt: dufresne
+        - func: cmd.run
+        - arg:
+          - echo 'thorium success' > /tmp/thorium.txt
+        - require:
+          - check: foo
+
+This stanza will only run if the ``check.contains`` function under the ``foo``
+ID returns true (meaning the match was found).
+
+There are a number of other functions in the ``check`` module which use
+different means of comparing values:
+
+* ``gt``: Check whether the register entry is greater than the given value
+* ``gte``: Check whether the register entry is greater than or equal to the given value
+* ``lt``: Check whether the register entry is less than the given value
+* ``lte``: Check whether the register entry is less than or equal to the given value
+* ``eq``: Check whether the register entry is equal to the given value
+* ``ne``: Check whether the register entry is not equal to the given value
+
+There is also a function called ``check.event`` which does not examine the
+register. Instead, it looks directly at an event as it is coming in on the
+event bus, and returns ``True`` if that event's tag matches. For example:
+
+.. code-block:: yaml
+
+        salt/foo/*/bar:
+          check.event
+
+        run_remote_ex:
+          local.cmd:
+            - tgt: '*'
+            - func: test.ping
+            - require:
+              - check: salt/foo/*/bar
+
+This formula will look for an event whose tag is ``salt/foo/<anything>/bar`` and
+if it comes in, issue a ``test.ping`` to all minions.
+
+
+Register Persistence
+--------------------
+It is possible to persist the register data to disk when a master is stopped
+gracefully, and reload it from disk when the master starts up again. This
+functionality is provided by the returner subsystem, and is enabled whenever
+any returner containing a ``load_reg`` and a ``save_reg`` function is used.

salt/cloud/clouds/digital_ocean.py

@@ -507,7 +507,6 @@ def __query_node_data(vm_name):
     log.debug('Found public IP address to use for ssh minion bootstrapping: {0}'.format(vm_['ssh_host']))
 
     vm_['key_filename'] = key_filename
-    vm_['ssh_host'] = ip_address
     ret = __utils__['cloud.bootstrap'](vm_, __opts__)
     ret.update(data)
 

salt/modules/aptpkg.py

@@ -404,9 +404,9 @@ def refresh_db(cache_valid_time=0):
             # Strip filesize from end of line
             ident = re.sub(r' \[.+B\]$', '', ident)
             ret[ident] = True
-        elif cols[0] == 'Ign':
+        elif 'Ign' in cols[0]:
             ret[ident] = False
-        elif cols[0] == 'Hit':
+        elif 'Hit' in cols[0]:
             ret[ident] = None
     return ret
 

salt/modules/daemontools.py

@@ -31,6 +31,8 @@
 
 log = logging.getLogger(__name__)
 
+__virtualname__ = 'service'
+
 VALID_SERVICE_DIRS = [
     '/service',
     '/var/service',
@@ -46,7 +48,7 @@
 def __virtual__():
     # Ensure that daemontools is installed properly.
     BINS = frozenset(('svc', 'supervise', 'svok'))
-    return all(salt.utils.which(b) for b in BINS)
+    return __virtualname__ if all(salt.utils.which(b) for b in BINS) else False
 
 
 def _service_path(name):

salt/modules/glance.py

@@ -162,7 +162,6 @@ def get(key, default=None):
         log.debug('Calling keystoneclient.v2_0.client.Client(' +
             '{0}, **{1})'.format(ks_endpoint, kwargs))
         keystone = kstone.Client(**kwargs)
-        log.debug(help(keystone.get_token))
         kwargs['token'] = keystone.get_token(keystone.session)
         # This doesn't realy prevent the password to show up
         # in the minion log as keystoneclient.session is