Metadata-Version: 2.1
Name: cloudformation_validator
Version: 0.6.29
Summary: Lightweight, extensible schema and data validation tool for Cloudformation Templates.
Home-page: https://github.com/rubelw/cloudformation-validator
Author: Will Rubel
Author-email: willrubel@gmail.com
License: UNKNOWN
Description: Cloudformation-Validator
        ========================
        
        Features
        ========
        
        cloudformation\_validator provides type checking and other base
        functionality out of the box and is designed to be non-blocking and
        easily extensible, allowing for custom validation. It has no
        dependencies and is thoroughly tested under Python 2.7, Python 3.3,
        Python 3.4, Python 3.5, Python 3.6.
        
        Funding
        =======
        
        cloudformation\_validator is a open source, collaboratively funded
        project. If you run a business and are using cloudformation\_validator
        in a revenue-generating product, it would make business sense to sponsor
        its development: it ensures the project that your product relies on
        stays healthy and actively maintained. Individual users are also welcome
        to make a recurring pledge or a one time donation if cfn-validator has
        helped you in your work or personal projects.
        
        Every single sign-up makes a significant impact towards making
        cloudformation\_validator possible.
        
        Want Custom Rules and Support For Your Application
        ==================================================
        
        Submit an issue on my github page if you would like additional custom
        rules and I will try and get them added as soon as possible.
        
        I you would like other functionality, just submit an issue and I will
        see what I can do to get it added.
        
        Installation
        ============
        
        cloudformation-validator is on PyPI so all you need is:
        
        ``` {.sourceCode .console}
        $ pip install cfn-validator
        ```
        
        
        Demonstration
        =============
        
        ![Image of Yaktocat](https://octodex.github.com/images/yaktocat.png)
        
        Just run:
        
        ``` {.sourceCode .console
        $ pip install virtualenv
        $ which python
        $ virtualenv ~/virtualenvs/my_project -p /home/example_username/opt/python-3.6.2/bin/python3
        $ git clone https://github.com/rubelw/cloudformation-validator.git
        $ cd cloudformation-validator
        $ pip install -r requirements-dev.txt
        $ python setup.py install --force
        $ python setup.py test}
        ```
        
        Or you can use tox to run the tests under all supported Python versions.
        Make sure the required python versions are installed and run:
        
        ``` {.sourceCode .console
        $ pip install virtualenv
        $ which python
        $ virtualenv ~/virtualenvs/my_project -p /home/example_username/opt/python-3.6.2/bin/python3
        $ git clone https://github.com/rubelw/cloudformation-validator.git
        $ cd cloudformation-validator
        $ pip install -r requirements-dev.txt
        $ python setup.py install --force
        $ pip install tox  # first time only
        $ tox}
        ```
        
        Listing Rules
        =============
        
        ``` {.sourceCode .console}
        $ cfn-validator dump_rules
        ##################################
        ########## WARNINGS ##############
        ##################################
        {'id': 'F4', 'type': 'VIOLATION::WARNING', 'message': 'IAM policy should not allow * action'}
        {'id': 'W1', 'type': 'VIOLATION::WARNING', 'message': 'Specifying credentials in the template itself is probably not the safest thing'}
        ...
        ```
        
        Example
        =======
        
        Getting help
        
        ``` {.sourceCode .console}
        $ cfn-validator validate --help
        Usage: cfn-validator validate [OPTIONS]
        
          primary function for validating a template :param template_path: :param
          template_file: :param debug: :param rules_directory: :param profile_path:
          :param allow_suppression: :param print_suppression: :param
          parameter_values_path: :param isolate_custom_rule_exceptions: :param
          version: :return:
        
        Options:
          -s, --suppress-errors           Whether to suppress misc errors to get hash only
          -t, --template-path TEXT        base directory to search for templates
          -f, --template-file TEXT        single_template_file
          --debug                         Turn on debugging
          -r, --rules-directory TEXT      Extra rule directory
          -o, --profile-path TEXT         Path to a profile file
          --allow-suppression / --no-allow-suppression
                                          Allow using Metadata to suppress violations
          -p, --print-suppression         Emit suppressions to stderr
          -m, --parameter-values-path TEXT
                                          Path to a JSON file to pull Parameter values
                                          from
          -i, --isolate-custom-rule-exceptions
                                          Isolate custom rule exceptions - just emit
                                          the exception without stack trace and keep
                                          chugging
          -v, --version                   Print version and exit
          --help                          Show this message and exit.
        ```
        
        Validate a file
        
        ``` {.sourceCode .console}
        $cfn-validator validate -f cloudfront_distribution_without_logging.json
        
        Evaluating: cloudfront_distribution_without_logging.json
        [
            {
                'failure_count': '0',
                'filename': 'cloudfront_distribution_without_logging.json',
                'file_results': [
                    {
                        'id': 'W10',
                        'type': 'VIOLATION::WARNING',
                        'message': 'CloudFront Distribution should enable access logging',
                        'logical_resource_ids': [
                            'rDistribution2'
                        ]
                    }
                ]
            }
        ]
        ```
        
        Validate all files in a path
        
        ``` {.sourceCode .console}
        $cfn-validator validate -f /projects
        ...
        ```
        
        Programmatically call cfn-validator to analyze a file
        
        ``` {.sourceCode .console}
        from cloudformation_validator.ValidateUtility import ValidateUtility
        
        config_dict = {}
        config_dict['template_file'] = '/tmp/template.json'
        validator = ValidateUtility(config_dict)
        real_result =  validator.validate()
        print(real_result)
        
        [
            {
                'failure_count': '0',
                'filename': '/tmp/template.json',
                'file_results': [
                    {
                        'id': 'W1',
                        'type': 'VIOLATION::WARNING',
                        'message': 'Specifying credentials in the template itself is probably not the safest thing',
                        'logical_resource_ids': [
                            'EC2I4LBA1'
                        ]
                    }
                ]
            }
        ]
        ```
        
        I you get some errors and warnings in your out put, you can pass-in the
        flag to suppress all errors
        
        ``` {.sourceCode .console}
        from cloudformation_validator.ValidateUtility import ValidateUtility
        
        config_dict = {}
        config_dict['suppress_errors'] = True
        config_dict['template_file'] = '/tmp/template.json'
        validator = ValidateUtility(config_dict)
        real_result =  validator.validate()
        print(real_result)
        
        [
            {
                'failure_count': '0',
                'filename': '/tmp/template.json',
                'file_results': [
                    {
                        'id': 'W1',
                        'type': 'VIOLATION::WARNING',
                        'message': 'Specifying credentials in the template itself is probably not the safest thing',
                        'logical_resource_ids': [
                            'EC2I4LBA1'
                        ]
                    }
                ]
            }
        ]
        ```
        
        Writing your own rules
        
        > -   Utilize the format for existing rules in the
        >     /cloudformation\_validator/custom\_rules directory
        > -   Places the files in a new directory
        > -   The \_\_init\_\_, rule\_text, rule\_type and rule\_id methods
        >     should be amount the same, just change of the rule, the text for a
        >     failure, and the type to either \'VIOLATION::FAILING\_VIOLATION\'
        >     or VIOLATION::WARNNING\'
        > -   Set the id to \'W\' for warnings, and \'F\' for failure. Pick a
        >     number not utilized elsewhere\...
        > -   NOTE: Currently working on functionality for controlling and
        >     listing rules
        > -   For the audit\_impl function - portion with will test the resource
        >     objects, you will need to review the object model for the resource
        >     to see what objects are available, and then review the parser for
        >     the resource. Also, look at other similar rules for the resource,
        >     and model after them. The basic concept of the function is to
        >     identify resources which apply, iterate over the selected
        >     resources, and identify specific aspects to evaluate in the rule
        > -   pass in the \--rules-directory /directory in the command line, and
        >     the extra rules directory will get added to the existing rules
        
        ``` {.sourceCode .console}
        def audit_impl(self):
        
          violating_rules = []
        
          # This defines which type of resource we are going to test
          resources = self.cfn_model.resources_by_type('AWS::SQS::QueuePolicy')
        
          if len(resources)>0:
            for resource in resources:
                if hasattr(resource, 'policy_document'):
                  if resource.policy_document:
                    if resource.policy_document.wildcard_allowed_actions():
                      violating_rules.append(resource.logical_resource_id)
        
          return violating_rules
        ```
        
        Example of writing a rule which requires custom tags for EC2 instances
        ======================================================================
        
        -   Create a directory to store your custom rule
        -   Create the custom rule
        
        ``` {.sourceCode .console}
        mkdir ~/custom_validator_rules
        ```
        
        ``` {.sourceCode .console}
        from __future__ import absolute_import, division, print_function
        import inspect
        import sys
        from builtins import (str)
        from cloudformation_validator.custom_rules.BaseRule import BaseRule
        from collections import Iterable
        from six import StringIO, string_types
        from builtins import (str)
        
        class Ec2CustomTagsRule(BaseRule):
        
          def __init__(self, cfn_model=None, debug=None):
            '''
            Initialize Ec2HasTagsRule
            :param cfn_model:
            '''
            BaseRule.__init__(self, cfn_model, debug=debug)
        
          def rule_text(self):
            '''
            Returns rule text
            :return:
            '''
            if self.debug:
              print('rule_text')
            return 'EC2 instance does not have the required tags'
        
          def rule_type(self):
            '''
            Returns rule type
            :return:
            '''
            self.type= 'VIOLATION::FAILING_VIOLATION'
            return 'VIOLATION::FAILING_VIOLATION'
        
          def rule_id(self):
            '''
            Returns rule id
            :return:
            '''
            if self.debug:
              print('rule_id')
            self.id ='F86'
            return 'F86'
        
          def tags_to_dict(self, aws_tags):
                """ Convert a list of AWS tags into a python dict """
                return {str(tag['Key']): str(tag['Value']) for tag in self.ensure_list(aws_tags)}
        
          def ensure_list(self, value):
                """
                Coerces a variable into a list; strings will be converted to a singleton list,
                and `None` or an empty string will be converted to an empty list.
                Args:
                    value: a list, or string to be converted into a list.
        
                Returns:
                    :py:class:`list`
                """
                ret_value = value
                if not value:
                    ret_value = []
                elif not isinstance(value, Iterable) or isinstance(value, string_types):
                    ret_value = [value]
                return ret_value
        
        
          def audit_impl(self):
            '''
            Audit
            :return: violations
            '''
            if self.debug:
              print('Ec2HasTagsRule - audit_impl')
        
            violating_volumes = []
        
            resources = self.cfn_model.resources_by_type('AWS::EC2::Instance')
        
            if len(resources) > 0:
        
              for resource in resources:
                if self.debug:
                  print('resource: ' + str(resource))
                  print('vars: '+str(vars(resource)))
        
                if hasattr(resource, 'tags'):
                  tags_dict = self.tags_to_dict(resource.cfn_model['Properties']['Tags'])
                  required_tags = ('Name', 'ResourceOwner','DeployedBy','Project')
                  if not set(required_tags).issubset(tags_dict):
                    violating_volumes.append(str(resource.logical_resource_id))
                else:
                  if self.debug:
                    print('does not tags property')
                  violating_volumes.append(str(resource.logical_resource_id))
        
            else:
              if self.debug:
                print('no violating_volumes')
        
            return violating_volumes
        ```
        
        -   Test the rule by creating a cloudformation template without the
            necessary tags and testing
        
        ``` {.sourceCode .console}
        {
          "Parameters": {
            "subnetId": {
              "Type": "String",
              "Default": "subnet-4fd01116"
            }
          },
        
          "Resources": {
            "EC2I4LBA1": {
              "Type": "AWS::EC2::Instance",
              "Properties": {
                "ImageId": "ami-6df1e514",
                "InstanceType": "t2.micro",
                "SubnetId": {
                  "Ref": "subnetId"
                }
              },
              "Metadata": {
                "AWS::CloudFormation::Authentication": {
                  "testBasic" : {
                    "type" : "basic",
                    "username" : "biff",
                    "password" : "badpassword",
                    "uris" : [ "http://www.example.com/test" ]
                  }
                }
              }
            }
          }
        }
        ```
        
        -   Run the test
        
        ``` {.sourceCode .console
        cfn-validator validate --template-file=/tmp/template.json --rules-directory=/home/user/custom_validator_rules}
        ```
        
        -   You should receive the following violations
        
        ``` {.sourceCode .console}
        {
            'failure_count': '1',
            'filename': '/tmp/template.json',
            'file_results': [
                {
                    'id': 'F86',
                    'type': 'VIOLATION::FAILING_VIOLATION',
                    'message': 'EC2 instance does not have the required tags',
                    'logical_resource_ids': [
                        'EC2I4LBA1'
                    ]
                },
                {
                    'id': 'W1',
                    'type': 'VIOLATION::WARNING',
                    'message': 'Specifying credentials in the template itself is probably not the safest thing',
                    'logical_resource_ids': [
                        'EC2I4LBA1'
                    ]
                }
            ]
        }
        ```
        
        -   No add tags property to the cloudformation template and run again
        
        ``` {.sourceCode .console
        {
        "Parameters": {
        "subnetId": {
        "Type": "String",
        "Default": "subnet-4fd01116"
        }
        },}
        "Resources": {
          "EC2I4LBA1": {
            "Type": "AWS::EC2::Instance",
            "Properties": {
              "ImageId": "ami-6df1e514",
              "InstanceType": "t2.micro",
              "SubnetId": {
                "Ref": "subnetId"
              },
              "Tags" : [
                {"Key" : "Name", "Value":"value"},
                {"Key":"ResourceOwner","Value":"resourceowner"},
                {"Key":"DeployedBy","Value":"deployedby"},
                {"Key":"Project","Value":"project"}
              ]
            },
            "Metadata": {
              "AWS::CloudFormation::Authentication": {
                "testBasic" : {
                  "type" : "basic",
                  "username" : "biff",
                  "password" : "badpassword",
                  "uris" : [ "http://www.example.com/test" ]
                }
              }
            }
          }
        }
        ```
        
        > }
        
        -   You should receive the following violations
        
        ``` {.sourceCode .console
        {
        'failure_count': '0',
        'filename': '/tmp/template.json',
        'file_results': [
        {
        'id': 'W1',
        'type': 'VIOLATION::WARNING',
        'message': 'Specifying credentials in the template itself is probably not the safest thing',
        'logical_resource_ids': [
        'EC2I4LBA1'
        ]
        }
        ]
        }}
        ```
        
        Unit Testing
        ============
        
        Run unit tests
        
        ``` {.sourceCode .console}
        (python3) => pytest
        ================================================ test session starts =================================================
        collected 22 items
        
        test/test_cloudfront_distribution.py .                                                                         [  4%]
        test/test_ec2_instance.py .                                                                                    [  9%]
        test/test_ec2_volume.py ..                                                                                     [ 18%]
        test/test_elasticloadbalancing_loadbalancer.py .                                                               [ 22%]
        test/test_iam_user.py .                                                                                        [ 27%]
        test/test_lambda_permission.py .                                                                               [ 31%]
        test/test_rds_instance.py ...                                                                                  [ 45%]
        test/test_s3_bucket.py .                                                                                       [ 50%]
        test/test_s3_bucket_policy.py .                                                                                [ 54%]
        test/test_security_group.py ........                                                                           [ 90%]
        test/test_sns_policy.py .                                                                                      [ 95%]
        test/test_sqs_policy.py .                                                                                      [100%]
        ```
        
        Source
        ======
        
        I am just getting started on this, so any suggestions would be welcome.
        \<<https://github.com/rubelw/cloudformation-validator>\>
        
        Copyright
        =========
        
        cloudformation\_validator is an open source project by Will Rubel
        \<<https://www.linkedin.com/in/will-rubel-03205b2a/>\>, that was ported
        from a ruby project by Stelligent. See the original LICENSE information
        \<<https://github.com/stelligent/cfn_nag/blob/master/LICENSE.md>\>.
        
Keywords: validation,schema,dictionaries,aws,cloudformation,python,rules,linter
Platform: any
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Natural Language :: English
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.6
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.3
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Description-Content-Type: text/markdown
