We’re just getting started. Check back here or on the SaltConf19 mobile app to see new talks and topics as they’re announced.
VP of Engineering, SaltStack
November 18 – 19
- Infrastructure Automation
- Security and Compliance
- Tips, Tricks & Best Practices
- Advanced Automation
- Integrate and Extend
- Salt on Salt
Learn how Innovative IT and Network teams are harnessing SaltStack to bring their infrastructure into the digital age.
Brad Chapin, Lead Architect
Matt Flynn, Lead Network Engineer
Erik Johnson, Lead System Engineer
Using Environments to Manage Safe, Large-scale Deployments
The CDN team within CenturyLink supports ~20K customer-facing servers, worldwide. Our production network is divided into three main segments (with many smaller roles and variations) Trial – 5%, Network Readiness Testing – 20%, and General Availability – 75%. We support both software deployment and automated repair, so any automated repair must rebuild the server exactly as before the failure. Each customer has control over a very flexible set of configurations, so corner cases are difficult to identify until the code and configurations have been rolled to production and we identify servers operating in a sub-standard way. We are using the formula model alongside environments within Salt to deploy multiple versions of the same codebase to the master at the same time, and the role/environment for the server determines which code base is used and which configurations applied.
Keith Murphy, Principal Database Reliability Engineer
Using Salt to manage more than configuration
At Everbridge we use Salt to handle hundreds of server deployments across Prod, Stage, QA and dev. Within our DRE (Data Reliability Engineering) group we manage several hundred servers using Salt automation far beyond simple configuration and server builds. After the initial server build / configuration we do the following:
- Configure replication between servers in a cluster ( mongo, elasticsearch , postgres, mysql )
- Configure backups as well as manage those backups long term
- Update Debian base packaging on servers for security updates
- Manage service upgrades of mongo, elasticsearch, kibana, postgres and mysql ( ie moving from Mongo 3.4 to Mongo 3.6)
- Configuration of ephemeral development environments including loading of seed data post setup
This session will cover some of the many ways we can use Salt as a tool to manage our large scale environments.
Greg Grieves, DevOps Engineer
Jim Watson, Systems Operations Engineer
Applying Cloud Tools in a Legacy Environment
Legacy applications were not designed for the cloud and take a long time to be modernized. Inherent difficulties exist when trying to manage legacy applications and legacy environments that are highly nonuniform. Needing distinct states for every server precludes some of the best advantages of automation using SaltStack. Here we show a technique that allows us to treat application deployment, configuration management, monitoring and automated validation as code in an environment generally where nearly every server is configured uniquely. This technique is easily deployed, managed in source control, and naturally reduces to simple roles as the environment evolves into higher uniformity and eventually transitions to the cloud.
Ben Gridley, Sr. Site Reliability Engineer Lead
Managing AWS Infrastructure using Salt, Git, and Docker
Managing infrastructure as code in AWS is not as easy as it seems. Many tools exist, but only seem to be focused on the setup and not the long term care of your infrastructure. Almost none of the other tools out there are able to manage AWS resources in the state defined in code or are unable to make all the required changes to existing infrastructure without deleting and starting over, which can result in errors. SaltStack has a solution to these problems with their state modules and can be extended to manage any issue that comes up.
We will discuss why Salt was selected to manage our AWS Infrastructure and some of the issues that the other tools couldn’t deal with, that SaltStack handles amazingly.
Using a docker container running a salt-minion we are able to schedule container tasks that run in regular intervals that uses external pillar from a git repository containing all of our infrastructure as Pillar data. This ensures zero drift in our infrastructure and allows us to make changes to our infrastructure using version control.
Security and Compliance Automation
Organizations are using SaltStack to turn their digital infrastructure into a digital fortress:
Sessions focused on vulnerability remediation, continuous compliance, security workflow orchestration, and more.
Stephen Dumesnil, Manager, Network Engineering Governance
Automating continuous compliance and security for the IBM Cloud network
In this talk, Stephen Dumesnil will discuss how his team uses SaltStack SecOps automation as part of a business-wide effort to scope, mitigate, and audit problems in hours rather than weeks across a global data center with over 80,000 network devices. Learn how the team is automating and orchestrating compliance detection and remediation efforts to deliver dramatic improvements in efficiency, including a 75% reduction in the time needed to coordinate priorities between security and IT operations teams and resolve network compliance and security issues.
Liberty Mutual Insurance
Greg Fraize, Security Engineer
Intelligent Security Automation for Juniper Firewalls and Beyond
In this talk, Greg Fraize will discuss learnings and takeaways from a recent project that required the Liberty Mutual security team—with support from Juniper, Integration Partners, and SaltStack—to use SaltStack intelligent automation to create self-healing Junos firewalls and orchestrate response to Splunk security events.
Wesley Whetstone, ClientOps Engineer and SaltStack Working Group Captain (Mac)
My Mac just got up and walked away. Now what?
Most of the time when people think of Salt and config management they think of servers sitting in a rack or on a nice fluffy cloud where it’s out of harm’s way. In this talk, we’ll go over what happens when the computer you’re trying to manage gets up and walks away, and randomly powers down or losses internet connectivity. In other words, how you can use Salt to manage macOS laptops that are actively used by someone else that probably knows more about computers than you do.
Gareth Greenaway, Senior Software Developer
Salt Pillar for Secrets Management
When using any sort of automation system for either remote execution or configuration management, one of the major advantages is the ability to reduce the repetition. By using state files with SaltStack, commonly used tasks can be automated so that the next these tasks need to be performed the action is repeatable and consistent. Often in these scenarios the need to securely store and securely provide sensitive information such as passwords arises. This is where the Salt Pillar system comes in. In this talk we’ll walk through some basic usage of the pillar system, including targeting, and then move onto using external systems for storing pillar information such as MySQL and Hashicorp Vault.
Georg Schmuecking, System Designer
DMZ-Salt-master for Onboarding Hardware
In this talk I will present the DMZ concept we use in Ericsson Industry Connect to onboard unconfigured servers at the customer site into our cloud environments.
Our Ericsson Industry Connect solution sets up an On-Premise private LTE solution. Next to our cloud, running the user frontend and many other configuration and maintenance tools, we deploy two servers at the customer site. These run the individual packet cores for each customer. We use SaltStack to install, configure, and manage a large number of these highly distributed individual servers, remotely.
Very quickly we noticed that we had to split our cloud environment into regional environments. This was necessary in order to be able to scale up to thousands of Customers, keep our latency commitments, and be present in different geographical areas. The split presented us with the challenge to remotely onboard the unconfigured servers into the different cloud environments, without any pre-configuration or identification of the server itself.
We therefore used a DMZ-Concept and introduced an additional Salt-Master, which onboards the server. It also does initial updates and checks on the software and physical installation, and is then able to migrate the server into the respective cloud environment and handover to the respective salt-master from the cloud environment.
I will present the details about the setup with the DMZ-Salt-Master, a custom remote-salt-api execution module for that purpose, and the migration process from the DMZ-Salt-Master to the Cloud-Salt-Master.
Tips, Tricks, & Best Practices
SaltStack power users dish the latest tips and tricks to help you get more Salt in your diet.
Brian LaShomb, Senior Engineer
Ensuring a healthy masterless environment with SaltShaker
What do you do if you run a masterless Salt setup and something in your config breaks Salt? That was the question we asked ourselves. We knew resiliency was needed, and we desired a process with as few dependencies as possible. We turned to Golang and created a binary that runs on macOS and Windows clients to ensure configuration is present and running. We call this SaltShaker. The binary checks for new configuration from the server, compares hashes, and puts new configuration into place, running a highstate afterwards.
Mircea Ulinic, Network Developer and Salt Working Group captain (Network)
Managing network devices and applications with Salt, without running (Proxy) Minions
As we all know very well, Salt is an agent-based software that automates the management and configuration of infrastructure and application at scale. It requires each Salt-managed node to have a Salt Minion service. Usually, this is not a blocker on a server, but in the networking world—but not limited to—it is not possible to install custom software on the network gear that you want to manage. To solve this problem, SaltStack introduced the Proxy Minion, a derivative of the regular Minion that does not need to be installed on the targeted device and can run anywhere. As with regular Minions, you will need to manage the same number of Proxy Minion services as network devices you have. But this comes with a considerable cost of infrastructure management, operations, and time. There are also many cases in the networking world where administrators only need to manage the device once or twice a year – at most, and those interactions must be consistent and safe, while constantly keeping a Proxy Minion service up and running during this time is far from ideal. However, automation is the only way to go in these scenarios. [salt-sproxy](https://github.com/mirceaulinic/salt-sproxy) enables you to manage your infrastructure using Salt, without the need to manage thousands of services. With this approach, you also have access to the Salt REST API, can leverage the event-driven automation, and more, as well as inheriting the usual flexibility and extensibility of Salt.
Philipp Lemke, TeC-S Solution Architect | Team Lead
Automate Monitoring with Salt and CheckMK
As Managed Service Provider in Healthcare Business, Agfa HealthCare is responsible for distributed clinical IT-Environments.
To ensure that we are able to deliver a high quality service to our customers we decided to support our processes with the solutions Salt & CheckMK.
Learn how to:
- Synergies between Salt & CheckMK
- Add your Salt-Minions automatically to a full functional monitoring environment
- Install CheckMK Monitoring Agents via Salt
- Use Salt Grains within CheckMK
Matthew Phillips, Engineer – R&D Network Design & Infrastructure
Orchestration Testing Lessons Learned
The Salt ecosystem provides a variety of approaches to testing Salt states for a single machine: Kitchen-Salt, Saltcheck, and Salt’s own internal test suite. But how do you test a multi-machine orchestration? This talk will look at the approach the Infrastructure Engineering team at Bloomberg has taken to handle multi-machine orchestration testing. In it, we will cover topics including Pytest, Docker, Docker Compose, test-kitchen, and Testinfra.
Advanced Automation Concepts
Learn how innovative teams are tackling big, complex IT challenges with bleeding-edge automation concepts and the power of SaltStack
Massimiliano Cuzzoli, Head of Cloud & System Engineering
SaltStack for Preventive Maintenance with Anomaly Detection and Root Cause Analysis
In this session, Massimiliano Cuzzoli will discuss how Domotz is experimenting with SaltStack to deploy features commonly exposed in AIOps (Artificial Intelligence for IT Operations). Specifically, Anomaly Detection + Root Cause Analysis. We monitor system parameters like cpu, memory, number of processes, network latency, and so on.
The huge amount of information collected to produce historical trends may be noisy and simple threshold-based alerts typically trigger a lot of false positives. We need something clever to spot unexpected behaviors. We need pattern recognition algorithms usually borrowed by the AI field. The SaltStack event bus is giving us a powerful data gathering system along with the ability to implement our own logic on top of collected data, performing correlations and taking actions.
Theo Cowan, DevOps Engineer
Turbocharge your DevOps pipeline with Salt
Learn how Pluralsight uses a standardized SaltStack formula called systemd-app, to deploy applications of any type including; java, nodejs, python, ruby as well as containerized applications as daemons, adhocs and scheduled tasks. The process is safe, reliable and fast, giving Pluralsight unprecedented ease as well as complete control of how it does DevOps.
As read here: https://trevon.dev/2019/08/04/Research-Introduction.html
Trevon Williams. PhD student, UNC Charlotte
Defense Automation: SaltStack in a BuzzWord Rich Environment
The goal of the body of research I am engaging in is to combine Software Defined Network (SDN) technologies, Network Function Virtualization techniques, and Linux containers (LXD) to create a programmable environment that introduces a higher a level of autonomic features. This proposed environment will address automated configuration and response in a plethora of environment/infrastructure OSI levels. Given the current centralized architecture of the software defined network switch and controller integration, I am utilizing SaltStack to create an event driven API to handle the container and network configurations when triggered by generically defined events.
As read here: https://trevon.dev/2019/08/04/Research-Introduction.html
Integrate and Extend
Extending the power of your existing toolset with SaltStack
Pablo Suárez Hernández, Senior Software Engineer
Running Ansible within Salt: How to smoothly migrate away from Ansible to Salt
At SUSE we firmly believe that Salt is the best choice for Configuration Management and Orchestration. We actively develop and integrate Salt as a core component of some of our products. Sometimes we face customers and users who chose Ansible as their Configuration Management engine. They invested time and efforts designing all states and playbooks to define their infrastructure. Then, at some point, they realize that with Salt they would have the same functionality and much more: real-time monitoring, reactors, event driven orchestration, superior flexibility, and more.
The latest Fluorine release of Salt comes with a new module called ansiblegate which has been started by SUSE and allows a user to run Ansible from within Salt. You can execute any Ansible module directly using Salt and you can even reuse your own Ansible playbooks and apply them using Salt! Since many customers and users have invested time designing their Ansible states and playbooks, this session will show how Salt is able to run Ansible using ansiblegate. That will allow users to smoothly migrate away from Ansible by switching to Salt and reusing all their previous Ansible efforts plus adding all extra value that Salt brings into the scene.
Sar Haidar, Senior DevOps Engineer
Salt-Proxy To The Rescue: Managing Heroku app configuration variables
This session is about leveraging salt-proxy in conjunction with an in-house Salt module to manage the configuration variables for the Heroku apps we have deployed. I will be covering the workflow prior to piecing this solution together, and then go over all the pieces that were put together, including scheduling, to automate configuration variables in Heroku apps and eliminate the manual intervention that was previously undertaken to replace expired credentials.
Brandon Bird, Sr. Systems Engineer
Mason Edgel, Systems Engineer
Scalable Salt State Version Control with ServiceNow and Git
Progressive Leasing wanted a better method to track Salt State versioning, and wanted to integrate ServiceNow CMDB into our Salt processes.
We started using GitFS for deployments, but encountered performance issues when deploying across multiple repos.
We implemented our own DevOps tool chain using GitLab runners. The resulting ServiceNow integration allows us to deploy and audit servers based on desired Salt state versions.
Giandomenico Avelluto, Lead Site Reliability Engineer
Create a One Click Migration (OCM) Process to Automate Repeatable Infrastructure migration With Salt
We love Automation, consequently we love SaltStack. Attend this session to learn how we managed part of LastMinute infrastructure migration using Salt capabilities such as Salt Orchestration system, Salt Macros and so on. You also will learn how a “flight plan” and the Salt Reactor system can help you to trace all these steps in a very simple way.
Learn how to:
- Integrate Salt with ChatOps
- Use a “flight plan” to trace steps
- Use Macros for eliminating redundant code
Salt on Salt
SaltStack employees share updates, best practices, and what’s next for SaltStack
Thomas Hatch, CTO and Co-founder
Megan Wilhite, Software engineer II
David Hilton, Senior Software Developer
Innovation Projects Deep Dive: POP, Heist and Umbra
SaltStack CTO and Founder, Thomas Hatch, will be joined by SaltStack team project leads Megan Wilhite and David Hilton to discuss three innovative and revolutionary new projects that introduce practical AI/ML concepts into the SaltStack platform, dramatically improve the pluggability of Salt components, and revolutionize agentless system control—and that’s just the beginning.
Pedro Algarvio, Platform Engineer
Wayne Werner, Senior Software Engineer
Test Driven Salt: Best practices for test cases and quality contributions
Creating a test-driven submission approach is critical to driving Salt to the next level of robustness and reliability. Come learn how to write tests for Salt, featuring real-world examples! We will discuss the three primary test styles used to solidify Salt.
Mehul Revankar, Director of Product
Kavya Chandrashekhar, Product Manager, Community
Daniel Wozniak, Software Developer
SaltStack Horizons – A look forward at what’s coming in future releases
The SaltStack product team will provide an in-depth look at the SaltStack roadmap across all open and commercial products.