Makefile, automating daily tasks & simplifying Gettings Started

This presentation dive inside Makefile workflow, properties and advanced features. With lots of examples of code it gives you everything you need to know to start using Makefiles as a tool for automating your daily tasks and also for simplifying users getting started with your project.

Transcript

1. Makefile Automating daily tasks & simplifying getting started Gustavo Pantuza
2. @gpantuza @pantuza https://blog.pantuza.com
3. Desconto de 50% na compra de SSL Utilize o código DEVBR50OFF
4. Agenda Introduction Properties Advanced • Motivation • How it works • File Format • Demo • Variables • Conditionals • Builtin functions • Includes • Defining Functions • Automatic Variables • Caveats • Other directory • Use cases
5. Introduction
6. • Automation • Simplicity • Learning curve • Getting Started • Almost no dependencies Motivation
7. • Program make zypper install make apk add make apt-get install make • A Makefile The file which make reads in order to execute rules How it work $> make build
8. By the original definition make is: “The make utility automatically determines which pieces of a large program need to be recompiled, and issues commands to recompile them.” How it work $> make what Overview
9. $> make build Building docker image Sending build context to Docker daemon 73.73kB Step 1/4 : FROM golang:1.14-alpine ---> 30df784d6206 Step 2/4 : COPY src /app ---> 5fa423e5e748 Step 3/4 : WORKDIR /app ---> Running in 3a969fe98be8 Removing intermediate container 3a969fe98be8 ---> 3a59a136b09c Step 4/4 : CMD go run main.go ---> Running in 49907a2f877a Removing intermediate container 49907a2f877a ---> 14ec4f19142c Successfully built 14ec4f19142c Successfully tagged makefile:latest How it work $> make build Overview
10. make rules File format Rule Shape
11. target: dependency command File format Rule Shape
12. • target Action to be executed by the rule • dependencies Needed to exist to satisfy the target • commands Sequence of commands to be executed File format
13. target..: dependency.. command.. .. File format Rule Shape
14. target..: dependency.. command.. .. File format Rule Shape
15. Hands On
16. Properties
17. # Branch that triggers production deployment PROD_BRANCH := production # Push current branch changes to production branch push: $(info Pushing changes to $(PROD_BRANCH)) @git push origin $(PROD_BRANCH) Variables $> make push Pushing changes to production Everything up-to-date
18. # Docker image version IMG_VERSION ?= latest # Builds the docker image based on the Dockerfile build: Dockerfile $(info Building docker image) @docker build --tag makefile:$(IMG_VERSION) . Variables $> make build ... Successfully tagged makefile:latest $> make build IMG_VERSION=0.2 ... Successfully tagged makefile:0.2 $> make build IMG_VERSION=beta ... Successfully tagged makefile:beta
19. # Source code directory SRC := src GO_FILES := $(SRC)/main.go $(SRC)/main_test.go # Runs code format on Golang files fmt: $(GO_FILES) $(info Running Go fmt on $(GO_FILES)) @go fmt $^ Variables $> make fmt Running Go fmt on src/main.go src/main_test.go
20. # Expanded once VERSION := 0.12 # Expanded on call NOW = $(shell date +%s) # Conditional assignment IMG_TAG ?= latest # Appending values CFLAGS += -O3 Variables
21. # Rolls back application to specific version rollback: ifndef ROLLBACK_VERSION $(error Missing ROLLBACK_VERSION variable) endif @kubectl rollout undo deployment app --to-revision=$(ROLLBACK_VERSION) Conditionals $> make rollback Makefile:36: *** Missing ROLLBACK_VERSION variable. Stop. $> make rollback ROLLBACK_VERSION=15 ...
22. # Operating system name OS := $(shell uname) # Install project dependencies on the Operating System deps: $(info Installing project dependencies locally) ifeq ($(OS),Linux) @zypper install --no-confirm nginx else ifeq ($(OS), Darwin) @brew install nginx else $(warning Unsupported Operating System) endif Conditionals $> make deps # Else example Makefile:36: Unsupported Operating System $> make deps # Linux example Installing project dependencies locally ... Conditionals
23. # List of files to be modified FILES := src/a src/b bin/a bin/b # String functions examples functions: @echo "Input: $(FILES)" @echo "subst -> " $(subst src, tmp, $(FILES)) @echo "filter -> " $(filter bin%, $(FILES)) @echo "findstring -> " $(findstring a, $(FILES)) @echo "sort -> " $(sort $(FILES)) @echo "word -> " $(word 2, $(FILES)) @echo "lastword -> " $(lastword, $(FILES)) Builtin functions $> make functions Input: src/a src/b bin/a bin/b subst -> tmp/a tmp/b bin/a bin/b filter -> bin/a bin/b findstring -> a sort -> bin/a bin/b src/a src/b word -> src/b lastword -> bin/b Text Functions
24. # List of files to be modified FILES := src/a src/b bin/a bin/b # Files functions examples functions: @echo "Input: $(FILES)" @echo "dir -> " $(dir $(FILES)) @echo "notdir -> " $(notdir $(FILES)) @echo "basename -> " $(basename $(FILES)) @echo "addsuffix -> " $(addsuffix .go, $(FILES)) @echo "addprefix -> " $(addprefix /tmp/, $(FILES)) @echo "abspath -> " $(abspath $(FILES)) Builtin functions $> make functions Input: src/a src/b bin/a bin/b dir -> src/ src/ bin/ bin/ notdir -> a b a b basename -> src/a src/b bin/a bin/b addsuffix -> src/a.go src/b.go bin/a.go bin/b.go addprefix -> /tmp/src/a /tmp/src/b /tmp/bin/a /tmp/bin/b abspath -> /makefile/src/a /makefile/src/b /makefile/bin/a /makefile/bin/b Files Functions
25. # List of directories DIRS := src bin logs # Directories with suffix SUF_DIRS := $(foreach dir, $(DIRS), $(addsuffix -dir, $(dir))) # Runtime functions examples runtime: @echo "$(shell hostname)" @echo $(SUF_DIRS) @echo $(wildcard src/*_test.go) $(eval AWS_ACCOUNT = $(shell aws sts get-caller-identity --query "Account")) @echo $(AWS_ACCOUNT) Builtin functions $> make runtime opensuse-machine src-dir bin-dir logs-dir src/main_test.go 151666326780
26. # Builtin functions to print messages messages: $(info Informative function) $(warning Warning function) $(error Error function) Builtin functions $> make messages Informative function Makefile:40: Warning function Makefile:41: *** Error function. Stop. Text Functions
27. # Includes an external Makefile include Makefile.conf # Gets project version read # included Makefile.conf version: @echo $(VERSION) @echo $(PROJECT_NAME) @echo $(DOCS) @echo $(PKG_MGR) Includes $> make version 0.42 makefile docs /usr/bin/zypper # =====* Makefile.conf *===== # Shell program SHELL := $(shell which bash) # Gets from shell the operating system OS := $(shell uname -s) PKG_MGR := $(shell which zypper) ifeq ($(OS), Darwin) PKG_MGR = $(shell which brew) endif # Project name PROJECT_NAME := makefile # Get the version from file VERSION := 0.42 # Documentation directory DOCS := docs
28. Advanced
29. # # Returns current timestamp (no parameters) # NOW = $(shell date +%s) # # Function to subtract the first number from the second # (with parameters) SUBTRACT = $(shell expr $(1) - $(2) ) time: @echo "Now: $(call NOW)" @echo "Subtract 20 from 30: $(call SUBTRACT, 30, 20)" Functions $> make time Now: 1595898056 Subtract 20 from 30: 10
30. # # Function that takes a template and a variable as input. # It returns template text interpolated with the variable. # define parse_template @echo $(1) | sed 's/{NAME}/$(2)/' endef # Builds templates dynamically template: $(call parse_template,"Learning {NAME} rocks!",Makefile) $(call parse_template,"Learning {NAME} rocks!",Computer Science) $(call parse_template,"Learning {NAME} rocks!",DevOps) Functions $> make template Learning Makefile rocks! Learning Computer Science rocks! Learning DevOps rocks!
31. # Magic variables example magic: Dockerfile README.md LICENSE @echo $< @echo $^ @echo $@ Automatic variables $> make magic Dockerfile Dockerfile README.md LICENSE magic Automatic Variables
32. # Resolve git command path GIT := $(shell which git) push: @$(GIT) push origin master Caveats $> make push Enumerating objects: 5, done. Counting objects: 100% (5/5), done. Delta compression using up to 8 threads Compressing objects: 100% (3/3), done. Writing objects: 100% (3/3), 1014 bytes | 1014.00 KiB/s, done. Total 3 (delta 2), reused 0 (delta 0) remote: Resolving deltas: 100% (2/2), completed with 2 local objects. To github.com:pantuza/makefile.git d865102..493d7cd master -> master
33. # @ before command avoid printing the command hide: @echo "No command shown, only result" show: echo "Command shown and also the result" Caveats $> make hide No command shown, only result $> make show echo "Command shown and also the result" Command shown and also the result
34. # .PHONY make sure to not fail if the target is # equal to a file or directory .PHONY: src src: @echo "Hi PHONY target" Caveats $> ls Dockerfile LICENSE Makefile README.md src $> make src # Whitout .PHONY make: 'src' is up to date. $> make src # Using .PHONY Hi PHONY target
35. # Using Wildcards # Validates all OpenAPI spec files check_spec: $(SRC)/%.yaml @openapi-spec-validator $^ Caveats $> make check_spec OK
36. Caveats # Chaining rules $> make start logs Running application container 6de3d88c680d57da5658a5007b988cdaed909e8cb588af20b1bc9100ca2bad07 [Http] Status=Listening, Port=8080
37. Caveats # Debugging with -n option $> make build start logs -n Building docker image docker build --tag makefile:latest . Running application container docker run -it --rm -p 8080:8080 --name makefile -d makefile:latest docker logs --tail 50 --follow makefile
38. # Makefile on an inner directory # For example /lib/Makefile binary: main.go @go build -o $@ Other directories $> make -C lib/ binary make: Entering directory '/makefile/lib' make: Leaving directory '/makefile/lib'
39. Use cases # Jenkinsfile ... stage('Unit Tests') { steps { echo 'Running Unit Tests..' sh 'make unit_test' } } stage('Integration Tests') { steps { echo 'Running Integration Tests..' sh 'make integration_test' } } ...
40. Use cases # .travis.yml sudo: required language: cpp services: - docker before_install: - make start script: - make test
41. Use cases # DRY - Don’t Repeat Yourself # CI/CD Pipeline execution chain pipeline: pep8 check_openapi check_k8s unit_test integration_test acceptance_test # Deploy application to production deploy: cloud_auth build_img push_img migrate_db patch @kubectl rollout status deployment $(DEPLOYMENT_NAME) @kubectl apply -f $(K8S_DIR)/deployment.yaml
42. Use cases # Getting started In order to have the project running locally you can just run as follows: ```bash $> make setup ``` > It will install all project dependencies and > and run application using containers You can check if everything is OK by running the tests: ```bash $> make test ``` If everything went well, please refer to the CONTRIBUTING.md and start contributing to the Project!
43. Use cases Kubernetes Python language NodeJS BusyBox Docker CLI Prometheus Traefik Terraform
44. Automating & simplifying
45. https://github.com/pantuza/makefile https://www.gnu.org/software/make/manual/make.html
46. Desconto de 50% na compra de SSL Utilize o código DEVBR50OFF
47. https://blog.pantuza.com https://github.com/pantuza https://twitter.com/gpantuza Dúvidas?