Introduction to iBeacons on iOS
Hello, I got my hands on some interesting devices from Estimote called iBeacons which they are used for sending signals to the users (iOS/ Android) phone using Bluetooth.
What I’m going to do next is to build an iOS app using these devices which changes the background color accordingly to the nearest one of these 3 beacons.
The first thing that you have to do after you create a new project from XCode of Single View Application type is to install ‘EstimoteSDK’ using Cocoa pods. If you don’t have Cocoapods installed on your Mac please do it by following the instructions from here.
From the terminal window use "cd" to navigate into your project directory and run "pod init". This will create a podfile in your project directory. Open it and under "# Pods for your project name" add the following line:
pod 'EstimoteSDK'
Then run "pod install" command in your terminal. After the installation of the cocoapod close the project and open the .workspace file and create a bridging header. Import there the EstimoteSDK with the code below.
#import <EstimoteSDK/EstimoteSDK.h>
Now let’s continue by creating a ‘iBeaconViewController’ with a UILabel inside of it having full width and height and the text aligned center, after this please create an IBOutlet to it and name it 'label' . Then set the new created view controller as the root view for the window.
func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplicationLaunchOptionsKey: Any]?) -> Bool { // Override point for customization after application launch. window?.rootViewController = UIBeaconViewController(nibName: String(describing: UIBeaconViewController.self), bundle: nil) return true }
The next step is creating the ‘BeaconManager’< file, you have it’s content below.
import UIKit enum MyBeacon: String { case pink = "4045" case magenta = "20372" case yellow = "22270" } let BeaconsUUID: String = "B9407F30-F5F8-466E-AFF9-25556B57FE6D" let RegionIdentifier: String = "IntelligentBee Office" let BeaconsMajorID: UInt16 = 11111 class BeaconManager: ESTBeaconManager { static let main : BeaconManager = BeaconManager() }
But let’s first explain what is the purpose of each item in this file. So an iBeacon contains the following main properties, an UUID, MajorID, MinorID. All of these properties represents a way for the phone to know which device should listen to.
The MajorID is used when having groups of beacons and the MinorID is to know each specific device, the minor ids are represented in the MyBeacon enum among with the beacon color. The RegionIdentifier represents a way for the app to know what region are the beacons part of and it’s used to differentiate all the regions that are monitored by the app.
Now let’s go back on the UIBeaconViewController and start writing some action.
import UIKit class UIBeaconViewController: UIViewController, ESTBeaconManagerDelegate { // MARK: - Props let region = CLBeaconRegion( proximityUUID: UUID(uuidString: BeaconsUUID)!, major: BeaconsMajorID, identifier: RegionIdentifier) let colors: [MyBeacon : UIColor] = [MyBeacon.pink: UIColor(red: 240/255.0, green: 183/255.0, blue: 183/255.0, alpha: 1), MyBeacon.magenta : UIColor(red: 149/255.0, green: 70/255.0, blue: 91/255.0, alpha: 1), MyBeacon.yellow : UIColor(red: 251/255.0, green: 254/255.0, blue: 53/255.0, alpha: 1)] // MARK: - IBOutlets @IBOutlet weak var label: UILabel!
You can guess what region does, it defines a location to detect beacons, pretty intuitive. The colors is an array which contains the mapping between the minorID and the color of each beacon.
// MARK: - UI Utilities func resetBackgroundColor() { self.view.backgroundColor = UIColor.green } // MARK: - ESTBeaconManagerDelegate - Utilities func setupBeaconManager() { BeaconManager.main.delegate = self if (BeaconManager.main.isAuthorizedForMonitoring() && BeaconManager.main.isAuthorizedForRanging()) == false { BeaconManager.main.requestAlwaysAuthorization() } } func startMonitoring() { BeaconManager.main.startMonitoring(for: region) BeaconManager.main.startRangingBeacons(in: region) }
The functions above are pretty self describing from their names, one thing I need to describe is Monitoring and Ranging. The monitoring actions are triggered when the phone enters/ exits a beacons area and the ranging is based on the proximity of the beacon.
// MARK: - ESTBeaconManagerDelegate func beaconManager(_ manager: Any, didChange status: CLAuthorizationStatus) { if status == .authorizedAlways || status == .authorizedWhenInUse { startMonitoring() } } func beaconManager(_ manager: Any, monitoringDidFailFor region: CLBeaconRegion?, withError error: Error) { label.text = "FAIL " + (region?.proximityUUID.uuidString)! } func beaconManager(_ manager: Any, didEnter region: CLBeaconRegion) { label.text = "Hello beacons from \(region.identifier)" } func beaconManager(_ manager: Any, didExitRegion region: CLBeaconRegion) { label.text = "Bye bye beacons from \(region.identifier)" } func beaconManager(_ manager: Any, didRangeBeacons beacons: [CLBeacon], in region: CLBeaconRegion) { let knownBeacons = beacons.filter { (beacon) -> Bool in return beacon.proximity != CLProximity.unknown } if let firstBeacon = knownBeacons.first, let myBeacon = MyBeacon(rawValue:firstBeacon.minor.stringValue) { let beaconColor = colors[myBeacon] self.view.backgroundColor = beaconColor } else { resetBackgroundColor() } } func beaconManager(_ manager: Any, didFailWithError error: Error) { label.text = "DID FAIL WITH ERROR" + error.localizedDescription } }
After the insertion of all the code, the app should run with no errors or warning and should look like this:
I hope this is a good introduction for iBeacons in iOS Mobile App Development. If you have any improvements or suggestions please leave a comment below.
You can get the code from here: https://github.com/intelligentbee/iBeaconTest
How to create a bridging header in iOS
Hello ! If you want to import a Objective-C code into a Swift Xcode project you definitely have to create a bridging header (this allows you to communicate with your old Objective-C classes from your Swift classes).
The process of doing this is very easy. Go to File -> New -> File… , a window will appear in which you will select “Objective-C File” , name the file however you choose, then select Create. A pop-up will appear asking you if you want to create a bridging header like in the image bellow.
Choose “Create Bridging Header” and voila, you a have it.
To complete the process delete the .m file that you choose the name and move the bridging header to a more suitable group inside the project navigator.
Read more iOS related articles.
If you want to create this manually follow the steps from here.
That’s it, hope you find this post useful and if you have suggestions please leave a comment below.
Golang Guide: A List of Top Golang Frameworks, IDEs & Tools
Since its introduction, Google’s Go Programming Language (Golang) has been experiencing an increasing popularity among mainstream users. In a December 2016 survey, 89% of the 3,595 respondents claimed that they program in Go at work or outside of work.
Additionally, Go ranks highest among the programming languages in terms of expertise and preference. This July 2017, Go ranks 10th in Tiobe's Programming Language of the Year, jumping from its 55th ranking last year.
Clearly, Go is attracting many programmers from various disciplines and software development outsourcing professionals. And it’s safe to say that this is due to the ease of using Go.
As a compiled, open-source programming language, Go makes it easy for developers to build simple, reliable, and efficient software. It is the product of the innovation and evolution of the more conservative languages such as C and C++.
With Go, the amount of code typing is reduced and writing robust APIs without sacrificing its performance has become easier. Designed for scalability and concurrency, Go makes optimizations possible. A compiler can perform all the code inspection work before runtime.
We’ve compiled a list of the top frameworks, IDEs, and tools for Golang for your quick reference. Bookmark it on your browser so that you can come back whenever you’re working with Go!
Frameworks for Golang
Web frameworks help developers build applications as easily and quickly as possible. Go is still relatively new, so it’s important to use frameworks with sufficient documentation.
Here are 9 frameworks you can use to help you build projects using the Go Language.
1. Revel
As a high productivity framework for Go, Revel includes a Hot Code Reload tool that lets you rebuild your project on every file change. It also includes a wide variety of comprehensive and high-performance features, so you don’t need to find external libraries to integrate into the framework.
2. Beego
Beego is a full-fledged MVC framework with its own logging library, ORM, and web frameworks. You don’t need to find and install third-party libraries. It features a built-in tool called Bee Tool that watches out for code changes and runs tasks when changes are detected.
Beego will save you a lot of hours, especially in the beginning of a project when you’re figuring out the logging framework or application structure.
3. Martini
Inspired by Sinatra, Martini is an extremely light but powerful framework. It was developed for writing modular web applications and services in Golang.
It features a non-intrusive design that’s quick and easy to use and includes a wide range of handlers and middleware. It’s capable of performing basic routing, exception handling, and default document serving for AngularJS apps in HTML5 mode.
Martini’s best feature is its use of reflection, which lets developers dynamically insert data into the handler functions and add new services. Martini is also fully compatible with the http.HandlerFunc interface. The downside, though, is that the Martini framework is no longer maintained.
4. Gin Gonic
Gin Gonic is a web framework with a martini-like API, but with much better performance. If you’ve used Martini before, then you’ll be familiar with Gin Gonic. Otherwise, it will only take you 10 minutes to learn Gin. It’s that easy!
Gin Gonic is a minimalistic framework that includes only the most essential libraries and features. This makes it perfect for developing high-performance REST APIs. Plus, it’s 40 times faster than Martini.
You can add middleware, nested groups, JSON validation, and rendering, but it still maintains its optimum performance. Gin Gonic uses httprouter, the fastest HTTP router for Go.
5. Buffalo
Building new web applications with Go is quick and simple with Buffalo. When you’re starting a new project, Buffalo already has everything setup for you—from front-end to back-end development.
It features Hot Reloading, which means that dev command will watch your .go and .html files automatically. It will then rebuild and restart your binary for you. Just run the dev command, and you’ll see the changes go live right before your eyes!
Buffalo is more than just a framework – it’s a holistic web development eco-system that lets you get straight to building your application.
6. Goji
Goji is a lightweight and fast web framework that has composability and simplicity as its main priority. Much like net/http.ServeMux, Goji is a minimalistic HTTP request multiplexer. It includes Einhorn support, which makes it possible for you to have websocket support in Goji.
Additional features include URL patterns, re-configurable middleware stack, graceful shutdown, and more. Goji can be used in production and has served billions of requests across several organizations.
7. Tiger Tonic
Inspired by Dropwizard, Tiger Tonic is a Go framework for developing JSON web services and building high-performance REST APIs. To stay true to the principles of Golang, Tiger Tonic strives to keep features orthogonal.
The downside to Tiger Tonic is its inadequacy when it comes to building large, back-end applications.
8. Gocraft
Another powerful yet minimalistic framework, Gocraft offers fast and scalable routing performance. It adds routing to the net/http package from the standard library.
Gocraft is a Go mux and middleware package that features casting and reflection capabilities so that you can type your code statically. You can also add an optional functionality with the built-in middleware or write your own.
Since performance is always one of the top concerns for developers, Gocraft is a great choice for developers. It’s very easy to write backend web applications using the Gocraft framework.
9. Mango
Although Mango is not actively maintained by its creator, Paul Bellamy, a lot of Go users still use it. The great thing about Mango is its modularity. You can choose from a variety of libraries to include in your project.
Mango lets you build reusable modules of HTTP functionality as quickly and easily as possible. It compiles a list of middleware and application into a single http server object to keep your code self-contained.
Integrated Development Environment (IDEs) for Golang
IDEs for Golang are gaining popularity, along with the Go Language. While many developers still prefer to use text editors, many prefer to use IDEs as well.
If you’re working on a large-scale project with an extensive codebase, an IDE can help you organize your code and navigate it with ease. Furthermore, IDEs can help you test your code and edit them accordingly.
Here are the top IDEs that work great with Golang.
1. Gogland
Software development company JetBrains released another reliable IDE, but this time, for Golang. Gogland is a commercial IDE that provides a robust ergonomic environment for Go developers. It also features coding assistance, debugger, and an integrated terminal.
Because an established company created Gogland, it has an extensive IntelliJ plugin ecosystem where you can get additional tools should you need more.
2. Visual Studio Code
Created by Microsoft, Visual Studio Code is a full-featured, open-source IDE and code editor that supports a wide variety of programming languages. It features smart completion with IntelliSense; debugging using break points, call stacks, and an interactive console; built-in Git integration; and hierarchical folder and file explorer.
As another popular IDE, Visual Studio Code has a supportive community of Go developers that regularly contribute. With Visual Studio Code, you can extend functionalities with the array of available plugins.
3. LiteIDE
LiteIDE is among the first Golang-centric, open-source IDEs that was created more than 5 years ago. As a C++ Qt application with a unique look and feel, LiteIDE offers code management, configurable build commands, gdb and Delve debugger, auto-completion and theming with WordApi, MIME type based system, and more. It also provides JSON and Golang support.
4. Wide
Wide is a web-based IDE for Golang programmers. It’s designed for collaborative development and works best for teams and web development agencies. Wide features include code highlight, debugging, Git integration, and more.
Because Wide is created and maintained by a Chinese developer, most of its documentation and support are in Chinese.
5. Atom with go-plus plugin
If you’re already using Atom, your code editing experience in Golang can be improved with an open-source package called go-plus. With go-plus, you get instant, real-time feedback on your syntax and build errors.
The go-plus package offers almost all Golang support in Atom. It can also be used for tools, build flows, linters, vet and coverage tools.
Go-plus also includes various code snippets and features such as autocomplete with gocode, code formatting with gofmt, goreturns, or goimports, and more.
6. Eclipse with GoClipse
Because Eclipse is a widely popular IDE, numerous plugins have been created for it. GoClipse is an Eclipse plugin for Golang that offers Go source code editing with configurable syntax highlighting and automatic indentation and brace completion.
GoClipse also serves as a project wizard and builder that reports syntax and build errors instantly. Additional features of GoClipse include debugging functionality and code assist.
7. Sublime Text with GoSublime
Sublime Text is another sophisticated text editor with a large community of contributors and developers. As such, a wide variety of plugins has been created for this IDE.
GoSublime is a Golang plugin for Sublime Text 3 that offers code completion from Gocode, lint/syntax check while you’re wiring code, automatic addition and removal of package imports, and more.
8. Vim with vim-go plugin
Vim is a free, open-source IDE that can be customized and configured with various plugins. If you’re a Golang programmer, you can use Vim with the vim-go plugin created by Fatih Arslan. Vim-go automatically installs all the necessary binaries for providing a smooth Vim integration for Golang.
Vim-go is a powerful plugin suite for writing and developing Go. Its features include advanced source code analysis, adding and removing import paths, multiple 3rd liner support, goto definition, quick file executions, and much more.
Vim-go is highly customizable, with individual features that can be enabled or disabled according to your need.
9. Komodo
Komodo is a full-featured Go language IDE that supports other programming languages such as Node.js, Python, Ruby, Perl, and more. With this Go IDE, you can write clean code easily. Its features include an advanced code editor, intelligent code completion, syntax checking, version control and unit testing, and a Go Code Intelligence that allows code browsing and code hinting.
The great thing about Komodo is that it works great for team collaboration since multiple developers can edit a document simultaneously. Komodo can be installed on Mac, Windows, or Linux with just one license.
10. IntelliJ IDEA with Go Language (golang.org) Support Plugin
IntelliJ IDEA (same company as JetBrains) is an IDE that can be used with Golang through the Go language support plugin. If you want to use IntelliJ IDEA with Golang, you need to install this plugin, albeit with limited features as opposed to Gogland.
Tools for Golang
Golang tools can be used for a wide variety of projects and web applications. Developers can write code and build applications as quickly and easily as possible with these helpful tools.
Here’s a list of the top Golang tools for your reference.
1. Apicompat
Apicompat is a new Go language tool that helps developers detect backwards, incompatible changes and exported declarations.
With Apicompat, you can avoid false positives. However, not every backwards incompatible change can be detected by Apicompat. Swapping argument parameters and other changes still need to be considered by the library author.
2. Checkstyle
Inspired by Java Checkstyle, Checkstyle for Golang prints out coding style suggestions. It also lets developers check file line/function and line/param number, which can then be configured by the user.
3. Depth
Depth is another useful Golang tool that helps web developers retrieve and visualize Go source code dependency trees. It can be used as a standalone command-line application or as a particular package within your own project. You can add customizations by simply setting the appropriate flags on the Tree before resolving.
4. Go-Swagger
This toolkit includes a wide variety of features and functions. Go-Swagger is an implementation of Swagger 2.0, and can serialize and deserialize swagger specifications. It’s a minimalist yet powerful representation of your RESTful API.
With Go-Swagger, you can swagger spec document, validate against jsonschema, and other extra rules. Other features include code generation, API generation based on swagger specs, spec document generation based on the code, extended string formats, and more.
5. Go Meta Linter
If you need to run Go lint tools and normalize their output concurrently, that’s exactly what Go Meta Linter can do for you. Go Meta Linter is intended to be used with a text editor or an IDE integration such as Sublime Linter plugin, Atom go-plus package, Emacs Flycheck checker, Vim/Neovim, and Go for Visual Studio Code. It also supports a wide variety of linters and configuration files like JSON.
6. Go-callvis
Go-callvis is a web development tool that allows you to visualize the call graph of your Go program with Graphviz's dot format. This tool is especially useful when building large projects with complex codebases. This is also useful when you want to understand another developer’s code structure or rebuild someone else’s project.
With go-callvis, developers can focus specific package within a program; group functions according to package and methods according to type; and limit packages to custom path prefixes, and ignore those that contain them.
7. Gonative
Gonative is a simple Golang tool that lets you build Go toolchains with native libs, which can be cross-compiled while still utilizing the Cgo-enabled versions of the stdlib packages.
Gonative downloads the binary distributions for each platform and copies their libraries into its proper places. At the same time, Gonative sets the correct mod time to avoid unnecessary rebuilds.
Unfortunately, Gonative remains untested on Windows. Additionally, there’s no Linux/arm support provided.
8. Grapes
Grapes is a lightweight Golang tool designed to distribute commands over ssh easily. It’s written and actively maintained by Yaron Sumel.
Grapes will soon support full host key validation, so that’s something developers should watch out for.
9. Gosimple
The great thing about this Golang linter is that it focuses on simplifying Go source code. Gosimple always targets the latest Go version, so it requires Go version 1.6 or later.
If there’s a new Go release, gosimple will suggest the easiest and simplest methods to avoid complicated constructs.
10. Go Vendor
Go Vendor is the Golang tool that works with the standard Vendor folder. It allows developers to copy existing dependencies from $GOPATH with govendor add/update. You can also directly pull new dependencies or update existing dependencies with govendor fetch and move legacy systems with govendor migrate.
Wrapping It Up
If you’re coming from a JS/Node background, you need to learn some new programming concepts such as coroutines, channels, strict typing with compilation, interfaces, structs, pointers, and some other differences. But, once you get into the groove, you’ll find Golang easier and faster to use.
Face Detection with Apple’s iOS 11 Vision Framework
Great stuff is coming from Apple this autumn! Among a lot of new APIs there is the Vision Framework which helps with detection of faces, face features, object tracking and others.
In this post we will take a look at how can one put the face detection to work. We will make a simple application that can take a photo (using the camera of from the library) and will draw some lines on the faces it detects to show you the power of Vision.
Select an Image
I will go fast through this so if you are a really beginner and you find this too hard to follow, please check the my previous iOS related post, Building a Travel Photo Sharing iOS App, first, as it has the same photo selection functionality but explained in greater detail.
You will need Xcode 9 beta and a device running iOS 11 beta to test this. Let’s start by creating a new Single View App project named FaceVision:
Open the Main.storyboard
and drag a button Take Photo
to the center of it. Use the constraints to make it stay there :) Create a takePhoto
action for it:
@IBAction func takePhoto(_ sender: UIButton) { let picker = UIImagePickerController() picker.delegate = self let alert = UIAlertController(title: nil, message: nil, preferredStyle: .actionSheet) if UIImagePickerController.isSourceTypeAvailable(.camera) { alert.addAction(UIAlertAction(title: "Camera", style: .default, handler: {action in picker.sourceType = .camera self.present(picker, animated: true, completion: nil) })) } alert.addAction(UIAlertAction(title: "Photo Library", style: .default, handler: { action in picker.sourceType = .photoLibrary // on iPad we are required to present this as a popover if UIDevice.current.userInterfaceIdiom == .pad { picker.modalPresentationStyle = .popover picker.popoverPresentationController?.sourceView = self.view picker.popoverPresentationController?.sourceRect = self.takePhotoButton.frame } self.present(picker, animated: true, completion: nil) })) alert.addAction(UIAlertAction(title: "Cancel", style: .cancel, handler: nil)) // on iPad this is a popover alert.popoverPresentationController?.sourceView = self.view alert.popoverPresentationController?.sourceRect = takePhotoButton.frame self.present(alert, animated: true, completion: nil) }
Here we used an UIImagePickerController
to get an image so we have to make our ViewController
implement the UIImagePickerControllerDelegate
and UINavigationControllerDelegate
protocols:
class ViewController: UIViewController, UIImagePickerControllerDelegate, UINavigationControllerDelegate {
We also need an outlet for the button:
@IBOutlet weak var takePhotoButton: UIButton!
And an image
var:
var image: UIImage!
We also need to add the following in the Info.plist to be able to access the camera and the photo library:
Privacy - Camera Usage Description
: Access to the camera is needed in order to be able to take a photo to be analyzed by the appPrivacy - Photo Library Usage Description
: Access to the photo library is needed in order to be able to choose a photo to be analyzed by the app
After the users chooses an image we will use another view controller to show it and to let the user start the processing or go back to the first screen. Add a new View Controller in the Main.storyboard
. In it, add an Image View with an Aspect Fit
Content Mode and two buttons like in the image below (don’t forget to use the necessary constraints):
Now, create a new UIViewController class named ImageViewControler.swift
and set it to be the class of the new View Controller you just added in the Main.storyboard
:
import UIKit class ImageViewController: UIViewController { override func viewDidLoad() { super.viewDidLoad() // Do any additional setup after loading the view. } }
Still in the Main.storyboard
, create a Present Modally
kind segue between the two view controllers with the showImageSegue
identifier:
Also add an outlet for the Image View and a new property to hold the image from the user:
@IBOutlet weak var imageView: UIImageView! var image: UIImage!
Now, back to our initial ViewController
class, we need to present the new ImageViewController
and set the selected image:
func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [String : Any]) { dismiss(animated: true, completion: nil) image = info[UIImagePickerControllerOriginalImage] as! UIImage performSegue(withIdentifier: "showImageSegue", sender: self) } override func prepare(for segue: UIStoryboardSegue, sender: Any?) { if segue.identifier == "showImageSegue" { if let imageViewController = segue.destination as? ImageViewController { imageViewController.image = self.image } } }
We also need an exit method to be called when we press the Close button from the Image View Controller:
@IBAction func exit(unwindSegue: UIStoryboardSegue) { image = nil }
To make this work, head back to the Main.storyboard
and Ctrl+drag from the Close button to the exit icon of the Image View Controller and select the exit method from the popup.
To actually show the selected image to the user we have to set it to the imageView
:
override func viewDidLoad() { super.viewDidLoad() // Do any additional setup after loading the view. imageView.image = image }
If you run the app now you should be able to select a photo either from the camera or from the library and it will be presented to you in the second view controller with the Close and Process! buttons below it.
Detect Face Features
It’s time to get to the fun part, detect the faces and faces features in the image.
Create a new process
action for the Process! button with the following content:
@IBAction func process(_ sender: UIButton) { var orientation:Int32 = 0 // detect image orientation, we need it to be accurate for the face detection to work switch image.imageOrientation { case .up: orientation = 1 case .right: orientation = 6 case .down: orientation = 3 case .left: orientation = 8 default: orientation = 1 } // vision let faceLandmarksRequest = VNDetectFaceLandmarksRequest(completionHandler: self.handleFaceFeatures) let requestHandler = VNImageRequestHandler(cgImage: image.cgImage!, orientation: orientation ,options: [:]) do { try requestHandler.perform([faceLandmarksRequest]) } catch { print(error) } }
After translating the image orientation from UIImageOrientation values to kCGImagePropertyOrientation values (not sure why Apple didn’t make them the same), the code will start the detection process from the Vision framework. Don’t forget to import Vision
to have access to it’s API.
We’ll add now the method that will be called when the Vision’s processing is done:
func handleFaceFeatures(request: VNRequest, errror: Error?) { guard let observations = request.results as? [VNFaceObservation] else { fatalError("unexpected result type!") } for face in observations { addFaceLandmarksToImage(face) } }
This also calls yet another method that does the actual drawing on the image based on the data received from the detect face landmarks request:
func addFaceLandmarksToImage(_ face: VNFaceObservation) { UIGraphicsBeginImageContextWithOptions(image.size, true, 0.0) let context = UIGraphicsGetCurrentContext() // draw the image image.draw(in: CGRect(x: 0, y: 0, width: image.size.width, height: image.size.height)) context?.translateBy(x: 0, y: image.size.height) context?.scaleBy(x: 1.0, y: -1.0) // draw the face rect let w = face.boundingBox.size.width * image.size.width let h = face.boundingBox.size.height * image.size.height let x = face.boundingBox.origin.x * image.size.width let y = face.boundingBox.origin.y * image.size.height let faceRect = CGRect(x: x, y: y, width: w, height: h) context?.saveGState() context?.setStrokeColor(UIColor.red.cgColor) context?.setLineWidth(8.0) context?.addRect(faceRect) context?.drawPath(using: .stroke) context?.restoreGState() // face contour context?.saveGState() context?.setStrokeColor(UIColor.yellow.cgColor) if let landmark = face.landmarks?.faceContour { for i in 0...landmark.pointCount - 1 { // last point is 0,0 let point = landmark.point(at: i) if i == 0 { context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } else { context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } } } context?.setLineWidth(8.0) context?.drawPath(using: .stroke) context?.saveGState() // outer lips context?.saveGState() context?.setStrokeColor(UIColor.yellow.cgColor) if let landmark = face.landmarks?.outerLips { for i in 0...landmark.pointCount - 1 { // last point is 0,0 let point = landmark.point(at: i) if i == 0 { context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } else { context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } } } context?.closePath() context?.setLineWidth(8.0) context?.drawPath(using: .stroke) context?.saveGState() // inner lips context?.saveGState() context?.setStrokeColor(UIColor.yellow.cgColor) if let landmark = face.landmarks?.innerLips { for i in 0...landmark.pointCount - 1 { // last point is 0,0 let point = landmark.point(at: i) if i == 0 { context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } else { context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } } } context?.closePath() context?.setLineWidth(8.0) context?.drawPath(using: .stroke) context?.saveGState() // left eye context?.saveGState() context?.setStrokeColor(UIColor.yellow.cgColor) if let landmark = face.landmarks?.leftEye { for i in 0...landmark.pointCount - 1 { // last point is 0,0 let point = landmark.point(at: i) if i == 0 { context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } else { context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } } } context?.closePath() context?.setLineWidth(8.0) context?.drawPath(using: .stroke) context?.saveGState() // right eye context?.saveGState() context?.setStrokeColor(UIColor.yellow.cgColor) if let landmark = face.landmarks?.rightEye { for i in 0...landmark.pointCount - 1 { // last point is 0,0 let point = landmark.point(at: i) if i == 0 { context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } else { context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } } } context?.closePath() context?.setLineWidth(8.0) context?.drawPath(using: .stroke) context?.saveGState() // left pupil context?.saveGState() context?.setStrokeColor(UIColor.yellow.cgColor) if let landmark = face.landmarks?.leftPupil { for i in 0...landmark.pointCount - 1 { // last point is 0,0 let point = landmark.point(at: i) if i == 0 { context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } else { context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } } } context?.closePath() context?.setLineWidth(8.0) context?.drawPath(using: .stroke) context?.saveGState() // right pupil context?.saveGState() context?.setStrokeColor(UIColor.yellow.cgColor) if let landmark = face.landmarks?.rightPupil { for i in 0...landmark.pointCount - 1 { // last point is 0,0 let point = landmark.point(at: i) if i == 0 { context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } else { context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } } } context?.closePath() context?.setLineWidth(8.0) context?.drawPath(using: .stroke) context?.saveGState() // left eyebrow context?.saveGState() context?.setStrokeColor(UIColor.yellow.cgColor) if let landmark = face.landmarks?.leftEyebrow { for i in 0...landmark.pointCount - 1 { // last point is 0,0 let point = landmark.point(at: i) if i == 0 { context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } else { context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } } } context?.setLineWidth(8.0) context?.drawPath(using: .stroke) context?.saveGState() // right eyebrow context?.saveGState() context?.setStrokeColor(UIColor.yellow.cgColor) if let landmark = face.landmarks?.rightEyebrow { for i in 0...landmark.pointCount - 1 { // last point is 0,0 let point = landmark.point(at: i) if i == 0 { context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } else { context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } } } context?.setLineWidth(8.0) context?.drawPath(using: .stroke) context?.saveGState() // nose context?.saveGState() context?.setStrokeColor(UIColor.yellow.cgColor) if let landmark = face.landmarks?.nose { for i in 0...landmark.pointCount - 1 { // last point is 0,0 let point = landmark.point(at: i) if i == 0 { context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } else { context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } } } context?.closePath() context?.setLineWidth(8.0) context?.drawPath(using: .stroke) context?.saveGState() // nose crest context?.saveGState() context?.setStrokeColor(UIColor.yellow.cgColor) if let landmark = face.landmarks?.noseCrest { for i in 0...landmark.pointCount - 1 { // last point is 0,0 let point = landmark.point(at: i) if i == 0 { context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } else { context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } } } context?.setLineWidth(8.0) context?.drawPath(using: .stroke) context?.saveGState() // median line context?.saveGState() context?.setStrokeColor(UIColor.yellow.cgColor) if let landmark = face.landmarks?.medianLine { for i in 0...landmark.pointCount - 1 { // last point is 0,0 let point = landmark.point(at: i) if i == 0 { context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } else { context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h)) } } } context?.setLineWidth(8.0) context?.drawPath(using: .stroke) context?.saveGState() // get the final image let finalImage = UIGraphicsGetImageFromCurrentImageContext() // end drawing context UIGraphicsEndImageContext() imageView.image = finalImage }
As you can see we have quite a lot of features that Vision is able to identify: the face contour, the mouth (both inner and outer lips), the eyes together with the pupils and eyebrows, the nose and the nose crest and, finally, the median line of the faces.
You can now run the app and take some unusual selfies of yourself. Here’s mine:
I hope you enjoyed this, please let me know in the comments how did it go and if there are things that can be improved. Also, some pictures taken with the app wouldn’t hurt at all :)
You can get the code from here: https://github.com/intelligentbee/FaceVision
Thanks!
Mysqldump Command - Useful Usage Examples
One of the tasks a sysadmin will always have on their list is backing up databases. These backups are also called dump files because, usually, they are generated with mysqldump
command.
I am going to share a few tricks on mysqldump that will help when handling servers with many relatively small databases.
The most simple way to backup databases would be using mysqldump
command with the the --all-databases
attribute. But I find that having each database saved in its own file more convenient to use.
Lets first suppose that you need to run a script that alters in databases, and that you just need a simple way to have a rollback point, just in case. I used to run something like this before:
for i in \ `ls /var/lib/mysql/`; \ do mysqldump -u root -p*** --skip-lock-tables --skip-add-locks --quick --single-transaction $i > $i.sql; done
where *** is your root password. The aditional parameters --skip-lock-tables --skip-add-locks --quick --single-transaction
assure availability and consistency of dump file for InnoDB databases (the default storage engine as of MySQL 5.5.5).
Mysql stores databases in folders using same name as database name in /var/lib/mysql
. The command picks database names from the listing of /var/lib/mysql
folder and exports to files using same name adding the .sql
.
There are 2 issues with the above command:
- It will try to execute a dump for every file/folder listed in
/var/lib/mysql
. So if you have error logs or whatever other files it will create.sql
dumps for them too. This will send just directory names as database names to export:for i in \ `find /var/lib/mysql/ -type d | sed 's/\/var\/lib\/mysql\///g'`;\ do mysqldump -u root -p*** --skip-lock-tables --skip-add-locks --quick --single-transaction $i > $i.sql; done
I find this to be hard to type and prefer to use one I will explain in point 2, since it also covers this.
- When database names have characters like
-
the folder name will have@002
instead. If that is the case, you can use something like:for i in \ `mysql -u root -p*** -e 'show databases'`;\ do mysqldump -u root -p*** --skip-lock-tables --skip-add-locks --quick --single-transaction $i > $i.sql;done
This picks database names to export form mysql
show databases
command.
But, one time I had to export databases with /
in their names. And there is no way to export as I showed above, since /
can't be used in file names since it is actually a markup for directories. So I did this:
for i in \ `mysql -u root -p*** -e 'show databases'`;\ do mysqldump -u root -p*** --skip-lock-tables --skip-add-locks --quick --single-transaction $i > `echo $i | sed "s/\//_/g"`.sql;done
This wil replace /
with _
for the dump file names.
For all of the above, we could (for obvious reasons) not use root mysql user. We could also run the backing up from a different location. In order to do this, we would need to create a mysql user with the right privileges on the machine we want to back up.
create user 'backupuser'@'111.222.333.444' identified by 'backuppassword'; grant select, show view, trigger, lock tables, reload, show databases on *.* to 'backupuser'@'111.222.333.444'; flush privileges;
where 111.222.333.444 is the ip of the remote machine.
Now you can issue mysqldump command from the other machine like this:
for i in \ `mysql -u backupuser -pbackuppassword -e 'show databases'`;\ do mysqldump -u backupuser -pbackuppassword -h 444.333.222.111 --skip-lock-tables --skip-add-locks --quick --single-transaction $i > `echo $i | sed "s/\//_/g"`.sql;done
where 444.333.222.111 is the ip of the machine we want to backup.
Lets take it to the next step , and put all our knowledge in a shell script.
#!/bin/bash echo "Starting the backup script..." ROOTDIR="/backup/mysql/" YEAR=`date +%Y` MONTH=`date +%m` DAY=`date +%d` HOUR=`date +%H` SERVER="444.333.222.111" BLACKLIST="information_schema performance_schema" ADDITIONAL_MYSQLDUMP_PARAMS="--skip-lock-tables --skip-add-locks --quick --single-transaction" MYSQL_USER="backupuser" MYSQL_PASSWORD="backuppassword" # Read MySQL password from stdin if empty if [ -z "${MYSQL_PASSWORD}" ]; then echo -n "Enter MySQL ${MYSQL_USER} password: " read -s MYSQL_PASSWORD echo fi # Check MySQL credentials echo exit | mysql --user=${MYSQL_USER} --password=${MYSQL_PASSWORD} --host=${SERVER} -B 2>/dev/null if [ "$?" -gt 0 ]; then echo "MySQL ${MYSQL_USER} - wrong credentials" exit 1 else echo "MySQL ${MYSQL_USER} - was able to connect." fi #creating backup path if [ ! -d "$ROOTDIR/$YEAR/$MONTH/$DAY/$HOUR" ]; then mkdir -p "$ROOTDIR/$YEAR/$MONTH/$DAY/$HOUR" chmod -R 700 $ROOTDIR fi echo "running mysqldump" dblist=`mysql -u ${MYSQL_USER} -p${MYSQL_PASSWORD} -h $SERVER -e "show databases" | sed -n '2,$ p'` for db in $dblist; do echo "Backuping $db" isBl=`echo $BLACKLIST |grep $db` if [ $? == 1 ]; then mysqldump ${ADDITIONAL_MYSQLDUMP_PARAMS} -u ${MYSQL_USER} -p${MYSQL_PASSWORD} -h $SERVER $db | gzip --best > "$ROOTDIR/$YEAR/$MONTH/$DAY/$HOUR/`echo $db | sed 's/\//_/g'`.sql.gz" echo "Backup of $db ends with $? exit code" else echo "Database $db is blacklisted, skipped" fi done echo echo "dump completed"
This will also compress the dump files to save storage.
Save the script as backup-mysql.sh
somewhere on the machine you want backups saved, ensure you have the mysql user with the right credentials on the server hosting the mysql. You will also need mysql installed on the backup server. Executesudo chmod 700 backup-mysql.sh
. Run the script with sudo sh backup-mysql.sh
. After making sure it works properly, you can also add it to your crontab, so that it runs on a regular schedule.
7 Mobile App Development Trends That Matter to Your Business
Mobile app development is a growing industry—thanks to the increasing number of smartphone users. According to the Digital in 2017 Global Overview report, 4.92 billion or more than half of the world’s population now uses a smartphone. The Ericsson Mobility Report predicts that there will be a total of 6.1 billion smartphone users by 2020.
As such, the overall usage of mobile apps is on the rise. The Apple App Store already boasts of almost 2 million apps, and it’s predicted that by 2020, global iOS application downloads will be at 35.2 billion. Google Play Store, on the other hand, has over 2.2 million apps, with an astounding prediction of 80.3 billion Android application downloads by 2020.
The statistics above clearly indicate how important it is for businesses to adapt to consumer trends. Some businesses resort to outsourcing app development experts to avoid being left behind. Part of keeping up is being aware of these seven mobile app development trends:
1. Strict Security Measures
Since smartphones hold highly sensitive personal and financial information of every user, stricter security measures are crucial in app development to protect users from hacking and breaches.
A Gartner study claims that 75% of mobile apps would fail even the basic security tests. Some security concerns such as insecure storage, privacy violation, system information leaks, and insecure deployment were also raised by an HPE Cyber Risk Report.
To address such issues, Apple already implemented the mandatory ‘App Transport Security’ feature last January 2017.
2. Augmented Reality and Virtual Reality
Just last year, smartphone users all over the world went crazy for Pokemon Go, which uses augmented reality to seamlessly blend the real world with the pocket monsters’ world. Meanwhile, real estate companies are adopting virtual reality apps to let home buyers feel what it’s like to live in the homes they sell.
Aside from gaming and real estate, we will see an upsurge in AR and VR apps in the coming years. Fashion, retail, TV, and other businesses will see the benefits of using AR and VR to woo consumers into buying their products.
3. Hybrid Apps for Enterprises
As more enterprises embrace the “bring your own device” model, and more workers are using mobile devices for work, the demand for hybrid apps will increase. A recent study shows that more than 72% of organizations are already starting to adopt BYOD models.
Hybrid apps are essentially mobile websites that run in an app form. Hybrid apps are responsive to multiple operating systems and devices—from smartphones and tablets to laptops and desktops.
Businesses will continue to see the benefits of hybrid apps for enterprises. They’re more cost effective and faster to develop.
4. Cloud-Based Mobile Apps
A lot of apps use heavy graphics, illustrations, and functions, and need to be accessible across all platforms. Cloud technologies can also be used to develop mobile apps that require integration with IoT, wearable devices, artificial intelligence, and the like.
Mobile cloud adoption and cloud-based applications will be more widespread. Apps no longer need to take up too much space in the smartphone’s internal memory, as files can be served directly from the cloud.
5. Big Data
Big data analytics has been a crucial tool for businesses and marketing professionals to measure the results of their digital products and investments. With big data integration, businesses, as well as developers, will be able to track user behavior that would help them arrive at informed decisions when updating mobile app UX and UI.
6. Artificial Intelligence (AI) and Chat Bots
AI and machine learning will help developers write better codes and boost the capabilities and functions of mobile apps. Microsoft's Intellisense is one of the many companies adapting AI and machine learning in their developer tools. Other mobile apps will increasingly feature AI and machine learning capabilities to predict user preference and behavior.
Chat bots are one of the many applications of AI and machine learning in mobile app development. As online shopping becomes more popular, customized and conversational commerce will soon be common, which will be made possible with chat bots.
7. Internet of Things
These days, it’s not just about being connected, it’s about being interconnected. That’s what Internet of Things (IoT) is all about.
Everything now is being made ‘smart’—from smartphones, smart watches, and smart TVs to smart homes and smart offices. Mobile app developers need to consider other devices as well aside from just the smartphone.
These trends are here to stay. Consider these when mapping out business strategies, business owners, entrepreneurs, and marketers. Also, be sure to hire trusted mobile app development companies to help you reach success.
Profiling web applications in Golang
I've been watching the 2017 Gophercon videos from here. There are many good talks that I would recommend watching from that list.
One that I really wanted to try out on my own was the profiling presentation that Peter Bourgon did. I assume for the sake of simplicity, he left some details out. I've been trying to figure them out on my own.
I was inspired to try to profile my own Golang web apps using the method he presented in his talk. So in this post I'll show you a simplified example of how to profile your own web applications in more detail.
The web app
To see a practical example of profiling and to keep it simple, we need to create a web app that when called on a particular route, performs some sort of calculation and returns a result in it's payload.
If we were to mimic real life scenarios, we would be here all day. So just to keep it simple, we will create a single route that calculates the 25th Fibonacci number when it's called and returns it in the response body.
I've written two versions of the Fibonacci function before hand, one that performs recursion (exponential time - very CPU intensive) and one that calculates the number using a vector (linear time - not very CPU intensive).
In order to use the profiling tool, you need to import the net/http/pprof package and register some routes. In the presentation I mentioned earlier, the speaker mentioned that we could leave this imported even in production environments since it does not affect performance.
package main import ( "fmt" "log" "net/http" "net/http/pprof" ) // O(n) Fibonacci func linearFibonacci(n int) int { // Create an int array of size n + 1 v := make([]int, n+1) // F(0) = 0 v[0] = 0 // F(1) = 1 v[1] = 1 // F(i) = F(i-1) + F(i-2) for i := 2; i <= n; i++ { v[i] = v[i-1] + v[i-2] } // F(n) - return the n-th Fibonacci number return v[n] } // O(2^n) Fibonacci func exponentialFibonacci(n int) int { // F(0) = 0 if n == 0 { return 0 } // F(1) = 1 if n == 1 { return 1 } // F(n) = F(n-1) + F(n-2) - return the n-th Fibonacci number return exponentialFibonacci(n-1) + exponentialFibonacci(n-2) } // HTTP request handler func handler(w http.ResponseWriter, r *http.Request) { // return the 25th Fibonacci number in the response payload fmt.Fprintf(w, "%d", exponentialFibonacci(25)) } func main() { // Create a new HTTP multiplexer mux := http.NewServeMux() // Register our handler for the / route mux.HandleFunc("/", handler) // Add the pprof routes mux.HandleFunc("/debug/pprof/", pprof.Index) mux.HandleFunc("/debug/pprof/cmdline", pprof.Cmdline) mux.HandleFunc("/debug/pprof/profile", pprof.Profile) mux.HandleFunc("/debug/pprof/symbol", pprof.Symbol) mux.HandleFunc("/debug/pprof/trace", pprof.Trace) mux.Handle("/debug/pprof/block", pprof.Handler("block")) mux.Handle("/debug/pprof/goroutine", pprof.Handler("goroutine")) mux.Handle("/debug/pprof/heap", pprof.Handler("heap")) mux.Handle("/debug/pprof/threadcreate", pprof.Handler("threadcreate")) // Start listening on port 8080 if err := http.ListenAndServe(":8080", mux); err != nil { log.Fatal(fmt.Sprintf("Error when starting or running http server: %v", err)) } }
As you can see, this is a really simple application, save it to a file called main.go and build it like this: go build -o myserver main.go .
Now you can run your binary: ./myserver and to check if it's working we'll send a request to it:
$ curl http://localhost:8080 75025
The CPU profile
Now, while the server is running, you will need to run two commands in parallel. You first need to start the profiling tool which will record data for 30 seconds after it is run AND as it is running, run the Apache Benchmark tool to send a few requests it's way.
So you will need to run the profiling tool like this:
go tool pprof -seconds 30 myserver http://localhost:8080/debug/pprof/profile
While that's running, run the benchmark:
ab -k -c 8 -n 100000 "http://127.0.0.1:8080/"
This will create a total of 100,000 keep-alive requests to your server, using 8 cores. To better understand what this benchmark command does, it is explained in the link provided.
After 30 seconds, the profiling tool will show a prompt for commands that will look something like this:
$ go tool pprof -seconds 30 myserver http://localhost:8080/debug/pprof/profile Fetching profile from http://localhost:8080/debug/pprof/profile?seconds=30 Please wait... (30s) Saved profile in /Users/username/pprof/pprof.myserver.localhost:8080.samples.cpu.013.pb.gz Entering interactive mode (type "help" for commands) (pprof)
Here you can run commands to show you how much of CPU time each function took and other useful information. For example, if I run top5 it will list the top 5 functions that are hogging the CPU:
(pprof) top5 84.57s of 85.18s total (99.28%) Dropped 87 nodes (cum <= 0.43s) Showing top 5 nodes out of 30 (cum >= 1.09s) flat flat% sum% cum cum% 51.93s 60.97% 60.97% 51.93s 60.97% main.exponentialFibonacci 20.57s 24.15% 85.11% 20.59s 24.17% fmt.(*pp).doPrintf 12.06s 14.16% 99.27% 12.06s 14.16% syscall.Syscall 0.01s 0.012% 99.28% 11.22s 13.17% net.(*netFD).Write 0 0% 99.28% 1.09s 1.28% bufio.(*Reader).ReadLine
As you can see the exponentialFibonacci function really hogs the CPU.
Please note: These values might differ on your machine. For reference I'm using a MacBook Pro (Retina, 13-inch, Early 2015), 2.7 GHz Intel Core i5 Processor and 8 GB 1867 MHz DDR3 of Memory.
If we wanted to see a graph of this profile we need to run the web command like this:
(pprof) web (pprof)
This will open up your default browser and display an image of the profile. Here's a crop of the image that concerns our Fibonacci function:
So during that profile, 60.97% of the time, the exponentialFibonacci was running on the CPU.
Optimizations
Now, we know from the theory that O(n) < O(2^n). Let's see if this holds up in practice, it we were to replace the exponentialFibonacci call with linearFibonacci inside the handler function.
Now we run the profile again. You can immediately see that it took less time because the benchmark actually finishes really fast this time.
If we run top5 now, the linearFibonacci function doesn't even make the cut. Even if you try to do top100 you will not find it because the compiler inlined that particular code.
So we need to rebuild the application with the compiler flags that disable inlining like this:
go build -gcflags -l -o myserver main.go
Now even with this flag enabled I had a hard time finding the function in the top. I went ahead and increased the hard-coded value for the n-th Fibonacci number to 10,000. So I'm looking for the 10,000th Fibonacci number, this number doesn't even fit inside the integer datatype in Golang. It will overflow several times before coming to a stop. I also increased the benchmark to 1,000,000 requests.
Now if I run top5 I get:
(pprof) top5 36.21s of 46.49s total (77.89%) Dropped 226 nodes (cum <= 0.23s) Showing top 5 nodes out of 102 (cum >= 1.97s) flat flat% sum% cum cum% 25.94s 55.80% 55.80% 26.23s 56.42% syscall.Syscall 3.38s 7.27% 63.07% 3.38s 7.27% runtime.kevent 2.60s 5.59% 68.66% 2.60s 5.59% runtime.usleep 2.32s 4.99% 73.65% 4.26s 9.16% main.linearFibonacci 1.97s 4.24% 77.89% 1.97s 4.24% runtime.mach_semaphore_signal
Or in graphical format:
As you can see, it barely even makes a dent.
So for this test, calculating the 25th Fibonacci number recursively takes 60% of the CPU while calculating the 10,000th Fibonacci number linearly takes 4% of the CPU (without inlining).
Another useful command for pprof to see how much CPU time a function takes is the list command. Or, if you're like me, to find out if a function is actually called.
For our linearFibonacci function it looks like this:
(pprof) list linearFibonacci Total: 46.49s ROUTINE ======================== main.linearFibonacci in /Users/username/workspace/go/src/github.com/username/test_profiling/main.go 2.32s 4.26s (flat, cum) 9.16% of Total . . 8:) . . 9: . . 10:// O(n) Fibonacci . . 11:func linearFibonacci(n int) int { . . 12: // Create an int array of size n + 1 10ms 1.95s 13: v := make([]int, n+1) . . 14: . . 15: // F(0) = 0 . . 16: v[0] = 0 . . 17: // F(1) = 1 . . 18: v[1] = 1 . . 19: . . 20: // F(i) = F(i-1) + F(i-2) 260ms 260ms 21: for i := 2; i <= n; i++ { 2.05s 2.05s 22: v[i] = v[i-1] + v[i-2] . . 23: } . . 24: . . 25: // F(n) - return the n-th Fibonacci number . . 26: return v[n] . . 27:}
A better comparison
A better way to compare the two methods, and the theory to practice, is this:
- Knowing that the exponentialFibonacci method is O(2^n), it would take approximately 2^25 = 33554432 instructions to calculate the 25th Fibonacci number.
- Linearly, calculating the 33554432th Fibonacci number should take roughly the same time as calculating the 25th number exponentially.
So following the methodology above we do this:
- Build the application using the exponentialFibonacci(25) call.
- Start the application.
- Start the Apache Benchmark for 1,000,000 requests.
- Start the CPU profile for 30s seconds.
We get this:
(pprof) top5 98.27s of 99.02s total (99.24%) Dropped 64 nodes (cum <= 0.50s) Showing top 5 nodes out of 30 (cum >= 1.30s) flat flat% sum% cum cum% 60.78s 61.38% 61.38% 60.78s 61.38% main.exponentialFibonacci 24.54s 24.78% 86.16% 24.54s 24.78% fmt.(*pp).doPrintf 12.95s 13.08% 99.24% 12.95s 13.08% syscall.Syscall 0 0% 99.24% 1.30s 1.31% bufio.(*Reader).ReadLine 0 0% 99.24% 1.30s 1.31% bufio.(*Reader).ReadSlice
Now for the second part:
- Build the application using the linearFibonacci(33554432) call.
- Start the application.
- Start the Apache Benchmark for 1,000,000 requests.
- Start the CPU profile for 30s seconds.
We get this:
(pprof) top5 49280ms of 49870ms total (98.82%) Dropped 92 nodes (cum <= 249.35ms) Showing top 5 nodes out of 29 (cum >= 470ms) flat flat% sum% cum cum% 28650ms 57.45% 57.45% 44400ms 89.03% main.linearFibonacci 15660ms 31.40% 88.85% 15660ms 31.40% runtime.memclr 3910ms 7.84% 96.69% 3910ms 7.84% runtime.usleep 590ms 1.18% 97.87% 590ms 1.18% runtime.duffcopy 470ms 0.94% 98.82% 470ms 0.94% runtime.mach_semaphore_timedwait
As you can see, the flat percentages, which is how much of the time was spent in the routine itself, is roughly the same. 61.38% vs 57.45%, it's about 4% difference between them.
Profiling memory
Using the same process, you can run the following command to profile memory:
go tool pprof -alloc_objects myserver http://localhost:8080/debug/pprof/heap
If you run a top command you should see something like this:
(pprof) top10 9741685 of 9927382 total (98.13%) Dropped 7 nodes (cum <= 49636) Showing top 10 nodes out of 33 (cum >= 99079) flat flat% sum% cum cum% 3182489 32.06% 32.06% 3182489 32.06% net/textproto.(*Reader).ReadMIMEHeader 2050835 20.66% 52.72% 2050835 20.66% context.WithCancel 1068043 10.76% 63.47% 8447075 85.09% net/http.(*conn).readRequest 675175 6.80% 70.28% 5155947 51.94% net/http.readRequest 667729 6.73% 77.00% 667729 6.73% net/url.parse 655370 6.60% 83.60% 1414760 14.25% main.handler 618866 6.23% 89.84% 618866 6.23% main.linearFibonacci 589833 5.94% 95.78% 589833 5.94% net/textproto.(*Reader).ReadLine 134266 1.35% 97.13% 172250 1.74% net/http.newBufioWriterSize 99079 1% 98.13% 99079 1% sync.(*Pool).pinSlow
Conclusion
Now that you've seen the basics on how to profile your Golang web apps, you can start diving into heavier stuff like this. Take some time and run a profile on your own Golang web apps.
Also, you should see the Gophercon talk I mentioned at the start of this post, it's quite good.