AppleTV Development – OpenSSL and libcurl

In doing development for the new AppleTV, I was unable to find a cross compiled version of OpenSSL and cURL (libcurl) to use for my project.   I found a helpful OpenSSL script (here) that addressed some the unique tvOS challenges (e.g. fork() is not available) and used that build script to construct an libcurl build script.

The following repo contains scripts to build OpenSSL and cURL (libcurl) for OS X, iOS and tvOS with Bitcode enabled for iOS, tvOS. It includes the required patching for tvOS to not use fork(). This requires Xcode7.1b or later for the tvOS SDK.

https://github.com/jasonacox/Build-OpenSSL-cURL

 

DevOps Enterprise Summit 2015

It was great to attend the DevOps Enterprise Summit again this year. The 2015 edition saw more than double the number of attendees of the 2014 conference with presentations from companies all over the world. There is definitely a feeling that DevOps is awakening across the enterprise.

I had the privilege of presenting again on Disney’s Journey,
“Disney DevOps – The Enterprise Awakens.”

Fellow DevOps Avengers from all over the world converged to swap stories, share new insight, technology and encourage each other to keep moving forward as the Force of positive change in our various industries.

DevOps Enterprise | The Agile, Continuous Delivery and DevOps Transformation Summit http://devopsenterprise.io

Some great reviews and observations:

Impressions from DevOps Enterprise Summit 2015

Gene Kim and Others Share What DevOps is Really “All About”

Impressions from DevOps Enterprise Summit 2015 – Micro Hering – Accenture

Infoworld – Gene Kim explains the joy of DevOps

Selection of favorite quotes:

  • “If you name your servers and treat them like pets they all develop individual personalities.” – @jasonacox
  • “Times and conditions change so rapidly that we must keep our aim constantly focused on the future.” – Walt Disney
  • “If you don’t know where you’re going, any road will take you there.” –@jasonacox
  • “Believe in what you do, and make a difference” – @jasonacox

DevOps Enterprise Summit 2014

I had the privilege of speaking at the first annual DevOps Enterprise Summit.

Jason Cox - DOES 2014DevOps Enterprise | The Agile, Continuous Delivery and DevOps Transformation Summit http://devopsenterprise.io

Wired Magazine – Web Insights
DevOps Innovation Takes Place in the Enterprise by Steve Brodie

Redhat Developer Blog
DevOps Enterprise Conference — Day One

Electric-Cloud
DevOps Enterprise Summit Puts Agile Transformation Front and Center
What we learned at DevOps Enterprise Summit 2014

Software Development Times
DevOps Enterprise Summit bashes silos

PuppetLabs Blog
Disney’s DevOps Journey: A DevOps Enterprise Summit Reprise

Tweets
http://seen.co/event/devops-enterprise-the-agile-continuous-delivery-and-devops-transformation-summit-san-francisco-airport-marriott-waterfront-2014-949/highlight/38769

Selection of quotes:

  • “You’ve got to pick technology that transforms. You need to find tools that change the way you think.” – Jason Cox from Disney
  • “It’s all about the people! We have to empower the edge.” – Jason Cox from Disney at #DOES14
  • #DOES14 Jason Cox of Disney: Remove waste. (Motion != Work)
  • Jason Cox #DevOps guy at @disney promotes positive rebellion. Red tape removers! #DOES14
  • “DevOps professionals must be courageous. They must have the candor to say that there are issues and be renegades enough to constantly challenge the status quo.” – Jason Cox

 

The Phoenix Project

phoenixprojectThe Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win

by Gene Kim, Kevin Behr, & George Spafford

Digital transformation has lit a fire that is burning through traditional IT shops across the world telling us to adapt or be reduced to the ash heap of obsolescence. The Phoenix Project grapples with this disruptive change by telling an engaging story about an IT manager who is thrust into this fiery turmoil.  The main character is ushered unwillingly and unprepared into a new leadership role where he uncovers the complex and unrelenting problems any IT shop knows all too well.  Fair warning: If you have any history working in, leading or managing IT teams, you will likely have a visceral reaction to the narrative and even suspect the authors were spying on your organization.  But there is hope!  The main character in the story finds enlightenment and begins to implement changes based on the “Three Ways” discussed in this book that ultimately transforms the teams, balances life for the employees and successfully propels the business forward.

IT leaders will find the narrative and advice relevant, potent and inspiring.  By applying the principles learned in this novel, your technology organization, like the phoenix of old, can rise from the ashes to take on the challenges of our new Digital age.


There is hope!  Technology workers and leaders will find the narrative and advice woven throughout The Phoenix Project, relevant, visceral, potent and inspiring.  By applying the principles learned in this novel, business and technology leaders will see how they can transform their own organizations, identify and break down unnecessary silos, improve life for technology workers, and successfully propel their businesses forward in this new Digital age.


 

Management Lessons From a Controversial Genius

leadingappleLeading Apple With Steve Jobs: Management Lessons From a Controversial Genius

by Jay Elliot

I happend to find this book while surfing through my Safari Online account.  This is a great perspective from someone who worked alongside Steve Jobs and experienced his leadership style and philosophy.

There are several great Quotes from Steve Jobs in this book and Jay uses some of the best as chapter leads:

  • I have looked in the mirror every morning and asked myself: “If today were the last day of my life, would I want to do what I am about to do today?” And whenever the answer has been “No” for too many days in a row, I know I need to change something. [Chapter 1]
  • Being the richest man in the cemetery doesn’t matter to me. Going to bed at night saying we’ve done something wonderful, that’s what matters to me. [Chapter 2]
  • A lot of people in our industry haven’t had very diverse experiences. So they don’t have enough dots to connect, and they end up with very linear solutions without a broad perspective on the problem. The broader one’s understanding of the human experience, the better design we will have. [Chapter 3]
  • A small team of A-players can run circles around a giant team of B- and C-players. [Chapter 4]
  • It’s building an environment that makes people feel they are surrounded by equally talented people and their work is bigger than they are. The feeling that the work will have tremendous influence and is part of a strong, clear vision. [Chapter 6]
  • We used to dream about this stuff. Now we get to build it. It’s pretty great. [Chapter 7]
  • Design is the fundamental soul of a man-made creation that ends up expressing itself in successive outer layers of the product or service. The iMac is not just the color or translucence or the shape of the shell. The essence of the iMac is to be the finest possible consumer computer in which each element plays together. [Chapter 8]
  • My job is to not be easy on people. My job is to take the great people we have and to push them and make them even better, coming up with more aggressive visions of how it could be. . . . My job is to make the whole executive team good enough to be successors, so that’s what I try to do. [Chapter 9]
  • You can’t talk about profit; you have to talk about emotional experiences. [Chapter 10]
  • I’m actually as proud of many of the things we haven’t done as the things we have done. [Chapter 11]
  • We don’t know where it will lead. We just know there’s something much bigger than any of us here. [Chapter 13]
  • It’s not just a job, it’s a journey. Let’s never forget that. . . . Your customers dream of a happier and better life. Don’t move products. Instead, enrich lives. [Chapter 14]

Lessons I picked up from this Book:

1. Think Different

Here’s to the crazy ones, the misfits, the rebels, the troublemakers, the round pegs in the square holes . . . because the ones who are crazy enough to think that they can change the world, are the ones who do. -Apple Ad

It’s important not to settle into a stagnant mechanical state.  Be the one to change things.  Create constructive discomfort and challenge the status quo.  When starting on work for the Mac, Steve wanted to create a team of “Pirates, not the Navy”.  In Chapter 4, Jay says:

You want people who dare to be different! You want Pirates—where the skull and crossbones may not be part of the company’s symbol, but they’re Pirates nonetheless. People who take risks, live at times a little on the edge, flaunt rules when justified, laugh loudly as the wind lashes their face and their pursuers fade from view into the distance behind. I always want to work with people like this, and so should you. 

Whether you think of them as nonconformists, dissenters, rebels, pirates, nutters, positive deviants, or as Blair’s crazies, make sure your team has a solid sprinkling of them. They will challenge your thinking, fuel your ideas, pump up your momentum, boost your competitive edge, and quite simply make your business a winner. And make sure you yourself provide a dose of this magic on occasion. You’re unlikely to accomplish anything great in your career without it.

 

One of the things that happens in organizations as well as with people is that they settle into ways of looking at the world and become satisfied with things. And the world changes and keeps evolving and new potential arises but these people who are settled in don’t see it. – Steve Jobs

2. Stay Focused on the Product

Product, not Profit or Process:  Businesses usual start out focused on product, their great idea to change the world.  The tendency is slip from product to profit and process focus.  Steve and others like him (Walt Disney included) spent all their energy thinking about making something great, not making a great profit.  As we have seen, those that keep that product perspective often end up with healthy profits as well.

Design with the customer in mind:  Steve drove product design by keeping the end user in mind, not just surface usage, but emotional connection as well.

You can’t talk about profit; you have to talk about emotional experiences. – Steve Jobs

Design for excellence.  Design to create an emotional experience, not just utility.  No more crap products.

In chapter 7, Jay says:

The reason for the outpouring of emotion around the world at the time of Steve’s death was that his products had touched people emotionally. Money and profits were never the motivators for Steve. His motivation was, “I’m going to build a product for myself, a product that will fill a need and at the same time give me pleasure to use.

It’s not just a job, it’s a journey. Let’s never forget that. . . . Your customers dream of a happier and better life. Don’t move products. Instead, enrich lives. – Steve Jobs

steve jobs

Raspberry Pi – AirPiano

Wouldn’t it be cool if you could control your MIDI equipped digital piano from your iPhone?  Imagine a player piano capable of jukebox queuing up MIDI or Audio files and playing them in order.  That was my goal!  RPI to the rescue.

RaspiModelB

Required Ingredients:

  • Raspberry Pi – Model B, 512MB RAM, 16GB SD Card, Raspbian Linux, Apple 12W USB Power Adapter, USB WiFi Dongle
  • USB MIDI Digital Device (e.g. Yamaha Clavinova CLP-440 with USB MIDI port and RCA L+R Audio Inputs)
  • Software:  Apache HTTP Server, PHP, MySQL, (Nice to have:  Netatalk file server, Shairport AirPlay server)

Instructions:

My original thought was to turn the Yamaha speakers into a AirPlay device for our many iOS devices.  It’s nice to pipe in background tunes without having to hook up a device.  AirPlay is great way to do that but I didn’t want to spend $100+ to be able to do that.  The Raspberry Pi is more than capable of doing this. I found an extremely helpful blog post here:  http://www.raywenderlich.com/44918/raspberry-pi-airplay-tutorial

NOTE: Power is a big issue for the Raspberry Pi.  If you are using a WiFi dongle or any other USB device, I recommend making sure your power supply has enough kick to keep the RPI going.   I started out with a microUSB adapter that advertised 700mA but performance became very unstable, especially under load.  I switched to a 12W Apple adapter that I had handy and the problems disappeared.   The RPI FAQ recommends 1.2A (1200mA).

Setup for AirPiano – MIDI Control

The Project Code: https://github.com/jasonacox/raspberrypi

  • MySQL: Database ‘piano’ – see piano.sql file
  • Dequeue Service: Run the cron.sh script to have the RPI scan for new midi or wave files to play. Run it with:
bash -x cron.sh 0<&- 1>/dev/null 2>/dev/null &
  • Apache: Install apache http with mod_php and mysql support
  • Website Code: Install index.php, setup.php and the folder.png files into the document root of your webserver. Upload the MIDI (*.mid) and WAVE (*.wav) files to this location. Be sure to update $globalBase in setup.php to the folder where these files are located.

 

AirPlay via Raspberry Pi

Adam Burkepile has the best tutorial I’ve found on how to set up a RPI for AirPlay:

http://www.raywenderlich.com/44918/raspberry-pi-airplay-tutorial

Apple Share (AFP) Server via Raspberry Pi

The RPI is great for a tiny file server!  At the least, having it run an AFP server will allow quick drag and drop transfer from your Macs.  A Samba service for Windows shares is easy to set up as well.

sudo apt-get install netatalk

Yes, it is that easy.  Finder will now show your Raspberry Pi under “shared” along with any other local network shares:

Screen Shot 2013-12-27 at 11.22.08 PM

 

 

 

 

iCurlHTTP – cURL for iPhone and iPad

iCurlHTTP

HTTP Server Response Diagnostic Tool

I love cURL.  Systems engineers, developers, administrators, QA engineers and web operations specialist everywhere have used the curl tool to help troubleshoot web server responses.  It gives a quick view into exactly what is being sent from a web server to a browser.

On a commute home from work I started to think how handy it would be if the small computer I was holding, my iPhone, had a simple to use app like curl to do the same troubleshooting.  I always wanted to write my own iPhone app so I started researching how I could do that.

My Xcode Adventure

I fired up Xcode on my Snow Leopard equipped MacBook Pro for the first time.  I found a few great iOS development books on SafariOnline and started learning Objective-C.  After a few chapters I started going off the page and adding views and controllers.  I wanted to deploy my test apps to my iPhone so that required that I sign up as an Apple Developer (and $100!).  I soon discovered that my Xcode version was too out of date to work with my iOS 6.1 devices so I attempted to upgrade, only to discover that I needed to upgrade my OS.   Now on to Xcode 4.6 and suddenly my projects were running on my iPhone and iPad.

My libcurl Adventure

I discovered the NSURL class and started trying to get raw responses from web servers to display to the textview.  It was too digested and would require a lot of effort to simulate a raw curl session.  I started looking at the NSStream class when a friend suggested that I investigate a precompiled version of libcurl (the cURL library from Daniel Stenberg) for the ARM processor.  Instead, I found a great post by Nick Zitzmann where he shows how he compiled libcurl and even provides a binary: http://seiryu.home.comcast.net/~seiryu/libcurl-ios.html

I managed to get the latest version of libcurl to compile and also included the Apple SecureTransport library (–with-darwinsll in libcurl config):

#!/bin/bash
cd curl-7.29.0
export IPHONEOS_DEPLOYMENT_TARGET="4.3"
export CC="/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/llvm-gcc-4.2"
export CFLAGS="-arch armv7 -arch armv7s -pipe -Os -gdwarf-2 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS6.1.sdk"
export LDFLAGS="-arch armv7 -arch armv7s -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS6.1.sdk"
./configure --disable-shared --enable-static --with-darwinssl --disable-dependency-tracking --host="armv7-apple-darwin" --prefix=~/Code/curl/device
make -j `sysctl -n hw.logicalcpu_max`
make install

I added the libcurl.a library to my Xcode project and started using the easy interface API via CURLcode curl_easy_perform(CURL * handle ). The realtime feel of iCurlHTTP comes from the use of the CURLOPT_WRITEFUNCTION option to push realtime code into a view controller method that renders to the textview object. It required  periodically having the thread give up time to the mainRunLoop (via message to NSRunLoop) in order to allow the display to render and respond to user events.

iCurlHTTP v1.0 Born

iCurlHTTP was born.  It was a simple, easy to use iOS app that allowed you to run simple cURL like tests, simulating different web browsers to retrive the raw HTTP headers and HTML response from the web servers.

icurlhttp-v1.0

OpenSSL Anyone?  iCurlHTTP v1.1

The SecureTransport library works great but provided less details on SSL connections than what I was accustomed to seeing with curl.  I wanted more x509 decoded information, specifcially SSL certificate details about expiration date.  The OpenSSL library would allow for this so I began another search to get  OpenSSL library for ARM7.  To get what I wanted, I cross-compiled both OpenSSL and libcurl together. With that I was able to get the great openssl detail in the output, including cert info:

icurlhttp-cert

I managed to cross compile libcurl and openssl separately for armv7, armv7s and i386 architectures but later found a great GitHub project script created by Miyabi Kazamatsuri that does both.  I forked his code and updated it to work with Xcode 5 (clang for iPhoneOS and iPhoneSimulator) and added the 64-bit arm64 and x86_64 architectures.  The script is available on my GitHub:  https://github.com/jasonacox/cURL-SSL-for-iOS

UPDATE: I later put together a much better build script for openssl+libcurl that creates libraries for Mac, iOS and tvOS (Apple TV) targets: https://github.com/jasonacox/Build-OpenSSL-cURL

HTTP Timing and iOS 7 support added to iCurlHTTP v1.2

Curl has the ability to provide timing data for DNS lookup, TCP Connect, SSL Handshake, First Byte and Total time.  For example…

curl -k -s -o /dev/null -w '%{time_namelookup} %{time_connect} %{time_appconnect} %{time_pretransfer} %{time_redirect} %{time_starttransfer} %{time_total}n' https://jasonacox.com

I wanted to add this to iCurlHTTP to give a mobile perspective on these numbers.  In v1.2, I added this in the detail response view and as timing data points listed along the bottom of the UI.

icurlhttp-timing3

The v1.2 version also includes the iOS 7 SDK updates and the new minimalist UI look and feel.

iCurlHTTP 1.3 Wish List 

Thanks to great feedback, I captured several feature requests from the iCurlHTTP user base that became the goal for version 1.3:

  1. Custom Headers – Allow more and customized browser emulations via a customized option.
  2. Print, Copy or Email Output – Add a one-button feature to send the output to the clipboard, printer, email, post, etc.
  3. Customized Post – Allow customized post data to simulate a POST form submit.

This meant I needed to be able to have a User Settings pop-up screen to allow users to enter custom fields for headers and posts.  I also wanted to allow the ability to have a custom user-agent, http authentication (user/password), SSL options and possibly others in the future.

I knew I wanted a table based form to allow for formatting, scrolling and auto-positioning.  I considered using the Settings bundle but I needed multi-line UITextView input (especially for custom headers).  There are several great open source solutions  but my favorite version was FXForms, written by Nick Lockwood  (https://github.com/jasonacox/FXForms).  I used a simple plist for settings persistence.

icurlhttp-settings

The following features are added in version 1.3:

  • Share Feature – Send output to Clipboard, Printer and Email
  • User Settings – Set Custom User-Agent, Custom Headers (replace defaults or add new ones), POST Data, Authentication and SSL Mode selection
  • Updates – New libcurl and openssl libraries.

iCurlHTTP 1.4 Enhanced History

The goal of the 1.4 update was to enhance the URL dropdown history by adding HEADER and POST data for easy repeated testing.  I used simple (h) and (p) symbols to denote header and post respectively:

Screen Shot 2016-03-05 at 8.53.39 AM

iCurlHTTP 1.5 Redirect

I had several requests to add a “follow redirect” features (similar to curl -L command) to make it easier for users to test.  I didn’t want this to be purely  automatic because it would defeat the purpose of the granular detail that cURL provides so I present the user with a popup dialog when a 301 or 302 redirect response is received and list the redirection URL they can follow.  In this version, I also replace the ugly text “User” (settings) and “Share” buttons with the well known icons to help make it more intuitive.

Screen Shot 2016-03-05 at 9.30.53 AM

I’m amazed at how this small and simple project continues to grow (both in users as well as feature requests).  It has been a fun adventure! I frequently recommend other try building their own apps.  iCurlHTTP is written in Objective-C but I would definitely recommend trying out Swift.  I’m in the process of porting iCurlHTTP to the Apple TV (seems ridiculous, I know but I have to do it just for fun) and hope to try my hand at Swift.

iCurlHTTP 1.6 & 1.7 – HTTP/2.0 and IPv6 Update

For those wanting the libcurl, openssl and nghttp2 (HTTP2) libraries for iOS or tvOS (AppleTV), I have posted a build script on github: https://github.com/jasonacox/Build-OpenSSL-cURL

Thanks to request from users, I added the following features:

  • Certificate Chain Details for HTTPS Sessions (Detail Mode)
  • Support for Authentication Credentials in URL (e.g. https://user:pass@jasonacox.com/gettest.php)
  • DNS Resolve Spoof Option for Manual Address Resolution (eg. HOST:PORT:ADDRESS)

iCurlHTTP 1.8 & 1.9 – Performance Improvements

  • Performance Improvements and Reported Bug Fixes
  • Added user defined DNS lookup & connection timeout setting (default 5s)

THANK YOU!

Thanks for your great feedback and support!  I really appreciate the kind words and critical advice.  Please send more!

iCurlHTTP Website:  www.jasonacox.com/icurlhttp

Download on the App Store

 

Reference and Thanks!