nil}2018-02-07T00:00:00-09:00{"modified_time"=>nil}" />

Blog

Programming: 20 Years In
Andrew McKnight – 7 February 2018

As I near 20 years of programming, here are the languages and tools I’ve used most intensively over my career, with many overlapping or making repeat appearances:

not to mention all the forays into stranger realms.

In my experience, developing on Apple platforms offers the most advanced and innovative tools and languages I’ve used. I only scratched the surface with Visual Studio, but many people I’ve worked with, even on iOS/Mac, see it as the most powerful (and .NET is a huge framework). Also, honorable mention for Jetbrains’ IDEs, which seemed very powerful the relatively small amount of times I’ve used them.

In the end, Apple has brought together the largest collection of diverse and powerful tools and made them accessible in the most mainstream way. Of course, many many powerful tools are at programmers’ disposal, obviously too many to cover in a blog post: valgrind, Hopper, dependency managers, the various tools in all the other IDEs mentioned above and all their competitors, and even Apple’s Instruments and other static and runtime analysis tools in Xcode. The purpose of this post is to talk about the tools that help programmers produce actual code and how they interact and coevolve with our psychology, and what I’d like to see next. And of course, all through the lens of Apple technologies–in this case, Xcode and Swift Playgrounds for iPad; there’s too much good stuff out there and I’m most familiar with Apple tooling these days.

Paradigms

Of course, this is not to diminish the breakthroughs that each generation of tools have achieved. As the complexity of software has grown, so have new tools and methods been developed to cope with that complexity. This holds true for leaps from punchcards (numbering and patches) to assembly to the high level languages we use today. Within modern languages, we’ve seen several high-level paradigms emerge: imperative, procedural, declarative, functional, logic/object/aspect oriented, reflective, event-driven and constraint-based (see this excellent overview and other introductions).

Developing on Apple platforms offers a chance to use many of these paradigms. You can write assembly and debug it, even that of a program written in higher level languages. Smalltalk brought objects to the procedural world of C to create Objective C. Later ObjC and now Swift have closures to compose functional programs, and libraries like ReactiveCocoa brought event-driven concepts into the mix. Hints of AOP lurk in Cocoa, with things like the UIAppearance proxy, and with Objective C swizzling and libraries like Aspects, you can cut across any dimension of your app. In general, the Objective C runtime allows for introspection of your program as it runs. And while you can even write declarative code with KVC, NSPredicate or SQLite, you’ll need to write it all inside string literals. AutoLayout is a constraint-based paradigm, and even has its own “Visual Format” DSL, also coded in string literals.

Other more specialized concepts exist in various languages: [un]checked exceptions, generics, optionals and concurrency are a few. All of these are at your disposal in some fashion in Objective C and Swift–both have exceptions, unchecked and checked (respectively). Limited generic-like syntax exists in Objective C with id<Protocol>, and the later collections generics emerged for compatibility with Swift’s much more robust generics system. A rich concurrency ecosystem offers POSIX threads, NSThread, run loops, GCD and NSOperation, along with POSIX mutexes, @synchronized and several types of locks. Probably one of Swift’s most notable features is its first-class optionals with its own syntax, a concept backported to Objective C as nullability macros whose confusing helpfulness remind me of const-correctness in C++.

Domain Specific Languages

Beyond Swift and Xcode, there are many problems where it might make sense to use a specialized language well-suited to describe the problem and solutions. Many are still built on the high-level general purpose languages of today, like C/C++. Neural Parallel Language (NPL) is one such example, and the Simplicity [PDF] blockchain language is another. Another interesting component of Simplicity is that its authors formally verify their claims with Coq (formal verification and automatic proving are a fascinating world I have only just begun to explore, with other entrants like Julia and ML). With DSLs built on top of C++ and the ability to interface with it, our general purpose high-level languages of today could be the glue that holds together the programs of tomorrow written in more specialized dialects.

Plaintext++

In theory, all of this can be done using nothing more than a text editor and a terminal, however unpleasant it might be. Consider the most notable difference when switching from an IDE to plaintext: no more syntax coloring! This simple enhancement allows much faster visual scanning of source code. But it doesn’t preclude you from understanding that code if it goes away. Syntax coloring, however, is just the beginning of the journey towards powerful IDE-assisted programming.

IDE Assistance

For all its faults, Xcode is a remarkable piece of software that I fell in love with right away. It can jump to definitions and its Assistant editor can show sub/super/sibling classes, callers/callees (entire call hierarchies are navigable in the Find navigator), includes and including files (for ObjC), associated IB files, and preprocessed or assembled versions of source. It has a Version editor allowing side-by-side views of how a file has changed over time, as well as blame and log views, and you can git-blame any line of code via contextual menus.

Another very powerful tool IDEs allow us to easily wield is refactoring. Find and replace can do many things, as can regular expressions, but program code can contain all sorts of sneaky gotchas that you can easily miss with a search, but are obvious to a compiler. Sadly, Xcode’s refactoring is among the worst I have used in a modern IDE, with IntelliJ/Netbeans/Eclipse having the best–must be something about that static typing in Java!

Features like these help give developers comprehensive understanding of ever more complex codebases and enable them to easily change widespread structural components. As such, they may be indispensable parts of many’s development process. As IDEs become more tightly integrated with the languages they serve, less language features–manifested as written source code–are required to uphold the same structures. I worry that programs’ source code may become incomprehensible if viewed without the required IDE.

Metaprogramming

Is the metaprogramming of today the programming of tomorrow? Was writing Fortran or Smalltalk like metaprogramming assembly, just like that must have been to punchcards? Is evolution of programming languages related to the abstraction level of code generation, and does that abstraction level continue on until you just say “give me a program that makes a broken cat gif resembling Nicholas Cage and posts it to my Twitter every day.” (I assume that by the time we reach this capability, all problems actually worth solving will be solved, people will be freed from the shackles of laboring for the bourgeoisie and live utopian lives of leisure.)

Metaprogrammatic code generation exists in Objective-C in many ways: as @property automatic setter/getter synthesis, and more recently as generated headers for Swift interfaces. You can also generate class definitions from Core Data model descriptions. Swift itself has numerous examples of automatic code generation, like Objective C bridging itself, with a more recent example being Codeable. The community is usually ahead of the institution, and the same holds true for Swift. Sourcery, a tool I admittedly have not explored, enjoys a lot of popularity today. All of these (except maybe Sourcery) happen invisibly at some point during compilation–they don’t leave code behind for you to read or check into source control (at least not easily or by default–things like Core Data class generation do have options to make generated code more accessible, and you can find generated Swift-to-ObjC headers in Derived Data).

Besides code generation itself, Swift has an abundance of keywords and constructs that encapsulate complex logic. lazy, defer, didSet and associatedType introduce effects that are removed from the place in code they appear, some more than others. Other concepts like protocol composition, generics, optionals, anonymous block parameters and tuple members and keypaths may combine to create extremely complicated code. Pattern matching is so powerful, it seems harder to me to know whether or not using an enum is appropriate. map can be used to transform a collection or unwrap an optional (which is a collection), whereas flatMap is like map followed by filtering for nonnil elements.

With so much “invisible code” via high-level abstractions and just-in-time code generation, source code becomes hard to reason about. Working memory [PDF] can only hold around 7 separate ideas in mind at once: maybe an overarching goal of a feature or bug, the current plan of attack, a couple intricately connected structures in code like aspects, a couple automatic code generation processes, and it’s not hard to imagine writing a crasher because you forgot to check if some API was introduced after your deployment target, or because a data structure has some funny interplay between ObjC/Swift you didn’t know about or forgot. It’s difficult to do in Xcode; to do so with TextEdit + Terminal seems impossible.

Visual Programming

What we are going to need are new ways to organize program specifications besides text, or at least novel ways to use text to describe complicated structures, that allow us to efficiently work within the limits of our working memory. Programmable Logic Controllers (PLCs) have five languages, two text-based and three graphical. Ballerina has equivalent graphical and textual syntaxes. For lovers of the abstract, Piet programs are more like pictures than code.

Probably the most well known visual programming tool is the WYSIWYG editor. Xcode’s is called Interface Builder (IB) and is the most advanced I’ve used. Individual views and view elements can be defined in separate files, and Storyboards (the only integrated tool of its kind that I have seen in an IDE) can define multiple views and the interactions between them, and reference other Storyboards–really, they are composable state machine diagrams. These days, I do not use IB because it is very hard to maintain the backing XML–I’d rather maintain the Swift code and derive reusable code utilities to build UI. (Although lately my usage of “flow controllers” are seeming an awful lot like storyboards-in-code… maybe one day the pendulum will swing back and I’ll again use IB!)

Xcode has many other GUI-based editors for documents backed by plaintext code. The Core Data visual data model editor complements the series of dropdown menus and text boxes in the normal model editor. That mode is similar to the visual Plist editor, the default way to view a Plist, even though you can also view it as its XML source (as can any IB file, or even the .xcodeproj itself). It also has SpriteKit and SceneKit editors (for 2D and 3D worlds, respectively), which I unfortunately have zero experience with. The build settings editor is backed by a powerful text-based configuration system.

Besides Xcode, Apple has several other visual programming tools. Automator organizes individual tasks and snippets into flows. Quartz Composer arranges tasks into nodes that can be connected together in a way very reminiscent of how IBOutlets are connected to objects in IB. Clearly these tools are not new or one-off explorations, but they also don’t feel mainstream in the way text-based code still reigns supreme.

ASCII Art

Looking back at text-based code, some options exist to get more graphical, even if they’re just glyphs like + that can replace wordy functions like add(). Swift allows identifiers containing Unicode characters, so I could make a new function (not operator, unfortunately, in Swift) named ➕. Specialized fonts exist with ligatures to pictorialize common operators like !=, <=, -> and ===. See this hackernews discussion. You can even name that nasty workaround hack function with zalgo text to scare off the weak-willed.

In addition to changing what glyphs we use to compose code, we can also overlay new information on top of the code. Things like highlighting all instances of an identifier, or showing compilation warnings and errors where they occur. Swift Playgrounds for iPad introduced some new concepts for decorating and performing simple refactors on control structures like switch and if/else, as well as functions. This seems like the tip of the iceberg of what is possible, but is a great start, and I think works especially well for learning. I’ve very limited use with Playgrounds for iPad, mostly solving specific algorithmic problems, which is a delight with the graphical REPL: I usually write a function, pass in a set of inputs, and then graph the outputs to see if it makes sense. (REPLs are another instance of something that has been around for a while, but Apple has taken to a new level.) Although my usage has been simple, people build up and tinker with entire UI frameworks using playgrounds.

One thing I’d love to see is the ability to annotate lines of code by different responsibility, like logging, analytics, error handling or business logic. Then, the ability to “fold all” of a particular type while viewing a code file. Essentially, defining different cross-sections or dimensions of code, with the ability to hide types of code you don’t care about at any given time. Logging and analytics can really clutter up business logic, but I’d prefer to keep it all together and take different “views” on it instead of using more magical aspect oriented solutions, from a maintenance perspective. Sort of like those anatomy books with a bunch of clear pages that show each organ system overlaid on a skeleton, code editors could also color or decorate each “type” of code differently while they’re visible.

Conversational Code

Using the new source editor in Xcode 9, I find myself going back and forth with the swift compiler for trivial things like where to put all the ?s, or no longer needing Void in a closure’s parameter list. It would be nice if SiriKit could be used so that I could just talk with` the computer, which is much faster/easier than keystrokes, which are faster than mousing/clicking on the little radio buttons for issues in the source editor. There are folks out there today who dictate all their code.

Onwards

It’s been a lot of fun, first learning to program, and then seeing all of the amazing tools people have created to build programs. As I like to do, it’s exciting to take the progress I’ve seen over the last two decades and extrapolate it into the future (with an exponential growth curve of course). As new software, paradigms, and even hardware are invented, creating applications could become as different from what it is today as IB + Swift is from punchcards–what will that look like?

🙏🏻 Enjoy the post? Please help support more like it by buying me a cup of coffee!.

💡 Suggestions, questions, comments? Please submit a PR!.