Initial start of documenting the ABI.

Author: davidchisnall

ABIDoc/abi.tex

@@ -0,0 +1,226 @@
+\documentclass[a4paper]{report}
+
+\usepackage[utf8x]{inputenc}
+\usepackage{listings}
+\usepackage{fancyref}
+\newcommand*{\fancyreflstlabelprefix}{lst}
+\fancyrefaddcaptions{english}{%
+  \providecommand*{\freflstname}{listing}%
+  \providecommand*{\Freflstname}{Listing}%
+}
+\frefformat{plain}{\fancyreflstlabelprefix}{\freflstname\fancyrefdefaultspacing#1}
+\Frefformat{plain}{\fancyreflstlabelprefix}{\Freflstname\fancyrefdefaultspacing#1}
+
+\frefformat{vario}{\fancyreflstlabelprefix}{%
+  \freflstname\fancyrefdefaultspacing#1#3%
+}
+\Frefformat{vario}{\fancyreflstlabelprefix}{%
+  \Freflstname\fancyrefdefaultspacing#1#3%
+}
+
+
+\usepackage[svgnames]{xcolor}
+\usepackage{hyperref}
+
+\lstset{
+    basicstyle={\footnotesize \ttfamily},
+    breaklines=true,
+    commentstyle={\color{Blue}},
+    extendedchars=true,
+    keywordstyle={[0]\color{Green}},
+    keywordstyle={[1]\color{Brown}},
+    keywordstyle={[2]\color{DarkMagenta}},
+    keywordstyle={[3]\color{Maroon}},
+    keywordstyle={[4]\color{Blue}},
+    showspaces=false,
+    showstringspaces=false,
+    stringstyle={\color{IndianRed}},
+    tabsize=2,
+}
+
+
+\newcommand{\file}[1]{\textsf{#1}}
+\newcommand{\keyword}[1]{\textit{#1}}
+\newcommand{\ccode}[1]{\lstinline[language={C}]{#1}}
+\newcommand{\objc}[1]{\lstinline[language={[Objective]C}]{#1}}
+
+\newcommand{\inccode}[4]{
+	\lstinputlisting[language=C,
+	  rangebeginprefix =//\ begin:\ ,
+	  rangeendprefix   =//\ end:\ ,
+	  includerangemarker=false,
+	  linerange=#3-#3,
+	  numbers     = left,
+	  label={lst:#2},
+	  float,
+	  caption={#4 {\small [From #1] }}
+	]{../#1}
+}
+
+
+\title{GNUstep Objective-C ABI version 10}
+\author{David Chisnall}
+
+\begin{document}
+\maketitle{}
+\tableofcontents
+
+\chapter{Introduction}
+
+The GNUstep Objective-C runtime has a complicated history.
+It began as the Étoilé Objective-C runtime, a research prototype that adopted a lot of ideas from the VPRI Combined Object-Lambda Architecture (COLA) model and was intended to support languages like Self (prototype-based, multiple inheritance) as well as Objective-C.
+This code was repurposed as \file{libobjc2} to provide an Objective-C runtime that clang could use.
+At the time, the GCC Objective-C runtime had a GPLv2 exemption that applied only to code compiled with GCC and so any code compiled with clang and linked against the GCC runtime had to be licensed as GPLv2 or later.
+
+GCC's Objective-C support at this time was lacking a number of features of more modern Objective-C (for example, declared properties) and showed no signs of improving.
+
+Eventually \file{libobjc2} was adopted by the GNUstep project and became the GNUstep Objective-C runtime.
+It was intended as a drop-in replacement for the GCC runtime and so adopted the GCC Objective-C ABI and extended it in a variety of backwards-compatible ways.
+
+The GCC ABI was, itself, inherited from the original NeXT Objective-C runtime.
+The Free Software Foundation used the GPL and the threat of legal action to force NeXT to release their GCC changes to support Objective-C.
+They were left with some shockingly bad code, which was completely useless without an Objective-C runtime.
+The GCC team committed the shockingly bad code and wrote a runtime that was almost, but not quite, compatible with the NeXT one.
+In particular, it did not implement \ccode{objc_msgSend}, which requires hand-written assembly, and instead modified the compiler to call a function to look up the method and then to call the result, giving a portable pure-C design.
+
+As such, the ABI supported by the GNUstep Objective-C runtime dates back to 1988 and is starting to show its age.
+It includes a number of hacks and misfeatures that are in dire need of replacing.
+This document describes the new ABI used by version 2.0 of the runtime.
+
+\section{Mistakes}
+
+Supporting a non-fragile ABI was one of the early design goals of the GNUstep Objective-C runtime.
+When Apple switched to a new runtime, they were able to require that everyone recompiled all of their code to support the non-fragile ABI and, in particular, were able to support only the new ABI on ARM and on 64-bit platforms.
+
+At the time, it was not possible to persuade everyone to recompile all of their code for a new GNUstep runtime, and so I made a number of questionable design decisions to allow classes compiled with the non-fragile ABI to subclass ones compiled with the fragile ABI.
+These decisions have led to some issues where code using the non-fragile ABI ends up being fragile.
+
+The new ABI makes no attempt to support mixing old and new ABI code.
+The runtime will work with either, but not with both at the same time.
+It will upgrade the old structures to the new on load (consuming more memory and providing an incentive to recompile) and will then use only the new structures internally.
+
+\section{Changed circumstances}
+
+When the original NeXT runtime was released, linkers were designed primarily to work with C.
+C guarantees that each symbol is defined in precisely one compilation unit.
+In contrast, C++ (10 years away from standardisation at the time the NeXT runtime was released) has a number of language features that rely on symbols appearing in multiple compilation units.
+The original 4Front C++ compiler worked by compiling without emitting any of these, parsing the linker errors, and then recompiling adding missing ones.
+
+More modern implementations of C++ emit these symbols in every compilation unit that references them and rely on the linker to discard duplicates.
+Modern linkers support \keyword{COMDATs} for this purpose.
+
+The NeXT runtime was able to work slightly differently.
+The Mach-O binary format (used by NeXT and Apple) provides a mechanism for registering code that will handle loading for certain sections, thus delegating some linker functionality to the runtime.
+
+In addition to COMDATs, modern linkers support generating symbols that correspond to the start and end of sections.
+This makes it possible for the new ABI to emit all declarations of a particular kind in a section and for the runtime to then receive an array of all of the objects in that section.
+
+\chapter{Entry point}
+
+The legacy GCC ABI provided a \ccode{__objc_exec_class} function that registered all of the Objective-C data for a single compilation unit.
+This has two downsides:
+
+\begin{itemize}
+	\item It means that the \objc{+load} methods will be called one at a time, as classes are loaded, because the runtime has no way of knowing when an entire library has been loaded.
+	\item It prevents any deduplication between compilation units, and so a selector used in 100 \file{.m} files and linked into a single binary will occur 100 times and be passed to the runtime for merging 100 times.
+\end{itemize}
+
+\section{The new entry point}
+
+The new runtime provides a \ccode{__objc_load} function for loading an entire library at a time.
+This function takes a pointer to the structure shown in \Fref{lst:initobjc}.
+
+For the current ABI, the \ccode{version} field must always be zero.
+This field exists to allow future versions of the ABI to add new fields to the end, which can be ignored by older runtime implementations.
+
+The remaining fields all contain pointers to the start and end of a section.
+The sections are listed in \fref{tab:sections}.
+
+\inccode{loader.c}{initobjc}{objc_init}{The Objective-C library description structure.}
+
+The \ccode{__objc_selectors} section contains all of the selectors referenced by this library.
+As described in \fref{chap:selectors}, these are deduplicated by the linker, so each library should contain only one copy of any given selector.
+
+\begin{table}
+	\begin{center}
+		\begin{tabular}{l|l}
+			Prefix & Section \\\hline
+			\ccode{sel_} & \ccode{__objc_selectors}\\
+			\ccode{cls_} & \ccode{__objc_classes}\\
+			\ccode{cls_ref_} & \ccode{__objc_class_refs}\\
+			\ccode{cat_} & \ccode{__objc_cats}\\
+			\ccode{proto_} & \ccode{__objc_protocols}\\
+		\end{tabular}
+		\caption{\label{tab:sections}Section names for Objective-C components.}
+	\end{center}
+\end{table}
+
+Similarly, the \ccode{__objc_classes}, \ccode{__objc_cats}, and \ccode{__objc_protocols} sections contain classes, categories, and protocols: the three top-level structural components in an Objective-C program.
+These are all described in later chapters.
+
+The \ccode{__objc_class_refs} section contains variables that are used for accessing classes.
+These are described in \Fref{sec:classref} and provide loose coupling between the representation of the class and accesses to it.
+
+\section{Compiler responsibilities}
+
+For each compilation unit, the compiler must emit a copy of both the \ccode{objc_init} structure and a function that passes it to the runtime, in such a way that the linker will preserve a single copy.
+On ELF platforms, these are hidden weak symbols with a comdat matching their name.
+The load function is called \ccode{.objcv2_load_function} and the initializer structure is called \ccode{.objc_init} (the dot prefix preventing conflicts with any C symbols).
+The compiler also emits a \ccode{.objc_ctor} variable in the \ccode{.ctors} section, with a \ccode{.objc_ctor} comdat.
+
+The end result after linking is a single copy of the \ccode{.objc_ctor} variable in the \ccode{.ctors} section, which causes a single copy of the  \ccode{.objcv2_load_function} to be called, passing a single copy of the \ccode{.objc_init} structure to the runtime on binary load.
+
+The \ccode{.objc_init} structure is initialised by the \ccode{__start_\{section name\}} and \ccode{__stop_\{section name\}} symbols, which the linker will replace with relocations describing the start and end of each section.
+
+The linker does not automatically initialise these variables if the sections do not exist, so compilation units that do not include any entries for one or more of them must emit a zero-filled section.
+The runtime will then ignore the zero entry.
+
+\chapter{Selectors}
+\label{chap:selectors}
+
+Typed selectors are one of the largest differences between the GNU family of runtimes (GCC, GNUstep, ObjFW) and the NeXT (NeXT, macOS, iOS) family.
+In the NeXT design, selectors are just (interned) strings representing the selector name.
+This can cause stack corruption when different leafs in the class hierarchy implement methods with the same name but different types and some code sends a message to one of them using a variable of type \objc{id}.
+In the GNU family, they are a pair of the method name and the type encoding.
+
+The GNUstep ABI represents selectors using the structure described in \Fref{lst:selector}.
+The first field is a union of the value emitted by the compiler and the value used by the runtime.
+The compiler initialises the \ccode{name} field with the string representation of the selector name, but when the runtime registers the selector it will replace this with an integer value that uniquely identifies the selector (it will also store the name in a table at this index so selectors can be mapped back to names easily).
+
+\inccode{selector.h}{selector}{objc_selector}{The selector structure.}
+
+\section{Symbol naming}
+
+In this ABI, unlike the GCC ABI, we try to ensure that the linker removes as much duplicate data as possible.
+As such, each selector, selector name, and selector type encoding is emitted as a weak symbol with a well-known name name, with hidden visibility.
+When linking, the linker will discard all except for one (though different shared libraries will have different copies).
+
+The selector names are emitted as \ccode{.objc_sel_name_\{selector name\}}, the type encodings as \ccode{.objc_sel_name_\{mangled type encoding\}} and the selectors themselves as \ccode{.objc_sel_name_\{selector name\}_\{mangled type encoding\}}.
+The \textit{mangled} type encoding replaces the @ character with a \ccode{'\\1'} byte.
+This mangling prevents conflicts with symbol versioning (which uses the @ character to separate the symbol name from its version).
+
+This deduplication is not required for correctness: the runtime ensures that selectors have unique indexes, but should reduce the binary size.
+
+\chapter{Classes}
+
+\inccode{class.h}{class}{objc_class}{The class structure.}
+
+\section{Class references}
+\label{sec:classref}
+
+Each entry in the \ccode{__objc_class_refs} section is a symbol (in a COMDAT of the same name) called \ccode{_OBJC_CLASS_REF_\{class name\}}, which is initialised to point to a variable called \ccode{_OBJC_CLASS_\{class name\}}, which is the symbol for the class.
+This is the \textit{only} place where the \ccode{_OBJC_CLASS_\{class name\}} symbols may be referenced.
+
+All other accesses to the class (i.e. from message sends to classes or to \objc{super}) must be via a load of the \ccode {_OBJC_CLASS_REF_\{class name\}} variable.
+
+The current version of the runtime ignores this section, but if a future runtime changes the class structure then it can update these pointers to heap-allocated versions of the new structure.
+
+
+\chapter{Categories}
+
+\chapter{Protocols}
+
+\chapter{Message sending}
+
+\end{document}
+

class.h

@@ -37,6 +37,7 @@ static inline BOOL objc_bitfield_test(uintptr_t bitfield, uint64_t field)
 }
 
 
+// begin: objc_class
 struct objc_class
 {
 	/**
@@ -131,6 +132,7 @@ struct objc_class
 	*/
 	struct objc_property_list *properties;
 };
+// end: objc_class
 
 struct legacy_gnustep_objc_class
 {

loader.c

@@ -91,6 +91,7 @@ static void init_runtime(void)
 	}
 }
 
+// begin: objc_init
 struct objc_init
 {
 	uint64_t version;
@@ -105,6 +106,7 @@ struct objc_init
 	struct objc_protocol2 *proto_begin;
 	struct objc_protocol2 *proto_end;
 };
+// end: objc_init
 #include <dlfcn.h>
 
 void registerProtocol(Protocol *proto);

selector.h

@@ -20,6 +20,7 @@ struct sel_type_list
 /**
  * Structure used to store selectors in the list.
  */
+// begin: objc_selector
 struct objc_selector
 {
 	union
@@ -40,6 +41,8 @@ struct objc_selector
 	 */
 	const char * types;
 };
+// end: objc_selector
+
 
 /**
  * Returns the untyped variant of a selector.