programming, swift

Swift 3.0 Unsafe World

Like most of the modern languages, in Swift you will be living in a happy world where all the memory is managed by an external element, it could be the compiler/runtime like swift or it could be worse and depend on a garbage collector. References to instances that fly around are hidden in the language and rarely you need to deal with these issues.

However, given Swift versatility you could need to call a dirty C Api like OpenGL or POSIX functions, and in those cases you will need to deal with that thing that has caused lots of headaches to many of us, yes, I'm talking about pointers and allocating memory manually in the heap

Before 3.0, Swift unsafe API was a little bit confusing, there were several paths to achieve the same result, and that caused that you usually ended by copying and pasting what you saw in stackoverflow without understanding what is really happening. In 3.0 everything has changed, and it has done it for the better.

In this post I won't write how to migrate from Swift 2.x to 3.0 code, instead I will describe how the things works in 3.0 and will be comparing occasionally with C, since usually the main purpose of using unsafe references is to interact with low level C APIs.

Let's start by doing the simplest operation, allocating memory to hold an Integer.

In C, you will do something like this:

int *a = malloc(sizeof(int));
*a = 42;
 
printf("a's value: %d", *a);
 
free(a)

The same is achieved in Swift with:

let a = UnsafeMutablePointer<Int>.allocate(capacity: 1)
a.pointee = 42
 
print("a's value: \(a.pointee)"// 42 
 
a.deallocate(capacity: 1)

The first type we see in Swift is UnsafeMutablePointer<T>, this generic struct represents a pointer to the type T, and as you can see, it has a static method, allocate that will reserve capacity number of elements.

As you can imagine, there is a variant of UnsafeMutablePointer called UnsafePointer that won't allow you to change the pointee element. Moreover, non-mutable UnsafePointer does not even have a allocate method.

In swift, there is another way to generate a Unsafe[Mutable]Pointer, it is by using the & operator. When passing parameters to a block or function, you can use & to take a pointer to it. Let's see an example

func receive(pointer: UnsafePointer<Int> ) {
    print("param value is: \(pointer.pointee)"// 42 
}
 
var a: Int = 42
receive(pointer: &a)

& operand requires type to be var, but it will give you what you need in each situation. For example, you could take a mutable reference and even change it, like this:

func receive(mutablePointer: UnsafeMutablePointer<Int>) {
    mutablePointer.pointee *= 2
}
 
var a = 42
receive(mutablePointer: &a)
print("A's value has changed in the function: \(a)"// 84 

There is an important difference between the first sample, and these last. In the first one we manually allocated the memory, (and we needed to manually allocate it afterwards), and in the samples with a function, we are creating a pointer from a Swift allocated memory. Obviously, managing the memory and taking pointers are two different topics. In the last part of the post we will talk about memory management.

But, what if we want to take a pointer to a Swift managed memory without having to create a function?. To do it we will use withUnsafeMutablePointer, that will take a reference to a Swift type and a block with the pointer as it's parameter. Let's see it in action

var a = 42
withUnsafeMutablePointer(to: &a) { $0.pointee *= 2 }
 
print("a's value is: \(a)"// 84 

Now that we know the tools, we are ready to call C APIs with pointers in its paremters, let's see an example of POSIX function opendir / readdir to list the contents of the current directory.

var dirEnt: UnsafeMutablePointer<dirent>?
var dp: UnsafeMutablePointer<DIR>?
 
let data = ".".data(using: .ascii)
data?.withUnsafeBytes({ (ptr: UnsafePointer<Int8>in
    dp = opendir(ptr)
})
 
repeat {
    dirEnt = readdir(dp)
    if let dir = dirEnt {
        withUnsafePointer(to: &dir.pointee.d_name, { ptr in
            let ptrStr = unsafeBitCast(ptr, to: UnsafePointer<CChar>.self)
            let name = String(cString: ptrStr)
            print("\(name)")
        })
    }
while dirEnt != nil

Cast between pointers

When dealing with C API you need sometimes to cast pointers to struct to a different struct. This is very easy to do en C (and very dangerous and error prone too), as you have seen in Swift, all pointers are typed, that means that an UnsafePointer<Int> cannot be used where an UnsafePointer<UInt8> is required, that's good in terms of producing a safer code, but at the same time that makes not possible to interact with C APIs that requires this types of casts, like for example socket bind() function. For theses cases, we will use withMemoryRebound which is a function that will convert a pointer from a type to a different one, let's see how we can use the cast in the bind function where you typically create a sockaddr_in struct and then cast to sockaddr

var addrIn = sockaddr_in()
// Fill sockaddr_in fields 
withUnsafePointer(to: &addrIn) { ptr in
    ptr.withMemoryRebound(to: sockaddr.self, capacity: 1, { ptrSockAddr in
        bind(socketFd, UnsafePointer(ptrSockAddr), socklen_t(MemoryLayout<sockaddr>.size))
    })
}

There is a special case when casting pointers, some C APIs requires to pass a void* pointer. Before Swift 3.0, you could do it with UnsafePointer<Void> however, in 3.0 a new type has been added to handle these types of pointers: UnsafeRawPointer. This struct is not generic, so it means that it won't hold information tied to any specific type and that will simplifly our code. To create a UnsafeRawPointer we can just wrap an existing pointer with it in its constructor. If we want to go the other way around, converting an UnsafeRawPointer to a specific typed pointer we would need to use a version of withMemoryRebound of the previous sample but in this case it is called assumingMemoryBound.

let intPtr = UnsafeMutablePointer<Int>.allocate(capacity: 1)
let voidPtr = UnsafeRawPointer(intPtr)
let intPtrAgain = voidPtr.assumingMemoryBound(to: Int.self)

Pointers as arrays

Up until here we have covered the typical usage of pointers, and you can deal with most of C API calls, however pointers are used for more usages, one of them is to iterate through a chunk of memory that we know it holds valuable data. In Swift we have several ways to do the same. Actually, UnsafePointer has a advanced(by:) method that allows you to iterate through the memory, advanced(by:) returns another UnsafePointer so we can store or read what's there.

let size = 10
var a = UnsafeMutablePointer<Int>.allocate(capacity: size)
for idx in 0..<10 {
    a.advanced(by: idx).pointee = idx
}
a.deallocate(capacity: size)

In addition to this, Swift has another struct that makes this usage easier, we are talking about UnsafeBufferPointer. This Struct is the bridge between Swift arrays and pointers. If we construct a UnsafeBufferPointer from an UnsafePointer we will be able to use most of the array functions of native Swift type given that UnsafeBufferPointer implements Collection, Indexable and RandomAccessCollection swift protocols. Said that we can iterate throught memory like this:

// Using a and size from previous code 
var b = UnsafeBufferPointer(start: a, count: size)
b.forEach({
    print("\($0)" // Prints 0 to 9 that we fill previously 
)})

When we said that UnsafeBufferPointer is the bridge with Swift arrays, it is also because it is easy to take a UnsafeBufferPointer from an existing array like this sample:

var a = [123456]
a.withUnsafeBufferPointer({ ptr in
    ptr.forEach({ print("\($0)") }) // 1, 2, 3... 
})

Memory management dangers

We have seen quite a lot ways to reference to raw memory, but we cannot forget that we are entering in a dangerous terrain. Probably the repating word Unsafe warns us to be careful when using it. Moreover, we are mixing two worlds when using unsafe references. Let's see the dangers in all its glory with an example:

var collectionPtr: UnsafeMutableBufferPointer<Int>?
 
func duplicateElements(inArray: UnsafeMutableBufferPointer<Int>) {
    for i in 0..<inArray.count {
        inArray[i] *= 2
    }
}
 
repeat {
    var collection = [123]
    collection.withUnsafeMutableBufferPointer({ collectionPtr = $0 })
while false
 
duplicateElements(inArray: collectionPtr!// Crash due to EXC_BAD_ACCESS 

Although this sample is not real, it illustrates what happens in more complex code when using pointers to swift allocated variables. Here, collection is created in a block, and therefore, the reference will be freed after the block ends. We, intentionally, save a reference in collectionPtr that we use after the original collection is not valid anymore so it crashes when trying to use it in duplicateElements(inArray:). If we want to take pointers to swift allocated elements we need to be sure that they are going to be available when we want to use it. Keep in mind that ARC will add a release to any reference that is leaving the scope, and if it is not strong referenced in any other place, it will be freed.

A solution to overcome Swift's memory management is to allocate the memory by ourselves, like we've done in some samples of this post, this eliminates the problem of accessing to not valid references, but it introduces another problem. If we don't deallocate manually what we have allocated our program will have memory leaks.

bitPattern for pointers with fixed values

To finish the post I would like to introduce a couple of more usages of pointers in Swift. One of them is something that it is useful in C APIs that require a void* pointer with a value instead with a memory address. Usually this happens when a function accept different types of parameters and to be generic wrap the value in a void*. It would be something like this:

void generic_function(int value_type, void* value);
 
generic_function(VALUE_TYPE_INT, (void *)2);
struct function_data data;
generic_function(VALUE_TYPE_STRUCT, (void *)&data);

If we want to use the first call of that funciton from swift we need to use a special constructor that will allow us to create a pointer pointing to a specific address. All the functions that we have seen doesn't allow you to change the address of the pointee, so we will be using UnsafePointer(bitPattern:) in this case.

generic_function(VALUE_TYPE_INT, UnsafeRawPointer(bitPattern: 2))
var data = function_data()
withUnsafePointer(to: &data, { generic_function(VALUE_TYPE_STRUCT, UnsafeRawPointer($0)) } )

Opaque Pointer

The last thing I would like to comment is the post is the usage of Opaque pointers to Swift types. It is very common to have a userData like parameter in some C API functions, that user data will be a void* pointer that will hold an arbitrary memory value that will be used after. A common use case is when dealing with functions that sets some callbacks that will be invoked when an event happens. In that case it will be useful to pass a reference to a Swift object so we can call its method from the C callback.

We could use a regular UnsafeRawPointer like we have seen in the rest of the post, however, as we've also seen, doing this could have problems with memory management. If we pass to a C world function a pointer to a object that we don't retain before, it can be freed and out program will crash.

Swift has an utility to take pointers to objects retaining its reference or not depending on our needs. Those are static functions of Unmanaged struct. With passRetained() we will create a retained reference to an object, so we can be sure that when using it from C world, it will be still there. If the object is already retained for the life of the callback we can also use passUnretained(). Both methods produces a instance of Unmanaged that will be converted to a UnsafeRawPointer by calling toOpaque()

On the other side, we can transform an UnsafeRawPointer to a class or struct instance using the inverse api fromOpaque() and takeRetained() or takeUnretained()`.

void set_callback(void (*functionPtr)(void*), void* userData));
struct CallbackUserData {
    func sayHello() { print("Hello world!" ) }
}
 
func callback(userData: UnsafeMutableRawPointer) {
    let callbackUserData = Unmanaged.fromOpaque(userData).takeRetainedValue()
 
    callbackUserData.sayHello() // "Hello world!" 
}
 
var userData = CallbackUserData()
let reference = Unmanaged.passRetained(userData).toOpaque()
set_callback(callback, reference)

Conclusions

As you can see, calling C code from Swift is completely doable, and knowing the tools we have, it is easy and it doesn't need a lot of lines of code to do it. Unsafe and Unmanaged API is bigger that we covered in this post, but I hope this is a good foundation to dive deeper in case you are interested or you need it.

5 comentarios en “Swift 3.0 Unsafe World

  1. func receive(pointer: UnsafePointer, ) {
    print(“param value is: \(pointer.pointee)”) // 42
    }

    let a: Int = 42
    receive(pointer: &a)

    Does not compile. You need to change `let a` to `var a` …
    also, you left in an extra comma in the function signature…..

    So, given this change, how would you edit your explanation since you can’t now should the difference between Unsafe and UnsafeMutable…?

    1. Thanks Joe for the comment. I’ll change that.

      Actually the Mutable or non-Mutable depends on the receiver type of the & operator. & operand needs to be a var instead of a let but you can take an inmutable pointer or a mutable one. Obviously, you can only mutate the value if it is a UnsafeMutablePointer.

      Take this code as an example:

      var a: Int = 42
      withUnsafePointer(to: &a) { print(“\($0.pointee)”) }
      withUnsafeMutablePointer(to: &a) { $0.pointee = 10; print(“\($0.pointee)”) }

      You can only mutate the value in the last line. & operator will give you what you need in each call.

  2. Thanks for good article. I have searched for good examples on opaque pointers in use with calling instances methods. This has been the best so far.

    An earlier sentence also seems wrong:
    “In the first one we manually allocated the memory, (and we needed to manually allocate it afterwards)” Shouldn’t it de-allocate afterwards?

  3. Great topic! Thanks for this.

    I wanted to let you know I think there’s an error in the last example. I was reading that as I was trying to pass an RxSwift observer in through the contextInfo param of a call to UIImageWriteToSavedPhotosAlbum. That exanple describes the solution perfectly other than it looks like Unmanaged.passRetained only works with AnyObject which would not inlclude structs. Your example is using a struct and as it turns out RxSwift observers are structs as well!

    I had to wrap it in a container class which is a shame but I wanted to let you know in case anyone else is looking for this solution

Los comentarios están cerrados.