SLAEx86 : Coding a custom Insertion Encoder and Decoder

In this blog post we will implement a custom encoding scheme and implement a custom decoder for it.

The implementation of the encoder

1. Copy some existing shellcode
2. For each shellcode value xor with 0x1
3. Append twice the same value.
4. Print the final shellcode
5. We need to remember the actual size of the shellcode without encoding

For example if my shellcode looks like "\xa1\xb2\c3\d4' the encoder will generate "\xa0\xa0\xb3\xb3\x5d\x5d\x62\x62\x32\x32\x5d\x5d\x65\x65\x35\x35"



The implementation of the decoder

1. Take the encoded shellcode
2. As .text section locations are not writable, we will create some reserved space in .bss segment
3. Read pointer ESI to read from the encoded shellcode location and a write pointer EDI to write the decoded shellcode.
4. Read each alternative positions of encoded shellcode because each value is repeated.
5. For each byte we read , we will xor with 0x1 and then store at the location pointed by EDI
6. Increase read read counter by 2 and write counter by 1
7. Continue the process till the loop counter AL value reaches 23 ( actual size of shellcode without encoding )
8. Once the value exceeds 23 it will jump to return_shellcode location
9. return_shellcode with stack implementation is to get rid of NULL characters which was generating due to jmp shellcode instruction.
10. Push the shellcode address onto the stack
11. Jump to the address stored on the stack

This blog post has been created for completing the requirements of SecurityTube Linux Assembly Expert Certification:
http://www.securitytube-training.com/online-courses/securitytube-linux-assembly-expert/
Student ID: PA-1191