ch06 done asyncTcp with libuv

- type puns are getting super confusing.

README.md

@@ -1,4 +1,4 @@
-# Scala Native, using Scala 3
+# Modern Systems in Scala Native, using Scala 3 :rocket:
 
 Updating the code in [Modern Systems Programming with Scala Native](https://pragprog.com/titles/rwscala/modern-systems-programming-with-scala-native/) to
 
@@ -28,17 +28,17 @@ You can compile and run `@main` methods in VS Code with Metals by clicking the r
 There are 35+ `@main` methods in the project. To compile a specific one to a binary, you can use inside the root directory, for example:
 
 ```bash
-scala-cli package . --main-class ch08.simplePipe.simplePipe
+scala-cli package . --main-class ch08.simplePipe.run
 ```
 
 This will place the binary executable in the project root directory:
 
 ```bash
-Wrote /home/spam/Projects/modern-systems-scala-native/ch08.simplePipe.simplePipe, run it with
-  ./ch08.simplePipe.simplePipe
+Wrote /home/spam/Projects/modern-systems-scala-native/ch08.simplePipe.run, run it with
+  ./ch08.simplePipe.run
 ```
 
-Here the class is the import path to the method: `ch08` and `simplePipe` are package names, and `simplePipe` is the name of the `@main` method:
+Here the class is the import path to the method: `ch08` and `simplePipe` are package names, and `run` is the name of the `@main` method:
 
 ```scala
 package ch08
@@ -47,50 +47,50 @@ package simplePipe
 // ...
 
 @main
-def simplePipe: Unit = ??? // so this is ch08.simplePipe.simplePipe
+def run: Unit = ??? // so this is ch08.simplePipe.run
 ```
 
 If in doubt, you can use the `--interactive` mode, which lets you pick the `@main` method you want:
 
 ```bash
 $ scala-cli package . --interactive
 Found several main classes. Which would you like to run?
-[0] ch01.hello.helloWorld
-[1] ch06.asyncTimer.asyncTimer
-[2] ch09.jsonSimple.jsonSimple
-[3] ch05.httpServer.httpServer05
-[4] ch08.fileOutputPipe
-[5] ch01.helloNative.helloNative
-[6] ch06.asyncHttp.asyncHttp
-[7] ch09.lmdbSimple.lmdbSimple
-[8] ch08.fileInputPipe
-[9] ch01.testing.testNullTermination
-[10] ch01.cStringExpr1.cStringExperiment1
-[11] ch01.sscanfInt.sscanfIntExample
-[12] ch01.badStuff.testingBadStuff
-[13] ch08.filePipeOut.filePipeOut
-[14] ch02.agg.aggregateAndCount
-[15] ch01.goodSscanf.goodSscanfStringParse
-[16] ch01.badSscanf.badSscanfStringParse
-[17] ch04.badExec.badExec
-[18] ch07.simpleAsync.simpleAsync
-[19] ch06.asyncTcp.asyncTcp
-[20] ch03.http.httpClient
-[21] ch08.simplePipe.simplePipe
-[22] ch01.maxNgramFast.maxNgramFast
-[23] ch07.curlAsync.curlAsync
-[24] ch02.sort.sortByCount
-[25] ch08.filePipe.filePipeMain
-[26] ch03.tcp.tcpClient
-[27] ch01.maxNgramNaive.maxNgramNaive
-[28] ch04.nativePipeTwo.nativePipeTwo
+[0] ch01.helloWorld.run
+[1] ch06.asyncTimer.run
+[2] ch09.jsonSimple.run
+[3] ch05.httpServer.run
+[4] ch08.fileOutputPipe.run
+[5] ch01.helloNative.run
+[6] ch06.asyncHttp.run
+[7] ch09.lmdbSimple.run
+[8] ch08.fileInputPipe.run
+[9] ch01.testNullTermination.run
+[10] ch01.cStringExperiment1.run
+[11] ch01.sscanfIntExample.run
+[12] ch01.testingBadStuff.run
+[13] ch08.filePipeOut.run
+[14] ch02.aggregateAndCount.run
+[15] ch01.goodSscanfStringParse.run
+[16] ch01.badSscanfStringParse.run
+[17] ch04.badExec.run
+[18] ch07.simpleAsync.run
+[19] ch06.asyncTcp.run
+[20] ch03.httpClient.run
+[21] ch08.simplePipe.run
+[22] ch01.maxNgramFast.run
+[23] ch07.curlAsync.run
+[24] ch02.sortByCount.run
+[25] ch08.filePipe.run
+[26] ch03.tcpClient.run
+[27] ch01.maxNgramNaive.run
+[28] ch04.nativePipeTwo.run
 [29] ch01.moreTesting.run
-[30] ch01.cStringExpr2.cStringExperiment2
-[31] ch04.nativeFork.nativeFork
-[32] ch04.nativePipe.nativePipe
-[33] ch10.libUvService.libuvService
-[34] bug.run
-[35] ch07.timerAsync.timerAsync
+[30] ch01.cStringExperiment2.run
+[31] ch04.nativeFork.run
+[32] ch04.nativePipe.run
+[33] ch10.libUvService.run
+[34] ch01.bug.run
+[35] ch07.timerAsync.run
 21
 [info] Linking (multithreadingEnabled=true, disable if not used) (2353 ms)
 [info] Discovered 1119 classes and 7040 methods after classloading
@@ -104,8 +104,8 @@ Found several main classes. Which would you like to run?
 [info] Linking native code (immix gc, none lto) (239 ms)
 [info] Postprocessing (0 ms)
 [info] Total (9395 ms)
-Wrote /home/spam/Projects/modern-systems-scala-native/ch08.simplePipe.simplePipe, run it with
-  ./ch08.simplePipe.simplePipe
+Wrote /home/spam/Projects/modern-systems-scala-native/ch08.simplePipe.run, run it with
+  ./ch08.simplePipe.run
 ```
 
 ### Linking to external C libraries
@@ -154,7 +154,7 @@ You need to compile and run the HTTP server from chapter 5.
 Read the compilation message for the name of the binary executable:
 
 ```bash
-scala-cli package . --main-class ch05.httpServer.httpServer
+scala-cli package . --main-class ch05.httpServer.run
 ...
 Wrote /home/spam/Projects/modern-systems-scala-native/project, run it with
   ./project
@@ -286,7 +286,7 @@ package badExec
 // then use code from common.scala here
 ```
 
-There is a lot of this duplication in later chapters. I'll fix them.
+There is a lot of this duplication in later chapters. I fixed them.
 
 ### `CSize / USize` instead of `Int`
 
@@ -376,6 +376,56 @@ The book clearly says, in a "warning box":
 
 ![type-puns](images/typePuns.png)
 
+There are many more issues. For example, given:
+
+```scala
+type TCPHandle = Ptr[Ptr[Byte]] // book and code
+type ClientState = CStruct3[Ptr[Byte], CSize, CSize]
+```
+
+but then:
+
+```scala
+val closeCB = CFuncPtr1.fromScalaFunction[TCPHandle, Unit]:
+  (client: TCPHandle) =>
+    // ...
+    val clientStatePtr = (!client).asInstanceOf[Ptr[ClientState]]
+```
+
+Since `client` is `TCPHandle = Ptr[Ptr[Byte]]`, `!client` is `Ptr[Byte]`.
+So we are casting a `Ptr[Byte]` into a `Ptr[CStruct3[Ptr[Byte], CSize, CSize]]`!
+Does this imply `Byte = CStruct3[Ptr[Byte], CSize, CSize]`?
+No, it does not work that way I think... :confused:
+
+There are many more instances of this. For example, given
+
+```scala
+type TCPHandle = Ptr[Ptr[Byte]]
+type ShutdownReq = Ptr[Ptr[Byte]]
+```
+
+we have:
+
+```scala
+def shutdown(client: TCPHandle): Unit =
+  val shutdownReq = malloc(uv_req_size(UV_SHUTDOWN_REQ_T)).asInstanceOf[ShutdownReq]
+  !shutdownReq = client.asInstanceOf[Ptr[Byte]]
+```
+
+Again, here `!shutdownReq` is a `Ptr[Byte]`, but `client` is a `Ptr[Ptr[Byte]]`.
+So we are trying to squeeze a `Ptr[Ptr[Byte]]` into a `Ptr[Byte]`!
+We do this by pretending that the nested pointer does not exist with `asInstanceOf[]`.
+OK fine, we can trick the compiler this way, but can we later actually use the inner
+nested pointer of `client` correctly?
+Because later these are passed to actual `libuv` functions...
+
+:confused: :confused: :confused:
+
+***Big brain moment:*** basically, pretty much *anything* can be cast to `Ptr[Byte]`...
+Since "everything is a byte", the "beginning of a block of anything" is a `Ptr[Byte]`!
+
+:brain: :brain: :brain: :tada: :confetti_ball: :partying_face:
+
 Not sure how to handle this, it will be guesswork.
 If compilation fails during linking phase then I'll know the types are wrong.
 But if linking does not fail, then I'll have to figure it out from the execution.

project.scala

@@ -4,5 +4,5 @@
 //> using exclude "gatling/*"
 //> using options -explain-cyclic -Ydebug-cyclic
 //> using dep io.argonaut::argonaut:6.3.9
-//> using dep io.gatling:gatling-app:3.11.1
+//> using dep io.gatling:gatling-app:3.11.2
 // // > using dep biz.enef:slogging_2.13:0.6.2

src/main/scala/ch06/asyncHttp/main.scala

@@ -4,8 +4,7 @@ package asyncHttp
 import scalanative.unsigned.UnsignedRichInt
 import scalanative.unsafe.{CQuote, Ptr, CSize, CSSize, CString, sizeof, stackalloc}
 import scalanative.unsafe.{CStruct3, CFuncPtr1, CFuncPtr2, CFuncPtr3}
-import scalanative.libc.{stdlib, string, stdio}
-import stdlib.malloc
+import scalanative.libc.{stdlib, string, stdio}, stdlib.malloc
 
 import LibUV.*, LibUVConstants.*
 import HTTP.RequestHandler

src/main/scala/ch06/asyncTcp/main.scala

@@ -5,7 +5,6 @@ import scalanative.unsigned.UnsignedRichInt
 import scalanative.unsafe.{CQuote, stackalloc, CString, CSize, CSSize, Ptr, sizeof}
 import scalanative.unsafe.{CStruct3, CFuncPtr1, CFuncPtr2, CFuncPtr3}
 import scalanative.libc.{stdio, stdlib, string}
-import stdlib.malloc
 
 import LibUV.*, LibUVConstants.*
 
@@ -21,7 +20,7 @@ val connectionCB = CFuncPtr2.fromScalaFunction[TCPHandle, Int, Unit]:
     println("received connection")
 
     // initialize the new client tcp handle and its state
-    val client = malloc(uv_handle_size(UV_TCP_T)).asInstanceOf[TCPHandle]
+    val client = stdlib.malloc(uv_handle_size(UV_TCP_T)).asInstanceOf[TCPHandle]
     checkError(uv_tcp_init(loop, client), "uv_tcp_init(client)")
 
     // this line shows that TCPHandle has to be Ptr[Ptr[Byte]] not Ptr[Byte]
@@ -48,3 +47,43 @@ val readCB = CFuncPtr3.fromScalaFunction[TCPHandle, CSSize, Ptr[Buffer], Unit]:
     else // instead of echoing back line-by-line, keep reading until completion.
       appendData(clientStatePtr, size, buffer)
       stdlib.free(buffer._1)
+
+// compile with:
+// $ scala-cli package . --main-class ch06.asyncTcp.run
+// ...
+// Wrote /home/spam/Projects/modern-systems-scala-native/project, run it with
+// ./project
+// Run it with:
+// $ ./project
+// In another Terminal, run Netcat to connect, then type some stuff:
+// $ nc localhost 8080
+
+// Here are the outputs:
+// $ nc localhost 8080
+// spam
+// Ctrl+C
+
+// $ ./project
+// hello!
+// uv_ip4_addr returned 0
+// uv_tcp_init(server) returned 0
+// uv_tcp_bind returned 0
+// uv_tcp_listen returned 0
+// received connection
+// uv_tcp_init(client) returned 0
+// allocated data at 70fa5420; assigning into handle storage at 70fa5320
+// uv_accept returned 0
+// uv_read_start returned 0
+// allocating 4096 bytes
+// read 5 bytes
+// client 70fa5420: 5/4096 bytes used
+// allocating 4096 bytes
+// read -4095 bytes
+// uv_write returned 0
+// connection is closed, shutting down
+// uv_shutdown returned 0
+// write completed
+// all pending writes complete, closing TCP connection
+// uv_close returned 1684660608: Unknown system error 1684660608: Unknown system error 1684660608
+// closed client connection
+// THESE ERRORS ARE COMPLETELY NORMAL.

src/main/scala/ch06/common/common.scala

@@ -79,6 +79,10 @@ def appendData(state: Ptr[ClientState], size: CSSize, buffer: Ptr[Buffer]): Unit
 
 def shutdown(client: TCPHandle): Unit =
   val shutdownReq = malloc(uv_req_size(UV_SHUTDOWN_REQ_T)).asInstanceOf[ShutdownReq]
+
+  // Here TCPHandle = Ptr[Byte] but actually TCPHandle = Ptr[Ptr[Byte]]! What's going on?
+  // ShutdownReq = Ptr[Ptr[Byte]] so !shutdownReq = Ptr[Byte]. To put TCPHandle in there,
+  // we have to pretend TCPHandle is not Ptr[Ptr[Byte]] but Ptr[Byte] instead?!
   !shutdownReq = client.asInstanceOf[Ptr[Byte]]
   checkError(uv_shutdown(shutdownReq, client, shutdownCB), "uv_shutdown")
 
@@ -93,27 +97,42 @@ val allocCB = CFuncPtr3.fromScalaFunction[TCPHandle, CSize, Ptr[Buffer], Unit]:
   (client: TCPHandle, size: CSize, buffer: Ptr[Buffer]) =>
     println("allocating 4096 bytes") // disregard size
     val buf = malloc(4096) // 0.5
-    buffer._1 = buf
+    buffer._1 = buf // Buffer = CStruct2[Ptr[Byte], CSize]
     buffer._2 = 4096.toUSize // 0.5
 
 val writeCB = CFuncPtr2.fromScalaFunction[WriteReq, Int, Unit]:
   (writeReq: WriteReq, status: Int) =>
     println("write completed")
-    val responseBuffer = (!writeReq).asInstanceOf[Ptr[Buffer]]
-    stdlib.free(responseBuffer._1)
-    stdlib.free(responseBuffer.asInstanceOf[Ptr[Byte]])
-    stdlib.free(writeReq.asInstanceOf[Ptr[Byte]])
+
+    // WriteReq = Ptr[Ptr[Byte]] so !writeReq = Ptr[Byte] = Ptr[Buffer].
+    val responseBuffer = (!writeReq).asInstanceOf[Ptr[Buffer]] // type puns galore!
+
+    // after done writing, we have to free everything.
+    // for freeing, we have to type pun / cast back to Ptr[Byte], annoying.
+    stdlib.free(responseBuffer._1) // Ptr[Buffer]._1 = first field of buffer = the buffer.
+    stdlib.free(responseBuffer.asInstanceOf[Ptr[Byte]]) // free the whole buffer CStruct2.
+    stdlib.free(writeReq.asInstanceOf[Ptr[Byte]]) // pun and free the write request too.
 
 val shutdownCB = CFuncPtr2.fromScalaFunction[ShutdownReq, Int, Unit]:
   (shutdownReq: ShutdownReq, status: Int) =>
     println("all pending writes complete, closing TCP connection")
+
+    // ShutdownReq = Ptr[Ptr[Byte]], so !shutdownReq = Ptr[Byte] = TCPHandle
+    // but TCPHandle = Ptr[Ptr[Byte]] too, so I don't get it!
     val client = (!shutdownReq).asInstanceOf[TCPHandle]
     checkError(uv_close(client, closeCB), "uv_close")
+
+    // to free, everything has to be cast back to Ptr[Byte]
     stdlib.free(shutdownReq.asInstanceOf[Ptr[Byte]])
 
 val closeCB = CFuncPtr1.fromScalaFunction[TCPHandle, Unit]: (client: TCPHandle) =>
   println("closed client connection")
+
+  // TCPHandle = Ptr[Ptr[Byte]] so !client = Ptr[Byte]
+  // So Ptr[Byte] = Ptr[ClientState] = Ptr[CStruct3[Ptr[Byte], CSize, CSize]]
+  // So Byte = CStruct3[Ptr[Byte], CSize, CSize]?
   val clientStatePtr = (!client).asInstanceOf[Ptr[ClientState]]
-  stdlib.free(clientStatePtr._1)
-  stdlib.free(clientStatePtr.asInstanceOf[Ptr[Byte]])
-  stdlib.free(client.asInstanceOf[Ptr[Byte]])
+
+  stdlib.free(clientStatePtr._1) // free the client state first field of CStruct3
+  stdlib.free(clientStatePtr.asInstanceOf[Ptr[Byte]]) // free the whole CStruct3
+  stdlib.free(client.asInstanceOf[Ptr[Byte]]) // free the whole TCPHandle

src/main/scala/ch06/common/libuv.scala

@@ -6,12 +6,15 @@ import scalanative.unsafe.{CStruct2, CFuncPtr1, CFuncPtr2, CFuncPtr3}
 @link("uv")
 @extern
 object LibUV:
+  // the book keeps calling these "opaque" but it was written in Scala 2.11.
+  // Should we make these actually opaque now in Scala 3 with: opaque type ...?
+  // If we do, there are many compiler errors to be fixed with .asInstanceOf[...].
   type TimerHandle = Ptr[Byte] // book says Ptr[Ptr[Byte]], what should it be?
   type PipeHandle = Ptr[Ptr[Byte]]
   type Loop = Ptr[Byte] // book says Ptr[Ptr[Byte]], what should it be?
-  type TCPHandle = Ptr[Ptr[Byte]]
+  type TCPHandle = Ptr[Ptr[Byte]] // code treats it as Ptr[Byte], what should it be?
   type WriteReq = Ptr[Ptr[Byte]]
-  type ShutdownReq = Ptr[Ptr[Byte]]
+  type ShutdownReq = Ptr[Ptr[Byte]] // should it be Ptr[TCPHandle]?
   type Buffer = CStruct2[Ptr[Byte], CSize]
 
   type TimerCB = CFuncPtr1[TimerHandle, Unit]