0.4.2

Added clarification that you are able to create multiple invitations by entering /connect multiple times without invalidating the previously created invitations.

.github/workflows/build.yml

@@ -4,6 +4,7 @@ on:
   push:
     branches:
       - master
+      - v4
     tags:
       - "v*"
   pull_request:
@@ -49,24 +50,24 @@ jobs:
           - os: ubuntu-20.04
             cache_path: ~/.stack
             stack_args: "--test"
-            artifact_rel_path: /bin/dog-food
-            asset_name: dog-food-ubuntu-20_04-x86-64
+            artifact_rel_path: /bin/simplex-chat
+            asset_name: simplex-chat-ubuntu-20_04-x86-64
           - os: ubuntu-18.04
             cache_path: ~/.stack
             stack_args: "--test"
-            artifact_rel_path: /bin/dog-food
-            asset_name: dog-food-ubuntu-18_04-x86-64
+            artifact_rel_path: /bin/simplex-chat
+            asset_name: simplex-chat-ubuntu-18_04-x86-64
           - os: macos-latest
             cache_path: ~/.stack
             stack_args: "--test"
-            artifact_rel_path: /bin/dog-food
-            asset_name: dog-food-macos-x86-64
+            artifact_rel_path: /bin/simplex-chat
+            asset_name: simplex-chat-macos-x86-64
           # TODO enable tests for windows once fixed (remove stack_args altogether)
           - os: windows-latest
             cache_path: C:/sr
             stack_args: ""
-            artifact_rel_path: /bin/dog-food.exe
-            asset_name: dog-food-windows-x86-64
+            artifact_rel_path: /bin/simplex-chat.exe
+            asset_name: simplex-chat-windows-x86-64
     steps:
       - name: Clone project
         uses: actions/checkout@v2

.gitignore

@@ -48,3 +48,4 @@ stack.yaml.lock
 
 # chat database
 *.db
+*.db.bak

Dockerfile

@@ -1,10 +1,10 @@
-FROM haskell:8.8.4 AS build-stage
+FROM haskell:8.10.4 AS build-stage
 # if you encounter "version `GLIBC_2.28' not found" error when running
 # chat client executable, build with the following base image instead:
-# FROM haskell:8.8.4-stretch AS build-stage
+# FROM haskell:8.10.4-stretch AS build-stage
 COPY . /project
 WORKDIR /project
 RUN stack install
 
 FROM scratch AS export-stage
-COPY --from=build-stage /root/.local/bin/dog-food /
+COPY --from=build-stage /root/.local/bin/simplex-chat /

README.md

@@ -1,56 +1,99 @@
+<img align="right" src="images/logo.svg" alt="SimpleX logo" height="90">
+
 # SimpleX chat
 
+## Private, secure, decentralized
+
 [![GitHub build](https://github.com/simplex-chat/simplex-chat/workflows/build/badge.svg)](https://github.com/simplex-chat/simplex-chat/actions?query=workflow%3Abuild)
 [![GitHub release](https://img.shields.io/github/v/release/simplex-chat/simplex-chat)](https://github.com/simplex-chat/simplex-chat/releases)
 
-## Federated chat - private, secure, decentralized
+> **NEW in v0.4: [groups](#groups) and [sending files](#sending-files)!**
+
+The motivation for SimpleX chat is [presented here](./simplex.md).
+
+SimpleX chat prototype is a thin terminal UI on top of [SimpleXMQ](https://github.com/simplex-chat/simplexmq) message broker that uses [SMP protocols](https://github.com/simplex-chat/simplexmq/blob/master/protocol).
 
 See [simplex.chat](https://simplex.chat) website for chat demo and the explanations of the system and how SMP protocol works.
 
-SimpleX chat prototype is a "thin" terminal UI on top of [SimpleXMQ](https://github.com/simplex-chat/simplexmq) message broker, that uses [SMP protocol](https://github.com/simplex-chat/simplexmq/tree/master/protocol) and SMP agent protocol.
+![simplex-chat](./images/connection.gif)
+
+## Table of contents
+
+- [Disclaimer](#disclaimer)
+- [Network topology](#network-topology)
+- [Terminal chat features](#terminal-chat-features)
+- [Installation](#installation)
+  - [Download chat client](#download-chat-client)
+  - [Build from source](#build-from-source)
+    - [Using Docker](#using-docker)
+    - [Using Haskell stack](#using-haskell-stack)
+- [Usage](#usage)
+  - [Running the chat client](#running-the-chat-client)
+  - [How to use SimpleX chat](#how-to-use-simplex-chat)
+  - [Groups](#groups)
+  - [Sending files](#sending-files)
+  - [Access chat history](#access-chat-history)
+- [Future roadmap](#future-roadmap)
+- [License](#license)
+
+## Disclaimer
+
+This is WIP implementation of SimpleX chat that implements a new network topology for asynchronous communication combining the advantages and avoiding the disadvantages of federated and P2P networks.
+
+If you expect a software being reliable most of the time and doing something useful, then this is probably not ready for you yet. We do use it for terminal chat though, and it seems to work most of the time - we would really appreciate if you try it and give us your feedback.
+
+**Please note:** The main differentiation of SimpleX network is the approach to internet message routing rather than encryption; for that reason no sufficient attention was paid to either TCP transport level encryption or to E2E encryption protocols - they are implemented in an ad hoc way based on RSA and AES algorithms. See [SMP protocol](https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#appendix-a) on TCP transport encryption protocol (AEAD-GCM scheme, with an AES key negotiation based on RSA key hash known to the client in advance) and [this section](https://github.com/simplex-chat/simplexmq/blob/master/rfcs/2021-01-26-crypto.md#e2e-encryption) on E2E encryption protocol (an ad hoc hybrid scheme a la PGP). These protocols will change in a consumer ready version to something more robust.
+
+## Network topology
+
+SimpleX is a decentralized client-server network that uses redundant, disposable nodes to asynchronously pass the messages via message queues, providing receiver and sender anonymity.
+
+Unlike P2P networks, all messages are passed through one or several (for redundancy) servers, that do not even need to have persistence (in fact, the current [SMP server implementation](https://github.com/simplex-chat/simplexmq#smp-server) uses in-memory message storage, persisting only the queue records) - it provides better metadata protection than P2P designs, as no global participant ID is required, and avoids many [problems of P2P networks](https://github.com/simplex-chat/simplex-chat/blob/master/simplex.md#comparison-with-p2p-messaging-protocols).
+
+Unlike federated networks, the participating server nodes do NOT have records of the users, do NOT communicate with each other, do NOT store messages after they are delivered to the recipients, and there is no way to discover the full list of participating servers - it avoids the problem of metadata visibility that federated networks suffer from and better protects the network, as servers do not communicate with each other. Each server node provides unidirectional "dumb pipes" to the users, that do authorization without authentication, having no knowledge of the the users or their contacts. Each queue is assigned two RSA keys - one for receiver and one for sender - and each queue access is authorized with a signature created using a respective key's private counterpart.
+
+The routing of messages relies on the knowledge of client devices how user contacts and groups map at any given moment of time to these disposable queues on server nodes.
+
+## Terminal chat features
 
-These features are implemented:
 - 1-to-1 chat with multiple people in the same terminal window.
-- auto-populated recipient name - just type your messages to reply to the sender.
-- default server is available to use - `smp1.simplex.im:5223` - you can deploy your own server (`smp-server` executable in [simplexmq](https://github.com/simplex-chat/simplexmq) repo).
-- no global identity or any names visible to the server(s) - for the full privacy of your contacts and conversations.
-- E2E encryption, with RSA public key that has to be passed out-of-band (see below).
-- message signing and verification with automatically generated RSA keys.
-- message integrity validation (via including the digests of the previous messages).
-- authentication of each command/message by SMP servers with automatically generated RSA key pairs.
+- Group messaging.
+- Sending files to contacts and groups.
+- Auto-populated recipient name - just type your messages to reply to the sender once the connection is established.
+- Demo SMP servers available and pre-configured in the app - or you can [deploy your own server](https://github.com/simplex-chat/simplexmq#using-smp-server-and-smp-agent).
+- No global identity or any names visible to the server(s), ensuring full privacy of your contacts and conversations.
+- E2E encryption, with RSA public key that has to be passed out-of-band (see [How to use SimpleX chat](#how-to-use-simplex-chat)).
+- Message signing and verification with automatically generated RSA keys.
+- Message integrity validation (via including the digests of the previous messages).
+- Authentication of each command/message by SMP servers with automatically generated RSA key pairs.
 - TCP transport encryption using SMP transport protocol.
 
-RSA keys are not used as identity, they are randomly generated for each contact. 2048 bit keys are used, it can be changed to 4096-bit in code via [rsaKeySize setting](https://github.com/simplex-chat/simplex-chat/blob/master/apps/dog-food/Main.hs).
+RSA keys are not used as identity, they are randomly generated for each contact.
 
-The chat client is stable, but it is quite new and unaudited, so you probably should NOT use it yet for high security communications - unless you know what you are doing.
+## Installation
 
-## SimpleX chat roadmap for 2021-22
+### Download chat client
 
-1. Switch to application level chat protocol. This will allow to separate physical server connection management from logical chat contacts, and to support all common chat functions.
-2. SMP queue redundancy and rotation in SMP agent protocol.
-3. Symmetric groups support in SMP agent protocol, as a foundation for chat groups.
-4. Delivery confirmation in SMP agent protocol.
-5. Multi-agent/device data synchronisation - to use chat on multiple devices.
-6. Synchronous streams support in SMP and SMP agent protocols, to support file transfer.
-7. Terminal chat UI and mobile apps.
-8. Scripts for simple SMP server deployment to hosting providers: Linode, Digital Ocean and Heroku.
-9. Public broadcast channels.
-10. Optional public contact/group addresses using DNS to establish connections, but not using it to send and receive messages - in this way you will keep all your contacts and groups even if you lose the control of the domain.
+Download the chat binary for your system from the [latest stable release](https://github.com/simplex-chat/simplex-chat/releases) and make it executable as shown below.
 
-## Installing the chat client
+#### Linux and MacOS
 
-You can:
-- download the executable file built on Github CI
-- build it from source
+```sh
+chmod +x <binary>
+mv <binary> ~/.local/bin/simplex-chat
+```
 
-### Downloading the executable chat client
+(or any other preferred location on PATH).
 
-You can download the executable binary file for your platform from the [latest stable release](https://github.com/simplex-chat/simplex-chat/releases) and make it executable:
+On MacOS you also need to [allow Gatekeeper to run it](https://support.apple.com/en-us/HT202491).
 
-  - Linux and MacOS: `chmod +x <binary>; mv <binary> ~/.local/bin/dog-food` (or any other preferred location on PATH). On MacOS you need to [allow Gatekeeper to run it](https://support.apple.com/en-us/HT202491).
-  - Windows: `move <binary> %APPDATA%\local\bin\dog-food.exe`.
+#### Windows
 
-### Building from source
+```sh
+move <binary> %APPDATA%/local/bin/simplex-chat.exe
+```
+
+### Build from source
 
 #### Using Docker
 
@@ -60,7 +103,6 @@ On Linux, you can build the chat executable using [docker build with custom outp
 $ git clone git@github.com:simplex-chat/simplex-chat.git
 $ cd simplex-chat
 $ DOCKER_BUILDKIT=1 docker build --output ~/.local/bin .
-$ dog-food
 ```
 
 > **Please note:** If you encounter ``version `GLIBC_2.28' not found`` error, rebuild it with `haskell:8.8.4-stretch` base image (change it in your local [Dockerfile](Dockerfile)).
@@ -79,62 +121,81 @@ and build the project:
 $ git clone git@github.com:simplex-chat/simplex-chat.git
 $ cd simplex-chat
 $ stack install
-$ dog-food
 ```
 
-## Running the chat client
+## Usage
 
-Run `dog-food` (as in "eating your own dog food"), or the downloaded file (if you did not move it to bin folder), to start the chat client.
+### Running the chat client
 
-By default, app data directory is created in the home directory (`~/.simplex`, or `%APPDATA%/simplex` on Windows), and SQLite database file `smp-chat.db` is initialized in it. The default SMP server is `smp1.simplex.im:5223#pLdiGvm0jD1CMblnov6Edd/391OrYsShw+RgdfR0ChA=` (base-64 encoded string after server port is the transport key digest) - it is pre-configured in the app.
+To start the chat client, run `simplex-chat` from the terminal.
 
-To specify a different file path for the chat database use `-d` command line option:
+By default, app data directory is created in the home directory (`~/.simplex`, or `%APPDATA%/simplex` on Windows), and two SQLite database files `simplex.chat.db` and `simplex.agent.db` are initialized in it.
+
+To specify a different file path prefix for the database files use `-d` command line option:
 
 ```shell
-$ dog-food -d my-chat.db
+$ simplex-chat -d alice
 ```
 
-If you deployed your own SMP server you can set client to use it via `-s` option:
+Running above, for example, would create `alice.chat.db` and `alice.agent.db` database files in current directory.
+
+Default SMP servers are hosted on Linode (London, UK and Fremont, CA) - they are [pre-configured in the app](https://github.com/simplex-chat/simplex-chat/blob/master/src/Simplex/Chat/Options.hs#L40). Base-64 encoded string after server host is the transport key digest.
+
+If you deployed your own SMP server(s) you can configure client via `-s` option:
 
 ```shell
-$ dog-food -s smp.example.com:5223#KXNE1m2E1m0lm92WGKet9CL6+lO742Vy5G6nsrkvgs8=
+$ simplex-chat -s smp.example.com:5223#KXNE1m2E1m0lm92WGKet9CL6+lO742Vy5G6nsrkvgs8=
 ```
 
 The base-64 encoded string in server address is the digest of RSA transport handshake key that the server will generate on the first run and output its digest.
 
-You can still talk to people using default or any other server, it only affects the location of the message queue when you initiate the connection (and the reply queue can be on another server, as set by the other party's client).
+You can still talk to people using default or any other server - it only affects the location of the message queue when you initiate the connection (and the reply queue can be on another server, as set by the other party's client).
 
-Run `dog-food --help` to see all available options.
+Run `simplex-chat -h` to see all available options.
 
-## Using chat client
+### How to use SimpleX chat
 
-Once chat client is started, use `/add <name1>` to create a new connection and generate an invitation to send to your contact via any other communication channel (`<name1>` - is any name you want to use for that contact).
+Once you have started the chat, you will be prompted to specify your "display name" and an optional "full name" to create a local chat profile. Your display name is an alias for your contacts to refer to you by - it is not unique and does not serve as a global identity. In case different contacts chose the same display name, the chat client adds a numeric suffix to their local display names.
 
-Invitation has format `smp::<server>::<queue_id>::<rsa_public_key_for_this_queue_only>` - this needs to be shared with another party, via any other channel. This invitation can only be used once - even if this is intercepted, the attacker would not be able to use it to send you the messages via this queue once your contact confirms that the connection is established.
+This diagram shows how to connect and message a contact:
 
-The party that received the invitation should use `/connect <name2> <invitation>` to accept the connection (`<name2>` is any name that the accepting party wants to use for you).
+<div align="center">
+  <img align="center" src="images/how-to-use-simplex.svg">
+</div>
 
-For example, if Alice and Bob want to chat, with Alice initiating, Alice would use [in her chat client]:
+Once you've set up your local profile, enter `/c` (for `/connect`) to create a new connection and generate an invitation. Send this invitation to your contact via any other channel.
 
-```
-/add bob
-```
+You are able to create multiple invitations by entering `/connect` multiple times and sending these invitations to the corresponding contacts you'd like to connect with.
 
-And then send the generated invitation to Bob out-of-band. Bob then would use [in his chat client]:
+The invitation has the format `smp::<server>::<queue_id>::<rsa_public_key_for_this_queue_only>`. The invitation can only be used once and even if this is intercepted, the attacker would not be able to use it to send you the messages via this queue once your contact confirms that the connection is established.
 
-```
-/connect alice <alice's invitation>
-```
+The contact who received the invitation should enter `/c <invitation>` to accept the connection. This establishes the connection, and both parties are notified.
 
-They would then use `@<name> <message>` commands to send messages. One may also press Space or just start typing a message to send a message to the contact that was the last.
-
-If your contact is disconnected, restart the chat client - it may happen if you lose internet connection.
+They would then use `@<name> <message>` commands to send messages. You may also just start typing a message to send it to the contact that was the last.
 
 Use `/help` in chat to see the list of available commands.
 
-## Accessing chat history
+### Groups
 
-SimpleX chat stores all your contacts and conversations in the local database file, making it private and portable by design, fully owned and controlled by you.
+To create a group use `/g <group>`, then add contacts to it with `/a <group> <name>`and send messages with `#<group> <message>`. Use `/help groups` for other commands.
+
+![simplex-chat](./images/groups.gif)
+
+> **Please note**: the groups are not stored on any server, they are maintained as a list of members in the app database to whom the messages will be sent.
+
+### Sending files
+
+You can send a file to your contact with `/f @<contact> <file_path>` - the recipient will have to accept it before it is sent. Use `/help files` for other commands.
+
+![simplex-chat](./images/files.gif)
+
+You can send files to a group with `/f #<group> <file_path>`.
+
+### Access chat history
+
+> 🚧 **Section currently out of date - will be updated soon** 🏗
+
+SimpleX chat stores all your contacts and conversations in a local database file, making it private and portable by design, fully owned and controlled by you.
 
 You can search your chat history via SQLite database file:
 
@@ -153,6 +214,19 @@ order by internal_id desc;
 
 > **Please note:** SQLite foreign key constraints are disabled by default, and must be **[enabled separately for each database connection](https://sqlite.org/foreignkeys.html#fk_enable)**. The latter can be achieved by running `PRAGMA foreign_keys = ON;` command on an open database connection. By running data altering queries without enabling foreign keys prior to that, you may risk putting your database in an inconsistent state.
 
+## Future roadmap
+
+1. Mobile and desktop apps (in progress).
+2. SMP protocol improvements:
+    - SMP queue redundancy and rotation.
+    - Message delivery confirmation.
+    - Support multiple devices.
+3. Privacy-preserving identity server for optional DNS-based contact/group addresses to simplify connection and discovery, but not used to deliver messages:
+    - keep all your contacts and groups even if you lose the domain.
+    - the server doesn't have information about your contacts and groups.
+4. Media server to optimize sending large files to groups.
+5. Channels server for large groups and broadcast channels.
+
 ## License
 
 [AGPL v3](./LICENSE)

agent-protocol.md

@@ -1,331 +0,0 @@
-# SMP agent protocol - duplex communication over SMP protocol
-
-## Table of contents
-
-- [Abstract](#abstract)
-- [SMP agent](#smp-agent)
-- [SMP agent protocol components](#smp-agent-protocol-components)
-- [Duplex connection procedure](#duplex-connection-procedure)
-- [Communication between SMP agents](#communication-between-smp-agents)
-  - [Message syntax](#messages-between-smp-agents)
-    - [HELLO message](#hello-message)
-    - [REPLY message](#reply-message)
-    - [MSG message](#msg-message)
-- [SMP agent commands](#smp-agent-commands)
-  - [Client commands and server responses](#client-commands-and-server-responses)
-    - [NEW command and INV response](#new-command-and-inv-response)
-    - [JOIN command](#join-command)
-    - [CON notification](#con-notification)
-    - [SUB command](#sub-command)
-    - [SEND command and SENT response](#send-command-and-sent-response)
-    - [MSG notification](#msg-notification)
-    - [END notification](#end-notification)
-    - [OFF command](#off-command)
-    - [DEL command](#del-command)
-- [Connection invitation](#connection-invitation)
-
-## Abstract
-
-The purpose of SMP agent protocol is to define the syntax and the semantics of communications between the client and the agent that connects to [SMP](./simplex-messaging.md) servers.
-
-It provides:
-- convenient protocol to create and manage a bi-directional (duplex) connection to the users of SMP agents consisting of two separate unidirectional (simplex) SMP queues, abstracting away multiple steps required to establish bi-directional connections.
-- management of E2E encryption between SMP agents, generating ephemeral RSA keys for each connection.
-- SMP command authentication on SMP servers, generating ephemeral RSA keys for each SMP queue.
-- TCP transport handshake and encryption with SMP servers.
-- validation of message integrity.
-
-SMP agent protocols provides no encryption or any security on the client side - it is assumed that the agent is executed in the trusted and secure environment.
-
-The future versions of this protocol could provide:
-- managing redundant SMP queues with more than 1 queue in each direction.
-- managing simple symmetric groups as a foundation for chat groups and device synchronization.
-- agent cluster - synchronizing states of multiple agents.
-- secure "synchronous" streams with symmetric message encryption and connection-level authentication (requires extending [SMP protocol](./simplex-messaging.md)) - it can be used, e.g., for file transfers.
-
-## SMP agent
-
-SMP agent is a client-side process or library that communicates via SMP servers using [simplex messaging protocol (SMP)](./simplex-messaging.md) with other SMP agents according to the commands received from its users. This protocol is a middle layer in SMP protocols stack (above SMP protocol but below any application level protocol) - it is intended to be used by client-side applications that need secure asynchronous bi-directional communication channels ("connections").
-
-The agent must have a persistent storage to manage the states of known connections and of the client-side information of two SMP queues that each connection consists of, and also the buffer of the most recent messages. The number of the messages that should be stored is implementation specific, depending on the error management approach that the agent implements; at the very least the agent must store the hash and id of the last received message.
-
-## SMP agent protocol components
-
-SMP agent protocol has 3 main parts:
-
-- the syntax and semantics of messages that SMP agents exchange between each other in order to:
-  - negotiate establishing unidirectional (simplex) encrypted queues on SMP server(s)
-  - exchange client messages and delivery notifications, providing sequential message IDs and message integrity (by including the hash of the previous message).
-- the syntax and semantics of the commands (a higher level interface than SMP protocol) that are sent over TCP or other sequential protocol by agent clients to the agents. This protocol allows to create and manage multiple connections, each consisting of two simplex SMP queues.
-- the syntax and semantics of the message that the clients of SMP agents should send out-of-band (as pre-shared "invitation" including SMP server, queue ID and encryption key) to ensure [E2E encryption][1] the integrity of SMP queues and protection against active attacks ([MITM attacks][2]).
-
-## Duplex connection procedure
-
-![Duplex connection procedure](/diagrams/duplex-messaging/duplex-creating.svg)
-
-The procedure of establishing a duplex connection is explained on the example of Alice and Bob creating a bi-directional connection comprised of two unidirectional (simplex) queues, using SMP agents (A and B) to facilitate it, and two different SMP servers (which could be the same server). It is shown on the diagram above and has these steps:
-
-1. Alice requests the new connection from the SMP agent A using `NEW` command.
-2. Agent A creates an SMP queue on the server (using [SMP protocol](./simplex-messaging.md)) and responds to Alice with the invitation that contains queue information and the encryption key Bob's agent B should use. The invitation format is described in [Connection invitation](#connection-invitation).
-3. Alice sends the invitation to Bob via any secure channel they have (out-of-band message).
-4. Bob sends `JOIN` command with the invitation as a parameter to agent B to accept the connection.
-5. Establishing Alice's SMP queue (with SMP protocol commands):
-  - Agent B sends unauthenticated message to SMP queue with ephemeral key that will be used to authenticate commands to the queue, as described in SMP protocol.
-  - Agent A receives the KEY and secures the queue.
-  - Agent B tries sending authenticated SMP SEND command with agent `HELLO` message until it succeeds. Once it succeeds, Bob's agent "knows" the queue is secured.
-6. Agent B creates a new SMP queue on the server.
-7. Establish Bob's SMP queue:
-  - Agent B sends `REPLY` message with the invitation to this 2nd queue to Alice's agent (via the 1st queue).
-  - Agent A having received this `REPLY` message sends unauthenticated message to SMP queue with Alice agent's ephemeral key that will be used to authenticate commands to the queue, as described in SMP protocol.
-  - Bob's agent receives the key and secures the queue.
-  - Alice's agent keeps sending `HELLO` message until it succeeds.
-8. Agents A and B notify Alice and Bob that connection is established.
-  - Once sending `HELLO` succeeds, Alice's agent sends to Alice `CON` notification that confirms that now both parties can communicate.
-  - Once Bob's agent receives `HELLO` from Alice's agent, it sends to Bob `CON` notification as well.
-
-At this point the duplex connection between Alice and Bob is established, they can use `SEND` command to send messages. The diagram also shows how the connection status changes for both parties, where the first part is the status of the SMP queue to receive messages, and the second part - the status of the queue to send messages.
-
-The most communication happens between the agents and servers, from the point of view of Alice and Bob they have only 3 steps to do:
-
-1. Alice requests a new connection with `NEW` command and receives the invitation.
-2. Alice passes invitation out-of-band to Bob.
-3. Bob accepts the connection by sending `JOIN` command with the invitation to his agent.
-
-## Communication between SMP agents
-
-SMP agents communicate via SMP servers managing creation, deletion and operations of SMP queues.
-
-Agents can use SMP message client body (the part of the SMP message after header - see [SMP protocol](./simplex-messaging.md)) to transmit agent client messages and exchange messages between each other.
-
-Each SMP message client body, once decrypted, contains 3 parts (one of them may include binary message body), as defined by `decryptedSmpMessageBody` syntax:
-
-- `agentMsgHeader` - agent message header that contains sequential agent message ID for a particular SMP queue, agent timestamp (ISO8601) and the hash of the previous message.
-- `agentMessage` - a command/message to the other SMP agent:
-  - to establish the connection with two SMP queues (`helloMsg`, `replyQueueMsg`)
-  - to send and to acknowledge user messages (`clientMsg`, `acknowledgeMsg`)
-  - to notify another agent about queue deletion (`deleteQueueMsg`)
-- `msgPadding` - an optional message padding to make all SMP messages have consistent size as an additional privacy protection measure.
-
-### Messages between SMP agents
-
-Message syntax below uses [ABNF][3] with [case-sensitive strings extension][4].
-
-```abnf
-decryptedSmpMessageBody = agentMsgHeader CRLF agentMessage CRLF msgPadding
-agentMsgHeader = agentMsgId SP agentTimestamp SP previousMsgHash
-agentMsgId = 1*DIGIT ; sequential agent message ID set by the sending agent
-
-agentMessage = helloMsg / replyQueueMsg / deleteQueueMsg
-               / clientMsg / acknowledgeMsg
-
-msgPadding = *OCTET ; optional random bytes to get messages to the same size (as defined in SMP message size)
-
-helloMsg = %s"HELLO" SP signatureVerificationKey [SP %s"NO_ACK"]
-; NO_ACK means that acknowledgements to client messages will NOT be sent in this connection by the agent that sent `HELLO` message.
-signatureVerificationKey = encoded ; base64 encoded
-
-replyQueueMsg = %s"REPLY" SP qInfo ; `qInfo` is the same as in out-of-band message
-; this message can only be sent by the second connection party
-
-deleteQueueMsg = %s"DEL" ; notification that recipient queue will be deleted
-; no need to notify the other party about suspending queue separately, as suspended and deleted queues are the same to the sender 
-; NOT SUPPORTED with the current implementation
-
-clientMsg = %s"MSG" SP size CRLF clientMsgBody CRLF ; CRLF is in addition to CRLF in decryptedSmpMessageBody
-size = 1*DIGIT
-clientMsgBody = *OCTET
-
-acknowledgeMsg = %s"ACK" SP agentMsgId SP ackStatus
-; NOT SUPPORTED with the current implementation
-
-ackStatus = %s"OK" / ackError
-
-ackError = %s"ERR" SP ackErrorType
-
-ackErrorType = ackUnknownMsg / ackProhibitedMsg / ackSyntaxErr
-
-ackUnknownMsg = %s"UNKNOWN"
-
-ackProhibitedMsg = %"PROHIBITED" ; e.g. "HELLO" or "REPLY"
-
-ackSyntaxErr = %"SYNTAX" SP syntaxErrCode
-syntaxErrCode = 1*DIGIT ; TODO
-```
-
-#### HELLO message
-
-This is the first message that both agents send after the respective SMP queue is secured by the receiving agent (see diagram). It contains the verification key that the sender will use to cryptographically sign the messages.
-
-Sending agent might need to retry sending HELLO message, as it would not have any other confirmation that the queue is secured other than the success of sending this message with the signed SEND command of SMP protocol.
-
-#### REPLY message
-
-This is the message that is sent by the agent that received an out-of-band invitation to pass the invitation to the reply SMP queue to the agent that originated the connection (see diagram).
-
-#### MSG message
-
-This is the agent envelope used to send client messages once the connection is established. Do not confuse it with the MSG response from SMP server to the agent and MSG response from SMP agent to the client that are sent in different contexts.
-
-## SMP agent commands
-
-This part describes the transmissions between users and client-side SMP agents: commands that the users send to create and operate duplex connections and SMP agent responses and messages they deliver.
-
-Commands syntax below is provided using [ABNF][3] with [case-sensitive strings extension][4].
-
-Each transmission between the user and SMP agent must have this format/syntax:
-
-```abnf
-agentTransmission = [corrId] CRLF [cAlias] CRLF agentCommand
-
-corrId = 1*(%x21-7F) ; any characters other than control/whitespace
-
-cAlias = cId / cName
-cId = encoded
-cName = 1*(ALPHA / DIGIT / "_" / "-")
-
-agentCommand = (userCmd / agentMsg) CRLF
-userCmd = newCmd / joinCmd
-          / acceptCmd / subscribeCmd
-          / sendCmd / acknowledgeCmd
-          / suspendCmd / deleteCmd
-
-agentMsg = invitation / confirmation
-          / connected / unsubscribed
-          / message / sent / received
-          / ok / error
-
-newCmd = %s"NEW" SP smpServer [SP %s"NO_ACK"]
-; response is `invitation` or `error`
-smpServer = srvHost [":" port] ["#" keyFingerprint]
-srvHost = hostname ; RFC1123, RFC5891
-port = 1*DIGIT
-keyFingerprint = encoded
-
-invitation = %s"INV" SP qInfo
-
-connected = %s"CON"
-
-subscribeCmd = %s"SUB" ; response is `ok` or `error`
-
-unsubscribed = %s"END"
-; when another agent (or another client of the same agent)
-; subscribes to the same SMP queue on the server
-
-joinCmd = %s"JOIN" SP qInfo
-                [SP (smpServer / %s"NO_REPLY")] ; reply queue SMP server
-                ; server from qInfo is used by default
-                [SP %s"NO_ACK"]         
-; response is `connected` or `error`
-
-confirmation = %s"CONF" SP partyId SP partyInfo
-; currently not implemented
-
-acceptCmd = %s"LET" SP partyId ; response is `ok` or `error`
-; currently not implemented
-
-suspendCmd = %s"OFF" ; can be sent by either party, response `ok` or `error`
-
-deleteCmd = %s"DEL" ; can be sent by either party, response `ok` or `error`
-
-sendCmd = %s"SEND" SP msgBody
-; send syntax is similar to that of SMP protocol, but it is wrapped in SMP message
-msgBody = stringMsg | binaryMsg
-stringMsg = ":" string ; until CRLF in the transmission
-string = *(%x01-09 / %x0B-0C / %x0E-FF %) ; any characters other than NUL, CR and LF
-binaryMsg = size CRLF msgBody CRLF ; the last CRLF is in addition to CRLF in the transmission
-size = 1*DIGIT ; size in bytes
-msgBody = *OCTET ; any content of specified size - safe for binary
-
-sent = %s"SENT" SP agentMsgId
-
-message = %s"MSG" SP msgIntegrity
-          SP %s"R=" agentMsgId "," agentTimestamp ; receiving agent
-          SP %s"B=" brokerMsgId "," srvTimestamp ; broker (server)
-          SP %s"S=" agentMsgId "," agentTimestamp ; sending agent
-          SP binaryMsg
-agentMsgId = 1*DIGIT
-srvTimestamp = date-time ; RFC3339
-agentTimestamp = date-time
-msgIntegrity = ok / messageError
-
-messageError = %s"ERR" SP messageErrorType
-messageErrorType = skippedMsgErr / badMsgIdErr / badHashErr
-
-skippedMsgErr = %"IDS" SP missingFromMsgId SP missingToMsgId
-badMsgIdErr = %"ID" SP previousMsgId ; ID is lower than the previous
-badHashErr = %"HASH"
-
-acknowledge = %s"ACK" SP agentMsgId ; ID assigned by receiving agent (in MSG "R")
-; currently not implemented
-
-received = %s"RCVD" SP agentMsgId ; ID assigned by sending agent (in SENT response)
-; currently not implemented
-
-ok = %s"OK"
-
-error = %s"ERR" SP errorType
-
-encoded = base64
-```
-
-### Client commands and server responses
-
-#### NEW command and INV response
-
-`NEW` command is used to create a connection and an invitation to be sent out-of-band to another protocol user. It should be used by the client of the agent that initiates creating a duplex connection.
-
-`INV` response is sent by the agent to the client.
-
-#### JOIN command
-
-It is used to create a connection and accept the invitation received out-of-band. It should be used by the client of the agent that accepts the connection.
-
-#### CON notification
-
-It is sent by both agents managing duplex connection to their clients once the connection is established and ready to accept client messages.
-
-#### SUB command
-
-This command can be used by the client to resume receiving messages from the connection that was created in another TCP/client session. Agent response to this command can be `OK` or `ERR` in case connection does not exist (or can only be used to send connections - e.g. when the reply queue was not created).
-
-#### SEND command and SENT response
-
-`SEND` command is used to the client to send messages
-
-`SENT` response is sent by the agent to confirm that the message was delivered to the SMP server. Message ID in this response is the sequential message number that includes both sent and received messages in the connection.
-
-#### MSG notification
-
-It is sent by the agent to the client when agent receives the message from the SMP server. It has message ID and timestamp from both the receiving and sending agents and from SMP server:
-- recipient agent ID is intended to be used to refer to the message in the future.
-- sender agent ID is intended to be used to identify any missed / skipped message(s)
-- broker ID should be used to detect duplicate deliveries (it would happen if TCP connection is lost before the message is acknowledged by the agent - see [SMP protocol](./simplex-messaging.md))
-
-#### END notification
-
-It is sent by the agent to the client when agent receives SMP protocol `END` notification from SMP server. It indicates that another agent has subscribed to the same SMP queue on the server and the server terminated the subscription of the current agent.
-
-#### OFF command
-
-It is used to suspend the receiving SMP queue - sender will no longer be able to send the messages to the connection, but the recipient can retrieve the remaining messages. Agent response to this command can be `OK` or `ERR`. This command is irreversible.
-
-#### DEL command
-
-It is used to delete the connection and all messages in it, as well as the receiving SMP queue and all messages in it that were remaining on the server. Agent response to this command can be `OK` or `ERR`. This command is irreversible.
-
-## Connection invitation
-
-Connection invitation `qInfo` is generated by SMP agent in response to `newCmd` command (`"NEW"`), used by another party user with `joinCmd` command (`"JOIN"`), and then another invitation is sent by the agent in `replyQueueMsg` and used by the first party agent to connect to the reply queue (the second part of the process is invisible to the users).
-
-Connection invitation is a text with the following syntax:
-
-```
-qInfo = %s"smp::" smpServer "::" queueId "::" ephemeralPublicKey
-queueId = encoded
-ephemeralPublicKey = %s"rsa:" encoded ; RSA key for sender to encrypt messages X509 base64 encoded
-```
-
-[1]: https://en.wikipedia.org/wiki/End-to-end_encryption
-[2]: https://en.wikipedia.org/wiki/Man-in-the-middle_attack
-[3]: https://tools.ietf.org/html/rfc5234
-[4]: https://tools.ietf.org/html/rfc7405

apps/dog-food/ChatOptions.hs

@@ -1,66 +0,0 @@
-{-# LANGUAGE LambdaCase #-}
-{-# LANGUAGE OverloadedStrings #-}
-
-module ChatOptions (getChatOpts, ChatOpts (..)) where
-
-import qualified Data.ByteString.Char8 as B
-import Options.Applicative
-import Simplex.Messaging.Agent.Transmission (SMPServer (..), smpServerP)
-import Simplex.Messaging.Parsers (parseAll)
-import System.FilePath (combine)
-import Types
-
-data ChatOpts = ChatOpts
-  { dbFileName :: String,
-    smpServer :: SMPServer,
-    termMode :: TermMode
-  }
-
-chatOpts :: FilePath -> Parser ChatOpts
-chatOpts appDir =
-  ChatOpts
-    <$> strOption
-      ( long "database"
-          <> short 'd'
-          <> metavar "DB_FILE"
-          <> help ("sqlite database file path (" <> defaultDbFilePath <> ")")
-          <> value defaultDbFilePath
-      )
-    <*> option
-      parseSMPServer
-      ( long "server"
-          <> short 's'
-          <> metavar "SERVER"
-          <> help "SMP server to use (smp1.simplex.im:5223#pLdiGvm0jD1CMblnov6Edd/391OrYsShw+RgdfR0ChA=)"
-          <> value (SMPServer "smp1.simplex.im" (Just "5223") (Just "pLdiGvm0jD1CMblnov6Edd/391OrYsShw+RgdfR0ChA="))
-      )
-    <*> option
-      parseTermMode
-      ( long "term"
-          <> short 't'
-          <> metavar "TERM"
-          <> help ("terminal mode: editor or basic (" <> termModeName TermModeEditor <> ")")
-          <> value TermModeEditor
-      )
-  where
-    defaultDbFilePath = combine appDir "smp-chat.db"
-
-parseSMPServer :: ReadM SMPServer
-parseSMPServer = eitherReader $ parseAll smpServerP . B.pack
-
-parseTermMode :: ReadM TermMode
-parseTermMode = maybeReader $ \case
-  "basic" -> Just TermModeBasic
-  "editor" -> Just TermModeEditor
-  _ -> Nothing
-
-getChatOpts :: FilePath -> IO ChatOpts
-getChatOpts appDir = execParser opts
-  where
-    opts =
-      info
-        (chatOpts appDir <**> helper)
-        ( fullDesc
-            <> header "Chat prototype using Simplex Messaging Protocol (SMP)"
-            <> progDesc "Start chat with DB_FILE file and use SERVER as SMP server"
-        )

apps/dog-food/ChatTerminal.hs

@@ -1,103 +0,0 @@
-{-# LANGUAGE CPP #-}
-{-# LANGUAGE LambdaCase #-}
-{-# LANGUAGE NamedFieldPuns #-}
-{-# LANGUAGE OverloadedStrings #-}
-
-module ChatTerminal
-  ( ChatTerminal (..),
-    newChatTerminal,
-    chatTerminal,
-    ttyContact,
-    ttyFromContact,
-  )
-where
-
-import ChatTerminal.Basic
-import ChatTerminal.Core
-import ChatTerminal.Editor
-import Control.Concurrent (threadDelay)
-import Control.Concurrent.Async (race_)
-import Control.Monad
-import Numeric.Natural
-import Styled
-import System.Terminal
-import Types
-import UnliftIO.STM
-
-newChatTerminal :: Natural -> TermMode -> IO ChatTerminal
-newChatTerminal qSize termMode = do
-  inputQ <- newTBQueueIO qSize
-  outputQ <- newTBQueueIO qSize
-  activeContact <- newTVarIO Nothing
-  termSize <- withTerminal . runTerminalT $ getWindowSize
-  let lastRow = height termSize - 1
-  termState <- newTVarIO newTermState
-  termLock <- newTMVarIO ()
-  nextMessageRow <- newTVarIO lastRow
-  threadDelay 500000 -- this delay is the same as timeout in getTerminalSize
-  return ChatTerminal {inputQ, outputQ, activeContact, termMode, termState, termSize, nextMessageRow, termLock}
-
-newTermState :: TerminalState
-newTermState =
-  TerminalState
-    { inputString = "",
-      inputPosition = 0,
-      inputPrompt = "> ",
-      previousInput = ""
-    }
-
-chatTerminal :: ChatTerminal -> IO ()
-chatTerminal ct
-  | termSize ct == Size 0 0 || termMode ct == TermModeBasic =
-    run basicReceiveFromTTY basicSendToTTY
-  | otherwise = do
-    withTerminal . runTerminalT $ updateInput ct
-    run receiveFromTTY sendToTTY
-  where
-    run receive send = race_ (receive ct) (send ct)
-
-basicReceiveFromTTY :: ChatTerminal -> IO ()
-basicReceiveFromTTY ct =
-  forever $ getLn >>= atomically . writeTBQueue (inputQ ct)
-
-basicSendToTTY :: ChatTerminal -> IO ()
-basicSendToTTY ct = forever $ readOutputQ ct >>= mapM_ putStyledLn
-
-withTermLock :: MonadTerminal m => ChatTerminal -> m () -> m ()
-withTermLock ChatTerminal {termLock} action = do
-  _ <- atomically $ takeTMVar termLock
-  action
-  atomically $ putTMVar termLock ()
-
-receiveFromTTY :: ChatTerminal -> IO ()
-receiveFromTTY ct@ChatTerminal {inputQ, activeContact, termSize, termState} =
-  withTerminal . runTerminalT . forever $
-    getKey >>= processKey >> withTermLock ct (updateInput ct)
-  where
-    processKey :: MonadTerminal m => (Key, Modifiers) -> m ()
-    processKey = \case
-      (EnterKey, _) -> submitInput
-      key -> atomically $ do
-        ac <- readTVar activeContact
-        modifyTVar termState $ updateTermState ac (width termSize) key
-
-    submitInput :: MonadTerminal m => m ()
-    submitInput = do
-      msg <- atomically $ do
-        ts <- readTVar termState
-        let s = inputString ts
-        writeTVar termState $ ts {inputString = "", inputPosition = 0, previousInput = s}
-        writeTBQueue inputQ s
-        return s
-      withTermLock ct $ printMessage ct [styleMessage msg]
-
-sendToTTY :: ChatTerminal -> IO ()
-sendToTTY ct = forever $ do
-  -- `readOutputQ` should be outside of `withTerminal` (see #94)
-  msg <- readOutputQ ct
-  withTerminal . runTerminalT . withTermLock ct $ do
-    printMessage ct msg
-    updateInput ct
-
-readOutputQ :: ChatTerminal -> IO [StyledString]
-readOutputQ = atomically . readTBQueue . outputQ

apps/dog-food/ChatTerminal/Basic.hs

@@ -1,89 +0,0 @@
-{-# LANGUAGE LambdaCase #-}
-
-module ChatTerminal.Basic where
-
-import Control.Monad.IO.Class (liftIO)
-import Styled
-import System.Console.ANSI.Types
-import System.Exit (exitSuccess)
-import System.Terminal as C
-
-getLn :: IO String
-getLn = withTerminal $ runTerminalT getTermLine
-
-putStyledLn :: StyledString -> IO ()
-putStyledLn s =
-  withTerminal . runTerminalT $
-    putStyled s >> C.putLn >> flush
-
--- Currently it is assumed that the message does not have internal line breaks.
--- Previous implementation "kind of" supported them,
--- but it was not determining the number of printed lines correctly
--- because of accounting for control sequences in length
-putStyled :: MonadTerminal m => StyledString -> m ()
-putStyled (s1 :<>: s2) = putStyled s1 >> putStyled s2
-putStyled (Styled [] s) = putString s
-putStyled (Styled sgr s) = setSGR sgr >> putString s >> resetAttributes
-
-setSGR :: MonadTerminal m => [SGR] -> m ()
-setSGR = mapM_ $ \case
-  Reset -> resetAttributes
-  SetConsoleIntensity BoldIntensity -> setAttribute bold
-  SetConsoleIntensity _ -> resetAttribute bold
-  SetItalicized True -> setAttribute italic
-  SetItalicized _ -> resetAttribute italic
-  SetUnderlining NoUnderline -> resetAttribute underlined
-  SetUnderlining _ -> setAttribute underlined
-  SetSwapForegroundBackground True -> setAttribute inverted
-  SetSwapForegroundBackground _ -> resetAttribute inverted
-  SetColor l i c -> setAttribute . layer l . intensity i $ color c
-  SetBlinkSpeed _ -> pure ()
-  SetVisible _ -> pure ()
-  SetRGBColor _ _ -> pure ()
-  SetPaletteColor _ _ -> pure ()
-  SetDefaultColor _ -> pure ()
-  where
-    layer = \case
-      Foreground -> foreground
-      Background -> background
-    intensity = \case
-      Dull -> id
-      Vivid -> bright
-    color = \case
-      Black -> black
-      Red -> red
-      Green -> green
-      Yellow -> yellow
-      Blue -> blue
-      Magenta -> magenta
-      Cyan -> cyan
-      White -> white
-
-getKey :: MonadTerminal m => m (Key, Modifiers)
-getKey =
-  flush >> awaitEvent >>= \case
-    Left Interrupt -> liftIO exitSuccess
-    Right (KeyEvent key ms) -> pure (key, ms)
-    _ -> getKey
-
-getTermLine :: MonadTerminal m => m String
-getTermLine = getChars ""
-  where
-    getChars s =
-      getKey >>= \(key, ms) -> case key of
-        CharKey c
-          | ms == mempty || ms == shiftKey -> do
-            C.putChar c
-            flush
-            getChars (c : s)
-          | otherwise -> getChars s
-        EnterKey -> do
-          C.putLn
-          flush
-          pure $ reverse s
-        BackspaceKey -> do
-          moveCursorBackward 1
-          eraseChars 1
-          flush
-          getChars $ if null s then s else tail s
-        _ -> getChars s

apps/dog-food/ChatTerminal/Editor.hs

@@ -1,61 +0,0 @@
-{-# LANGUAGE NamedFieldPuns #-}
-{-# LANGUAGE ScopedTypeVariables #-}
-
-module ChatTerminal.Editor where
-
-import ChatTerminal.Basic
-import ChatTerminal.Core
-import Styled
-import System.Terminal
-import UnliftIO.STM
-
--- debug :: MonadTerminal m => String -> m ()
--- debug s = do
---   saveCursor
---   setCursorPosition $ Position 0 0
---   putString s
---   restoreCursor
-
-updateInput :: forall m. MonadTerminal m => ChatTerminal -> m ()
-updateInput ct@ChatTerminal {termSize = Size {height, width}, termState, nextMessageRow} = do
-  hideCursor
-  ts <- readTVarIO termState
-  nmr <- readTVarIO nextMessageRow
-  let ih = inputHeight ts ct
-      iStart = height - ih
-      prompt = inputPrompt ts
-      Position {row, col} = positionRowColumn width $ length prompt + inputPosition ts
-  if nmr >= iStart
-    then atomically $ writeTVar nextMessageRow iStart
-    else clearLines nmr iStart
-  setCursorPosition $ Position {row = max nmr iStart, col = 0}
-  putString $ prompt <> inputString ts <> " "
-  eraseInLine EraseForward
-  setCursorPosition $ Position {row = iStart + row, col}
-  showCursor
-  flush
-  where
-    clearLines :: Int -> Int -> m ()
-    clearLines from till
-      | from >= till = return ()
-      | otherwise = do
-        setCursorPosition $ Position {row = from, col = 0}
-        eraseInLine EraseForward
-        clearLines (from + 1) till
-
-printMessage :: forall m. MonadTerminal m => ChatTerminal -> [StyledString] -> m ()
-printMessage ChatTerminal {termSize = Size {height, width}, nextMessageRow} msg = do
-  nmr <- readTVarIO nextMessageRow
-  setCursorPosition $ Position {row = nmr, col = 0}
-  mapM_ printStyled msg
-  flush
-  let lc = sum $ map lineCount msg
-  atomically . writeTVar nextMessageRow $ min (height - 1) (nmr + lc)
-  where
-    lineCount :: StyledString -> Int
-    lineCount s = sLength s `div` width + 1
-    printStyled :: StyledString -> m ()
-    printStyled s = do
-      putStyled s
-      eraseInLine EraseForward
-      putLn

apps/dog-food/Main.hs

@@ -1,301 +0,0 @@
-{-# LANGUAGE DataKinds #-}
-{-# LANGUAGE DuplicateRecordFields #-}
-{-# LANGUAGE FlexibleContexts #-}
-{-# LANGUAGE GADTs #-}
-{-# LANGUAGE LambdaCase #-}
-{-# LANGUAGE NamedFieldPuns #-}
-{-# LANGUAGE OverloadedStrings #-}
-{-# LANGUAGE ScopedTypeVariables #-}
-
-module Main where
-
-import ChatOptions
-import ChatTerminal
-import ChatTerminal.Core
-import Control.Applicative ((<|>))
-import Control.Concurrent.STM
-import Control.Logger.Simple
-import Control.Monad.Reader
-import Data.Attoparsec.ByteString.Char8 (Parser)
-import qualified Data.Attoparsec.ByteString.Char8 as A
-import Data.ByteString.Char8 (ByteString)
-import qualified Data.ByteString.Char8 as B
-import Data.Functor (($>))
-import Data.List (intersperse)
-import qualified Data.Text as T
-import Data.Text.Encoding
-import Numeric.Natural
-import Simplex.Markdown
-import Simplex.Messaging.Agent (getSMPAgentClient, runSMPAgentClient)
-import Simplex.Messaging.Agent.Client (AgentClient (..))
-import Simplex.Messaging.Agent.Env.SQLite
-import Simplex.Messaging.Agent.Transmission
-import Simplex.Messaging.Client (smpDefaultConfig)
-import Simplex.Messaging.Parsers (parseAll)
-import Simplex.Messaging.Util (raceAny_)
-import Styled
-import System.Console.ANSI.Types
-import System.Directory (getAppUserDataDirectory)
-import Types
-
-cfg :: AgentConfig
-cfg =
-  AgentConfig
-    { tcpPort = undefined, -- TODO maybe take it out of config
-      rsaKeySize = 2048 `div` 8,
-      connIdBytes = 12,
-      tbqSize = 16,
-      dbFile = "smp-chat.db",
-      smpCfg = smpDefaultConfig
-    }
-
-logCfg :: LogConfig
-logCfg = LogConfig {lc_file = Nothing, lc_stderr = True}
-
-data ChatClient = ChatClient
-  { inQ :: TBQueue ChatCommand,
-    outQ :: TBQueue ChatResponse,
-    smpServer :: SMPServer
-  }
-
--- | GroupMessage ChatGroup ByteString
--- | AddToGroup Contact
-data ChatCommand
-  = ChatHelp
-  | MarkdownHelp
-  | AddConnection Contact
-  | Connect Contact SMPQueueInfo
-  | DeleteConnection Contact
-  | SendMessage Contact ByteString
-
-chatCommandP :: Parser ChatCommand
-chatCommandP =
-  ("/help" <|> "/h") $> ChatHelp
-    <|> ("/markdown" <|> "/m") $> MarkdownHelp
-    <|> ("/add " <|> "/a ") *> (AddConnection <$> contact)
-    <|> ("/connect " <> "/c ") *> connect
-    <|> ("/delete " <> "/d ") *> (DeleteConnection <$> contact)
-    <|> "@" *> sendMessage
-  where
-    connect = Connect <$> contact <* A.space <*> smpQueueInfoP
-    sendMessage = SendMessage <$> contact <* A.space <*> A.takeByteString
-    contact = Contact <$> A.takeTill (== ' ')
-
-data ChatResponse
-  = ChatHelpInfo
-  | MarkdownInfo
-  | Invitation SMPQueueInfo
-  | Connected Contact
-  | Confirmation Contact
-  | ReceivedMessage Contact ByteString MsgIntegrity
-  | Disconnected Contact
-  | YesYes
-  | ContactError ConnectionErrorType Contact
-  | ErrorInput ByteString
-  | ChatError AgentErrorType
-  | NoChatResponse
-
-serializeChatResponse :: ChatOpts -> ChatResponse -> [StyledString]
-serializeChatResponse _ = \case
-  ChatHelpInfo -> chatHelpInfo
-  MarkdownInfo -> markdownInfo
-  Invitation qInfo ->
-    [ "pass this invitation to your contact (via any channel): ",
-      "",
-      (bPlain . serializeSmpQueueInfo) qInfo,
-      "",
-      "and ask them to connect: /c <name_for_you> <invitation_above>"
-    ]
-  Connected c -> [ttyContact c <> " connected"]
-  Confirmation c -> [ttyContact c <> " ok"]
-  ReceivedMessage c t mi ->
-    prependFirst (ttyFromContact c) (msgPlain t)
-      ++ showIntegrity mi
-  Disconnected c -> ["disconnected from " <> ttyContact c <> " - restart chat"]
-  YesYes -> ["you got it!"]
-  ContactError e c -> case e of
-    UNKNOWN -> ["no contact " <> ttyContact c]
-    DUPLICATE -> ["contact " <> ttyContact c <> " already exists"]
-    SIMPLEX -> ["contact " <> ttyContact c <> " did not accept invitation yet"]
-  ErrorInput t -> ["invalid input: " <> bPlain t]
-  ChatError e -> ["chat error: " <> plain (show e)]
-  NoChatResponse -> [""]
-  where
-    prependFirst :: StyledString -> [StyledString] -> [StyledString]
-    prependFirst s [] = [s]
-    prependFirst s (s' : ss) = (s <> s') : ss
-    msgPlain :: ByteString -> [StyledString]
-    msgPlain = map styleMarkdownText . T.lines . safeDecodeUtf8
-    showIntegrity :: MsgIntegrity -> [StyledString]
-    showIntegrity MsgOk = []
-    showIntegrity (MsgError err) = msgError $ case err of
-      MsgSkipped fromId toId ->
-        "skipped message ID " <> show fromId
-          <> if fromId == toId then "" else ".." <> show toId
-      MsgBadId msgId -> "unexpected message ID " <> show msgId
-      MsgBadHash -> "incorrect message hash"
-      MsgDuplicate -> "duplicate message ID"
-    msgError :: String -> [StyledString]
-    msgError s = [styled (Colored Red) s]
-
-chatHelpInfo :: [StyledString]
-chatHelpInfo =
-  map
-    styleMarkdown
-    [ Markdown (Colored Cyan) "Using Simplex chat prototype.",
-      "Follow these steps to set up a connection:",
-      "",
-      Markdown (Colored Green) "Step 1: " <> Markdown (Colored Cyan) "/add bob" <> " -- Alice adds her contact, Bob (she can use any name).",
-      indent <> "Alice should send the invitation printed by the /add command",
-      indent <> "to her contact, Bob, out-of-band, via any trusted channel.",
-      "",
-      Markdown (Colored Green) "Step 2: " <> Markdown (Colored Cyan) "/connect alice <invitation>" <> " -- Bob accepts the invitation.",
-      indent <> "Bob also can use any name for his contact, Alice,",
-      indent <> "followed by the invitation he received out-of-band.",
-      "",
-      Markdown (Colored Green) "Step 3: " <> "Bob and Alice are notified that the connection is set up,",
-      indent <> "both can now send messages:",
-      indent <> Markdown (Colored Cyan) "@bob Hello, Bob!" <> " -- Alice messages Bob.",
-      indent <> Markdown (Colored Cyan) "@alice Hey, Alice!" <> " -- Bob replies to Alice.",
-      "",
-      Markdown (Colored Green) "Other commands:",
-      indent <> Markdown (Colored Cyan) "/delete" <> " -- deletes contact and all messages with them.",
-      indent <> Markdown (Colored Cyan) "/markdown" <> " -- prints the supported markdown syntax.",
-      "",
-      "The commands may be abbreviated to a single letter: " <> listCommands ["/a", "/c", "/d", "/m"]
-    ]
-  where
-    listCommands = mconcat . intersperse ", " . map highlight
-    highlight = Markdown (Colored Cyan)
-    indent = "        "
-
-markdownInfo :: [StyledString]
-markdownInfo =
-  map
-    styleMarkdown
-    [ "Markdown:",
-      "  *bold*          - " <> Markdown Bold "bold text",
-      "  _italic_        - " <> Markdown Italic "italic text" <> " (shown as underlined)",
-      "  +underlined+    - " <> Markdown Underline "underlined text",
-      "  ~strikethrough~ - " <> Markdown StrikeThrough "strikethrough text" <> " (shown as inverse)",
-      "  `code snippet`  - " <> Markdown Snippet "a + b // no *markdown* here",
-      "  !1 text!        - " <> red "red text" <> " (1-6: red, green, blue, yellow, cyan, magenta)",
-      "  #secret#        - " <> Markdown Secret "secret text" <> " (can be copy-pasted)"
-    ]
-  where
-    red = Markdown (Colored Red)
-
-main :: IO ()
-main = do
-  opts@ChatOpts {dbFileName, smpServer, termMode} <- welcomeGetOpts
-  t <- getChatClient smpServer
-  ct <- newChatTerminal (tbqSize cfg) termMode
-  -- setLogLevel LogInfo -- LogError
-  -- withGlobalLogging logCfg $ do
-  env <- newSMPAgentEnv cfg {dbFile = dbFileName}
-  dogFoodChat t ct env opts
-
-welcomeGetOpts :: IO ChatOpts
-welcomeGetOpts = do
-  appDir <- getAppUserDataDirectory "simplex"
-  opts@ChatOpts {dbFileName} <- getChatOpts appDir
-  putStrLn "SimpleX chat prototype"
-  putStrLn $ "db: " <> dbFileName
-  putStrLn "type \"/help\" or \"/h\" for usage info"
-  pure opts
-
-dogFoodChat :: ChatClient -> ChatTerminal -> Env -> ChatOpts -> IO ()
-dogFoodChat t ct env opts = do
-  c <- runReaderT getSMPAgentClient env
-  raceAny_
-    [ runReaderT (runSMPAgentClient c) env,
-      sendToAgent t ct c,
-      sendToChatTerm t ct opts,
-      receiveFromAgent t ct c,
-      receiveFromChatTerm t ct,
-      chatTerminal ct
-    ]
-
-getChatClient :: SMPServer -> IO ChatClient
-getChatClient srv = atomically $ newChatClient (tbqSize cfg) srv
-
-newChatClient :: Natural -> SMPServer -> STM ChatClient
-newChatClient qSize smpServer = do
-  inQ <- newTBQueue qSize
-  outQ <- newTBQueue qSize
-  return ChatClient {inQ, outQ, smpServer}
-
-receiveFromChatTerm :: ChatClient -> ChatTerminal -> IO ()
-receiveFromChatTerm t ct = forever $ do
-  atomically (readTBQueue $ inputQ ct)
-    >>= processOrError . parseAll chatCommandP . encodeUtf8 . T.pack
-  where
-    processOrError = \case
-      Left err -> writeOutQ . ErrorInput $ B.pack err
-      Right ChatHelp -> writeOutQ ChatHelpInfo
-      Right MarkdownHelp -> writeOutQ MarkdownInfo
-      Right cmd -> atomically $ writeTBQueue (inQ t) cmd
-    writeOutQ = atomically . writeTBQueue (outQ t)
-
-sendToChatTerm :: ChatClient -> ChatTerminal -> ChatOpts -> IO ()
-sendToChatTerm ChatClient {outQ} ChatTerminal {outputQ} opts = forever $ do
-  atomically (readTBQueue outQ) >>= \case
-    NoChatResponse -> return ()
-    resp -> atomically . writeTBQueue outputQ $ serializeChatResponse opts resp
-
-sendToAgent :: ChatClient -> ChatTerminal -> AgentClient -> IO ()
-sendToAgent ChatClient {inQ, smpServer} ct AgentClient {rcvQ} = do
-  atomically $ writeTBQueue rcvQ ("1", "", SUBALL) -- hack for subscribing to all
-  forever . atomically $ do
-    cmd <- readTBQueue inQ
-    writeTBQueue rcvQ `mapM_` agentTransmission cmd
-    setActiveContact cmd
-  where
-    setActiveContact :: ChatCommand -> STM ()
-    setActiveContact = \case
-      SendMessage a _ -> setActive ct a
-      DeleteConnection a -> unsetActive ct a
-      _ -> pure ()
-    agentTransmission :: ChatCommand -> Maybe (ATransmission 'Client)
-    agentTransmission = \case
-      AddConnection a -> transmission a $ NEW smpServer
-      Connect a qInfo -> transmission a $ JOIN qInfo $ ReplyVia smpServer
-      DeleteConnection a -> transmission a DEL
-      SendMessage a msg -> transmission a $ SEND msg
-      ChatHelp -> Nothing
-      MarkdownHelp -> Nothing
-    transmission :: Contact -> ACommand 'Client -> Maybe (ATransmission 'Client)
-    transmission (Contact a) cmd = Just ("1", a, cmd)
-
-receiveFromAgent :: ChatClient -> ChatTerminal -> AgentClient -> IO ()
-receiveFromAgent t ct c = forever . atomically $ do
-  resp <- chatResponse <$> readTBQueue (sndQ c)
-  writeTBQueue (outQ t) resp
-  setActiveContact resp
-  where
-    chatResponse :: ATransmission 'Agent -> ChatResponse
-    chatResponse (_, a, resp) = case resp of
-      INV qInfo -> Invitation qInfo
-      CON -> Connected contact
-      END -> Disconnected contact
-      MSG {msgBody, msgIntegrity} -> ReceivedMessage contact msgBody msgIntegrity
-      SENT _ -> NoChatResponse
-      OK -> Confirmation contact
-      ERR (CONN e) -> ContactError e contact
-      ERR e -> ChatError e
-      where
-        contact = Contact a
-    setActiveContact :: ChatResponse -> STM ()
-    setActiveContact = \case
-      Connected a -> setActive ct a
-      ReceivedMessage a _ _ -> setActive ct a
-      Disconnected a -> unsetActive ct a
-      _ -> pure ()
-
-setActive :: ChatTerminal -> Contact -> STM ()
-setActive ct = writeTVar (activeContact ct) . Just
-
-unsetActive :: ChatTerminal -> Contact -> STM ()
-unsetActive ct a = modifyTVar (activeContact ct) unset
-  where
-    unset a' = if Just a == a' then Nothing else a'

apps/dog-food/Types.hs

@@ -1,14 +0,0 @@
-{-# LANGUAGE LambdaCase #-}
-
-module Types where
-
-import Data.ByteString.Char8 (ByteString)
-
-newtype Contact = Contact {toBs :: ByteString} deriving (Eq)
-
-data TermMode = TermModeBasic | TermModeEditor deriving (Eq)
-
-termModeName :: TermMode -> String
-termModeName = \case
-  TermModeBasic -> "basic"
-  TermModeEditor -> "editor"

apps/simplex-chat/Demo.hs

@@ -0,0 +1,119 @@
+{-# LANGUAGE LambdaCase #-}
+{-# LANGUAGE OverloadedStrings #-}
+
+module Demo where
+
+import Simplex.Chat.Styled
+import System.Console.ANSI.Types
+import System.Terminal
+
+someViewUpdate :: Monad m => m ()
+someViewUpdate = pure ()
+
+chatLayoutDemo :: MonadTerminal m => m ()
+chatLayoutDemo =
+  mapM_
+    putStyledLn
+    [ "  search " <> Styled gray "(ctrl-s)   " <> lineV <> Styled toContact " @bob " <> "Bob Roberts                                    " <> Styled greenColor "@john" <> "",
+      "                    " <> lineV <> Styled gray " 14:15 online                             profile (ctrl-p)",
+      lineH 20 <> crossover <> lineH 59,
+      "* " <> Styled [SetConsoleIntensity BoldIntensity] "all chats   " <> "      " <> lineV <> "",
+      Styled gray "  (ctrl-a)          " <> lineV <> "",
+      "*" <> Styled toContact " @alice      " <> Styled darkGray "14:37 " <> lineV <> "",
+      Styled gray "  Hello there!  ... " <> lineV <> "",
+      Styled selected " " <> Styled (toContact <> selected) " @bob        " <> Styled (selected <> gray) "12:35 " <> lineV <> "",
+      Styled selected "  All good, John... " <> lineV <> "",
+      "*" <> Styled group " #team       " <> Styled darkGray "10:55 " <> lineV <> "",
+      Styled gray "  What's up ther... " <> lineV <> "",
+      " " <> Styled toContact " @tom          " <> Styled darkGray "Wed " <> lineV <> "",
+      Styled gray "  Have you seen ... " <> lineV <> "",
+      "                    " <> lineV,
+      "                    " <> lineV,
+      "                    " <> lineV,
+      "                    " <> lineV,
+      "                    " <> lineV,
+      "                    " <> lineV <> Styled greenColor " ✔︎" <> Styled darkGray " 12:30" <> Styled toContact " @bob" <> " hey bob - how is it going?",
+      "                    " <> lineV <> Styled greenColor " ✔︎" <> Styled darkGray "      " <> Styled toContact "     " <> " let's meet soon!",
+      "                    " <> lineV <> " *" <> Styled darkGray " 12:35" <> Styled contact " bob>" <> " All good, John! How are you?",
+      "                    " <> teeL <> lineH 59,
+      "                    " <> lineV <> " > " <> Styled toContact "@bob" <> " 😀 This is the message that will be sent to @bob"
+    ]
+    >> putStyled (Styled ctrlKeys " help (ctrl-h)  new contact (ctrl-n)  choose chat (ctrl-↓↑)  new group (ctrl-g) ")
+
+contact :: [SGR]
+contact = [SetConsoleIntensity BoldIntensity, SetColor Foreground Vivid Yellow]
+
+toContact :: [SGR]
+toContact = [SetConsoleIntensity BoldIntensity, SetColor Foreground Vivid Cyan]
+
+group :: [SGR]
+group = [SetConsoleIntensity BoldIntensity, SetColor Foreground Vivid Cyan]
+
+selected :: [SGR]
+selected = [SetColor Background Vivid Black]
+
+ctrlKeys :: [SGR]
+ctrlKeys = [SetColor Background Dull White, SetColor Foreground Dull Black]
+
+gray :: [SGR]
+gray = [SetColor Foreground Dull White]
+
+darkGray :: [SGR]
+darkGray = [SetColor Foreground Vivid Black]
+
+greenColor :: [SGR]
+greenColor = [SetColor Foreground Vivid Green]
+
+lineV :: StyledString
+lineV = Styled selected " " -- "\x2502"
+
+lineH :: Int -> StyledString
+lineH n = Styled darkGray $ replicate n '\x2500'
+
+teeL :: StyledString
+teeL = Styled selected " " -- "\x251C"
+
+crossover :: StyledString
+crossover = Styled selected " " -- "\x253C"
+
+putStyledLn :: MonadTerminal m => StyledString -> m ()
+putStyledLn s = putStyled s >> putLn
+
+putStyled :: MonadTerminal m => StyledString -> m ()
+putStyled (s1 :<>: s2) = putStyled s1 >> putStyled s2
+putStyled (Styled [] s) = putString s
+putStyled (Styled sgr s) = setSGR sgr >> putString s >> resetAttributes
+
+setSGR :: MonadTerminal m => [SGR] -> m ()
+setSGR = mapM_ $ \case
+  Reset -> resetAttributes
+  SetConsoleIntensity BoldIntensity -> setAttribute bold
+  SetConsoleIntensity _ -> resetAttribute bold
+  SetItalicized True -> setAttribute italic
+  SetItalicized _ -> resetAttribute italic
+  SetUnderlining NoUnderline -> resetAttribute underlined
+  SetUnderlining _ -> setAttribute underlined
+  SetSwapForegroundBackground True -> setAttribute inverted
+  SetSwapForegroundBackground _ -> resetAttribute inverted
+  SetColor l i c -> setAttribute . layer l . intensity i $ color c
+  SetBlinkSpeed _ -> pure ()
+  SetVisible _ -> pure ()
+  SetRGBColor _ _ -> pure ()
+  SetPaletteColor _ _ -> pure ()
+  SetDefaultColor _ -> pure ()
+  where
+    layer = \case
+      Foreground -> foreground
+      Background -> background
+    intensity = \case
+      Dull -> id
+      Vivid -> bright
+    color = \case
+      Black -> black
+      Red -> red
+      Green -> green
+      Yellow -> yellow
+      Blue -> blue
+      Magenta -> magenta
+      Cyan -> cyan
+      White -> white

apps/simplex-chat/Main.hs

@@ -0,0 +1,58 @@
+{-# LANGUAGE DuplicateRecordFields #-}
+{-# LANGUAGE FlexibleContexts #-}
+{-# LANGUAGE NamedFieldPuns #-}
+{-# LANGUAGE OverloadedStrings #-}
+
+module Main where
+
+import Simplex.Chat
+import Simplex.Chat.Options
+import System.Directory (getAppUserDataDirectory)
+import System.Terminal (withTerminal)
+
+main :: IO ()
+main = do
+  opts <- welcomeGetOpts
+  t <- withTerminal pure
+  simplexChat defaultChatConfig opts t
+
+welcomeGetOpts :: IO ChatOpts
+welcomeGetOpts = do
+  appDir <- getAppUserDataDirectory "simplex"
+  opts@ChatOpts {dbFile} <- getChatOpts appDir
+  putStrLn "SimpleX chat prototype v0.4.0"
+  putStrLn $ "db: " <> dbFile <> ".chat.db, " <> dbFile <> ".agent.db"
+  putStrLn "type \"/help\" or \"/h\" for usage info"
+  pure opts
+
+-- defaultSettings :: C.Size -> C.VirtualTerminalSettings
+-- defaultSettings size =
+--   C.VirtualTerminalSettings
+--     { C.virtualType = "xterm",
+--       C.virtualWindowSize = pure size,
+--       C.virtualEvent = retry,
+--       C.virtualInterrupt = retry
+--     }
+
+-- main :: IO ()
+-- main = do
+--   void $ createStore "simplex-chat.db" 4
+
+--   hFlush stdout
+--   -- ChatTerminal {termSize} <- newChatTerminal
+--   -- pos <- C.withVirtualTerminal (defaultSettings termSize) $
+--   --   \t -> runTerminalT (C.setAlternateScreenBuffer True >> C.putString "a" >> C.flush >> C.getCursorPosition) t
+--   -- print pos
+--   -- race_ (printEvents t) (updateTerminal t)
+--   void . withTerminal . runTerminalT $ chatLayoutDemo >> C.flush >> C.awaitEvent
+
+-- printEvents :: C.VirtualTerminal -> IO ()
+-- printEvents t = forever $ do
+--   event <- withTerminal . runTerminalT $ C.flush >> C.awaitEvent
+--   runTerminalT (putStringLn $ show event) t
+
+-- updateTerminal :: C.VirtualTerminal -> IO ()
+-- updateTerminal t = forever $ do
+--   threadDelay 10000
+--   win <- readTVarIO $ C.virtualWindow t
+--   withTerminal . runTerminalT $ mapM_ C.putStringLn win >> C.flush

definitions/LICENSE

@@ -1,661 +0,0 @@
-                    GNU AFFERO GENERAL PUBLIC LICENSE
-                       Version 3, 19 November 2007
-
- Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
- Everyone is permitted to copy and distribute verbatim copies
- of this license document, but changing it is not allowed.
-
-                            Preamble
-
-  The GNU Affero General Public License is a free, copyleft license for
-software and other kinds of works, specifically designed to ensure
-cooperation with the community in the case of network server software.
-
-  The licenses for most software and other practical works are designed
-to take away your freedom to share and change the works.  By contrast,
-our General Public Licenses are intended to guarantee your freedom to
-share and change all versions of a program--to make sure it remains free
-software for all its users.
-
-  When we speak of free software, we are referring to freedom, not
-price.  Our General Public Licenses are designed to make sure that you
-have the freedom to distribute copies of free software (and charge for
-them if you wish), that you receive source code or can get it if you
-want it, that you can change the software or use pieces of it in new
-free programs, and that you know you can do these things.
-
-  Developers that use our General Public Licenses protect your rights
-with two steps: (1) assert copyright on the software, and (2) offer
-you this License which gives you legal permission to copy, distribute
-and/or modify the software.
-
-  A secondary benefit of defending all users' freedom is that
-improvements made in alternate versions of the program, if they
-receive widespread use, become available for other developers to
-incorporate.  Many developers of free software are heartened and
-encouraged by the resulting cooperation.  However, in the case of
-software used on network servers, this result may fail to come about.
-The GNU General Public License permits making a modified version and
-letting the public access it on a server without ever releasing its
-source code to the public.
-
-  The GNU Affero General Public License is designed specifically to
-ensure that, in such cases, the modified source code becomes available
-to the community.  It requires the operator of a network server to
-provide the source code of the modified version running there to the
-users of that server.  Therefore, public use of a modified version, on
-a publicly accessible server, gives the public access to the source
-code of the modified version.
-
-  An older license, called the Affero General Public License and
-published by Affero, was designed to accomplish similar goals.  This is
-a different license, not a version of the Affero GPL, but Affero has
-released a new version of the Affero GPL which permits relicensing under
-this license.
-
-  The precise terms and conditions for copying, distribution and
-modification follow.
-
-                       TERMS AND CONDITIONS
-
-  0. Definitions.
-
-  "This License" refers to version 3 of the GNU Affero General Public License.
-
-  "Copyright" also means copyright-like laws that apply to other kinds of
-works, such as semiconductor masks.
-
-  "The Program" refers to any copyrightable work licensed under this
-License.  Each licensee is addressed as "you".  "Licensees" and
-"recipients" may be individuals or organizations.
-
-  To "modify" a work means to copy from or adapt all or part of the work
-in a fashion requiring copyright permission, other than the making of an
-exact copy.  The resulting work is called a "modified version" of the
-earlier work or a work "based on" the earlier work.
-
-  A "covered work" means either the unmodified Program or a work based
-on the Program.
-
-  To "propagate" a work means to do anything with it that, without
-permission, would make you directly or secondarily liable for
-infringement under applicable copyright law, except executing it on a
-computer or modifying a private copy.  Propagation includes copying,
-distribution (with or without modification), making available to the
-public, and in some countries other activities as well.
-
-  To "convey" a work means any kind of propagation that enables other
-parties to make or receive copies.  Mere interaction with a user through
-a computer network, with no transfer of a copy, is not conveying.
-
-  An interactive user interface displays "Appropriate Legal Notices"
-to the extent that it includes a convenient and prominently visible
-feature that (1) displays an appropriate copyright notice, and (2)
-tells the user that there is no warranty for the work (except to the
-extent that warranties are provided), that licensees may convey the
-work under this License, and how to view a copy of this License.  If
-the interface presents a list of user commands or options, such as a
-menu, a prominent item in the list meets this criterion.
-
-  1. Source Code.
-
-  The "source code" for a work means the preferred form of the work
-for making modifications to it.  "Object code" means any non-source
-form of a work.
-
-  A "Standard Interface" means an interface that either is an official
-standard defined by a recognized standards body, or, in the case of
-interfaces specified for a particular programming language, one that
-is widely used among developers working in that language.
-
-  The "System Libraries" of an executable work include anything, other
-than the work as a whole, that (a) is included in the normal form of
-packaging a Major Component, but which is not part of that Major
-Component, and (b) serves only to enable use of the work with that
-Major Component, or to implement a Standard Interface for which an
-implementation is available to the public in source code form.  A
-"Major Component", in this context, means a major essential component
-(kernel, window system, and so on) of the specific operating system
-(if any) on which the executable work runs, or a compiler used to
-produce the work, or an object code interpreter used to run it.
-
-  The "Corresponding Source" for a work in object code form means all
-the source code needed to generate, install, and (for an executable
-work) run the object code and to modify the work, including scripts to
-control those activities.  However, it does not include the work's
-System Libraries, or general-purpose tools or generally available free
-programs which are used unmodified in performing those activities but
-which are not part of the work.  For example, Corresponding Source
-includes interface definition files associated with source files for
-the work, and the source code for shared libraries and dynamically
-linked subprograms that the work is specifically designed to require,
-such as by intimate data communication or control flow between those
-subprograms and other parts of the work.
-
-  The Corresponding Source need not include anything that users
-can regenerate automatically from other parts of the Corresponding
-Source.
-
-  The Corresponding Source for a work in source code form is that
-same work.
-
-  2. Basic Permissions.
-
-  All rights granted under this License are granted for the term of
-copyright on the Program, and are irrevocable provided the stated
-conditions are met.  This License explicitly affirms your unlimited
-permission to run the unmodified Program.  The output from running a
-covered work is covered by this License only if the output, given its
-content, constitutes a covered work.  This License acknowledges your
-rights of fair use or other equivalent, as provided by copyright law.
-
-  You may make, run and propagate covered works that you do not
-convey, without conditions so long as your license otherwise remains
-in force.  You may convey covered works to others for the sole purpose
-of having them make modifications exclusively for you, or provide you
-with facilities for running those works, provided that you comply with
-the terms of this License in conveying all material for which you do
-not control copyright.  Those thus making or running the covered works
-for you must do so exclusively on your behalf, under your direction
-and control, on terms that prohibit them from making any copies of
-your copyrighted material outside their relationship with you.
-
-  Conveying under any other circumstances is permitted solely under
-the conditions stated below.  Sublicensing is not allowed; section 10
-makes it unnecessary.
-
-  3. Protecting Users' Legal Rights From Anti-Circumvention Law.
-
-  No covered work shall be deemed part of an effective technological
-measure under any applicable law fulfilling obligations under article
-11 of the WIPO copyright treaty adopted on 20 December 1996, or
-similar laws prohibiting or restricting circumvention of such
-measures.
-
-  When you convey a covered work, you waive any legal power to forbid
-circumvention of technological measures to the extent such circumvention
-is effected by exercising rights under this License with respect to
-the covered work, and you disclaim any intention to limit operation or
-modification of the work as a means of enforcing, against the work's
-users, your or third parties' legal rights to forbid circumvention of
-technological measures.
-
-  4. Conveying Verbatim Copies.
-
-  You may convey verbatim copies of the Program's source code as you
-receive it, in any medium, provided that you conspicuously and
-appropriately publish on each copy an appropriate copyright notice;
-keep intact all notices stating that this License and any
-non-permissive terms added in accord with section 7 apply to the code;
-keep intact all notices of the absence of any warranty; and give all
-recipients a copy of this License along with the Program.
-
-  You may charge any price or no price for each copy that you convey,
-and you may offer support or warranty protection for a fee.
-
-  5. Conveying Modified Source Versions.
-
-  You may convey a work based on the Program, or the modifications to
-produce it from the Program, in the form of source code under the
-terms of section 4, provided that you also meet all of these conditions:
-
-    a) The work must carry prominent notices stating that you modified
-    it, and giving a relevant date.
-
-    b) The work must carry prominent notices stating that it is
-    released under this License and any conditions added under section
-    7.  This requirement modifies the requirement in section 4 to
-    "keep intact all notices".
-
-    c) You must license the entire work, as a whole, under this
-    License to anyone who comes into possession of a copy.  This
-    License will therefore apply, along with any applicable section 7
-    additional terms, to the whole of the work, and all its parts,
-    regardless of how they are packaged.  This License gives no
-    permission to license the work in any other way, but it does not
-    invalidate such permission if you have separately received it.
-
-    d) If the work has interactive user interfaces, each must display
-    Appropriate Legal Notices; however, if the Program has interactive
-    interfaces that do not display Appropriate Legal Notices, your
-    work need not make them do so.
-
-  A compilation of a covered work with other separate and independent
-works, which are not by their nature extensions of the covered work,
-and which are not combined with it such as to form a larger program,
-in or on a volume of a storage or distribution medium, is called an
-"aggregate" if the compilation and its resulting copyright are not
-used to limit the access or legal rights of the compilation's users
-beyond what the individual works permit.  Inclusion of a covered work
-in an aggregate does not cause this License to apply to the other
-parts of the aggregate.
-
-  6. Conveying Non-Source Forms.
-
-  You may convey a covered work in object code form under the terms
-of sections 4 and 5, provided that you also convey the
-machine-readable Corresponding Source under the terms of this License,
-in one of these ways:
-
-    a) Convey the object code in, or embodied in, a physical product
-    (including a physical distribution medium), accompanied by the
-    Corresponding Source fixed on a durable physical medium
-    customarily used for software interchange.
-
-    b) Convey the object code in, or embodied in, a physical product
-    (including a physical distribution medium), accompanied by a
-    written offer, valid for at least three years and valid for as
-    long as you offer spare parts or customer support for that product
-    model, to give anyone who possesses the object code either (1) a
-    copy of the Corresponding Source for all the software in the
-    product that is covered by this License, on a durable physical
-    medium customarily used for software interchange, for a price no
-    more than your reasonable cost of physically performing this
-    conveying of source, or (2) access to copy the
-    Corresponding Source from a network server at no charge.
-
-    c) Convey individual copies of the object code with a copy of the
-    written offer to provide the Corresponding Source.  This
-    alternative is allowed only occasionally and noncommercially, and
-    only if you received the object code with such an offer, in accord
-    with subsection 6b.
-
-    d) Convey the object code by offering access from a designated
-    place (gratis or for a charge), and offer equivalent access to the
-    Corresponding Source in the same way through the same place at no
-    further charge.  You need not require recipients to copy the
-    Corresponding Source along with the object code.  If the place to
-    copy the object code is a network server, the Corresponding Source
-    may be on a different server (operated by you or a third party)
-    that supports equivalent copying facilities, provided you maintain
-    clear directions next to the object code saying where to find the
-    Corresponding Source.  Regardless of what server hosts the
-    Corresponding Source, you remain obligated to ensure that it is
-    available for as long as needed to satisfy these requirements.
-
-    e) Convey the object code using peer-to-peer transmission, provided
-    you inform other peers where the object code and Corresponding
-    Source of the work are being offered to the general public at no
-    charge under subsection 6d.
-
-  A separable portion of the object code, whose source code is excluded
-from the Corresponding Source as a System Library, need not be
-included in conveying the object code work.
-
-  A "User Product" is either (1) a "consumer product", which means any
-tangible personal property which is normally used for personal, family,
-or household purposes, or (2) anything designed or sold for incorporation
-into a dwelling.  In determining whether a product is a consumer product,
-doubtful cases shall be resolved in favor of coverage.  For a particular
-product received by a particular user, "normally used" refers to a
-typical or common use of that class of product, regardless of the status
-of the particular user or of the way in which the particular user
-actually uses, or expects or is expected to use, the product.  A product
-is a consumer product regardless of whether the product has substantial
-commercial, industrial or non-consumer uses, unless such uses represent
-the only significant mode of use of the product.
-
-  "Installation Information" for a User Product means any methods,
-procedures, authorization keys, or other information required to install
-and execute modified versions of a covered work in that User Product from
-a modified version of its Corresponding Source.  The information must
-suffice to ensure that the continued functioning of the modified object
-code is in no case prevented or interfered with solely because
-modification has been made.
-
-  If you convey an object code work under this section in, or with, or
-specifically for use in, a User Product, and the conveying occurs as
-part of a transaction in which the right of possession and use of the
-User Product is transferred to the recipient in perpetuity or for a
-fixed term (regardless of how the transaction is characterized), the
-Corresponding Source conveyed under this section must be accompanied
-by the Installation Information.  But this requirement does not apply
-if neither you nor any third party retains the ability to install
-modified object code on the User Product (for example, the work has
-been installed in ROM).
-
-  The requirement to provide Installation Information does not include a
-requirement to continue to provide support service, warranty, or updates
-for a work that has been modified or installed by the recipient, or for
-the User Product in which it has been modified or installed.  Access to a
-network may be denied when the modification itself materially and
-adversely affects the operation of the network or violates the rules and
-protocols for communication across the network.
-
-  Corresponding Source conveyed, and Installation Information provided,
-in accord with this section must be in a format that is publicly
-documented (and with an implementation available to the public in
-source code form), and must require no special password or key for
-unpacking, reading or copying.
-
-  7. Additional Terms.
-
-  "Additional permissions" are terms that supplement the terms of this
-License by making exceptions from one or more of its conditions.
-Additional permissions that are applicable to the entire Program shall
-be treated as though they were included in this License, to the extent
-that they are valid under applicable law.  If additional permissions
-apply only to part of the Program, that part may be used separately
-under those permissions, but the entire Program remains governed by
-this License without regard to the additional permissions.
-
-  When you convey a copy of a covered work, you may at your option
-remove any additional permissions from that copy, or from any part of
-it.  (Additional permissions may be written to require their own
-removal in certain cases when you modify the work.)  You may place
-additional permissions on material, added by you to a covered work,
-for which you have or can give appropriate copyright permission.
-
-  Notwithstanding any other provision of this License, for material you
-add to a covered work, you may (if authorized by the copyright holders of
-that material) supplement the terms of this License with terms:
-
-    a) Disclaiming warranty or limiting liability differently from the
-    terms of sections 15 and 16 of this License; or
-
-    b) Requiring preservation of specified reasonable legal notices or
-    author attributions in that material or in the Appropriate Legal
-    Notices displayed by works containing it; or
-
-    c) Prohibiting misrepresentation of the origin of that material, or
-    requiring that modified versions of such material be marked in
-    reasonable ways as different from the original version; or
-
-    d) Limiting the use for publicity purposes of names of licensors or
-    authors of the material; or
-
-    e) Declining to grant rights under trademark law for use of some
-    trade names, trademarks, or service marks; or
-
-    f) Requiring indemnification of licensors and authors of that
-    material by anyone who conveys the material (or modified versions of
-    it) with contractual assumptions of liability to the recipient, for
-    any liability that these contractual assumptions directly impose on
-    those licensors and authors.
-
-  All other non-permissive additional terms are considered "further
-restrictions" within the meaning of section 10.  If the Program as you
-received it, or any part of it, contains a notice stating that it is
-governed by this License along with a term that is a further
-restriction, you may remove that term.  If a license document contains
-a further restriction but permits relicensing or conveying under this
-License, you may add to a covered work material governed by the terms
-of that license document, provided that the further restriction does
-not survive such relicensing or conveying.
-
-  If you add terms to a covered work in accord with this section, you
-must place, in the relevant source files, a statement of the
-additional terms that apply to those files, or a notice indicating
-where to find the applicable terms.
-
-  Additional terms, permissive or non-permissive, may be stated in the
-form of a separately written license, or stated as exceptions;
-the above requirements apply either way.
-
-  8. Termination.
-
-  You may not propagate or modify a covered work except as expressly
-provided under this License.  Any attempt otherwise to propagate or
-modify it is void, and will automatically terminate your rights under
-this License (including any patent licenses granted under the third
-paragraph of section 11).
-
-  However, if you cease all violation of this License, then your
-license from a particular copyright holder is reinstated (a)
-provisionally, unless and until the copyright holder explicitly and
-finally terminates your license, and (b) permanently, if the copyright
-holder fails to notify you of the violation by some reasonable means
-prior to 60 days after the cessation.
-
-  Moreover, your license from a particular copyright holder is
-reinstated permanently if the copyright holder notifies you of the
-violation by some reasonable means, this is the first time you have
-received notice of violation of this License (for any work) from that
-copyright holder, and you cure the violation prior to 30 days after
-your receipt of the notice.
-
-  Termination of your rights under this section does not terminate the
-licenses of parties who have received copies or rights from you under
-this License.  If your rights have been terminated and not permanently
-reinstated, you do not qualify to receive new licenses for the same
-material under section 10.
-
-  9. Acceptance Not Required for Having Copies.
-
-  You are not required to accept this License in order to receive or
-run a copy of the Program.  Ancillary propagation of a covered work
-occurring solely as a consequence of using peer-to-peer transmission
-to receive a copy likewise does not require acceptance.  However,
-nothing other than this License grants you permission to propagate or
-modify any covered work.  These actions infringe copyright if you do
-not accept this License.  Therefore, by modifying or propagating a
-covered work, you indicate your acceptance of this License to do so.
-
-  10. Automatic Licensing of Downstream Recipients.
-
-  Each time you convey a covered work, the recipient automatically
-receives a license from the original licensors, to run, modify and
-propagate that work, subject to this License.  You are not responsible
-for enforcing compliance by third parties with this License.
-
-  An "entity transaction" is a transaction transferring control of an
-organization, or substantially all assets of one, or subdividing an
-organization, or merging organizations.  If propagation of a covered
-work results from an entity transaction, each party to that
-transaction who receives a copy of the work also receives whatever
-licenses to the work the party's predecessor in interest had or could
-give under the previous paragraph, plus a right to possession of the
-Corresponding Source of the work from the predecessor in interest, if
-the predecessor has it or can get it with reasonable efforts.
-
-  You may not impose any further restrictions on the exercise of the
-rights granted or affirmed under this License.  For example, you may
-not impose a license fee, royalty, or other charge for exercise of
-rights granted under this License, and you may not initiate litigation
-(including a cross-claim or counterclaim in a lawsuit) alleging that
-any patent claim is infringed by making, using, selling, offering for
-sale, or importing the Program or any portion of it.
-
-  11. Patents.
-
-  A "contributor" is a copyright holder who authorizes use under this
-License of the Program or a work on which the Program is based.  The
-work thus licensed is called the contributor's "contributor version".
-
-  A contributor's "essential patent claims" are all patent claims
-owned or controlled by the contributor, whether already acquired or
-hereafter acquired, that would be infringed by some manner, permitted
-by this License, of making, using, or selling its contributor version,
-but do not include claims that would be infringed only as a
-consequence of further modification of the contributor version.  For
-purposes of this definition, "control" includes the right to grant
-patent sublicenses in a manner consistent with the requirements of
-this License.
-
-  Each contributor grants you a non-exclusive, worldwide, royalty-free
-patent license under the contributor's essential patent claims, to
-make, use, sell, offer for sale, import and otherwise run, modify and
-propagate the contents of its contributor version.
-
-  In the following three paragraphs, a "patent license" is any express
-agreement or commitment, however denominated, not to enforce a patent
-(such as an express permission to practice a patent or covenant not to
-sue for patent infringement).  To "grant" such a patent license to a
-party means to make such an agreement or commitment not to enforce a
-patent against the party.
-
-  If you convey a covered work, knowingly relying on a patent license,
-and the Corresponding Source of the work is not available for anyone
-to copy, free of charge and under the terms of this License, through a
-publicly available network server or other readily accessible means,
-then you must either (1) cause the Corresponding Source to be so
-available, or (2) arrange to deprive yourself of the benefit of the
-patent license for this particular work, or (3) arrange, in a manner
-consistent with the requirements of this License, to extend the patent
-license to downstream recipients.  "Knowingly relying" means you have
-actual knowledge that, but for the patent license, your conveying the
-covered work in a country, or your recipient's use of the covered work
-in a country, would infringe one or more identifiable patents in that
-country that you have reason to believe are valid.
-
-  If, pursuant to or in connection with a single transaction or
-arrangement, you convey, or propagate by procuring conveyance of, a
-covered work, and grant a patent license to some of the parties
-receiving the covered work authorizing them to use, propagate, modify
-or convey a specific copy of the covered work, then the patent license
-you grant is automatically extended to all recipients of the covered
-work and works based on it.
-
-  A patent license is "discriminatory" if it does not include within
-the scope of its coverage, prohibits the exercise of, or is
-conditioned on the non-exercise of one or more of the rights that are
-specifically granted under this License.  You may not convey a covered
-work if you are a party to an arrangement with a third party that is
-in the business of distributing software, under which you make payment
-to the third party based on the extent of your activity of conveying
-the work, and under which the third party grants, to any of the
-parties who would receive the covered work from you, a discriminatory
-patent license (a) in connection with copies of the covered work
-conveyed by you (or copies made from those copies), or (b) primarily
-for and in connection with specific products or compilations that
-contain the covered work, unless you entered into that arrangement,
-or that patent license was granted, prior to 28 March 2007.
-
-  Nothing in this License shall be construed as excluding or limiting
-any implied license or other defenses to infringement that may
-otherwise be available to you under applicable patent law.
-
-  12. No Surrender of Others' Freedom.
-
-  If conditions are imposed on you (whether by court order, agreement or
-otherwise) that contradict the conditions of this License, they do not
-excuse you from the conditions of this License.  If you cannot convey a
-covered work so as to satisfy simultaneously your obligations under this
-License and any other pertinent obligations, then as a consequence you may
-not convey it at all.  For example, if you agree to terms that obligate you
-to collect a royalty for further conveying from those to whom you convey
-the Program, the only way you could satisfy both those terms and this
-License would be to refrain entirely from conveying the Program.
-
-  13. Remote Network Interaction; Use with the GNU General Public License.
-
-  Notwithstanding any other provision of this License, if you modify the
-Program, your modified version must prominently offer all users
-interacting with it remotely through a computer network (if your version
-supports such interaction) an opportunity to receive the Corresponding
-Source of your version by providing access to the Corresponding Source
-from a network server at no charge, through some standard or customary
-means of facilitating copying of software.  This Corresponding Source
-shall include the Corresponding Source for any work covered by version 3
-of the GNU General Public License that is incorporated pursuant to the
-following paragraph.
-
-  Notwithstanding any other provision of this License, you have
-permission to link or combine any covered work with a work licensed
-under version 3 of the GNU General Public License into a single
-combined work, and to convey the resulting work.  The terms of this
-License will continue to apply to the part which is the covered work,
-but the work with which it is combined will remain governed by version
-3 of the GNU General Public License.
-
-  14. Revised Versions of this License.
-
-  The Free Software Foundation may publish revised and/or new versions of
-the GNU Affero General Public License from time to time.  Such new versions
-will be similar in spirit to the present version, but may differ in detail to
-address new problems or concerns.
-
-  Each version is given a distinguishing version number.  If the
-Program specifies that a certain numbered version of the GNU Affero General
-Public License "or any later version" applies to it, you have the
-option of following the terms and conditions either of that numbered
-version or of any later version published by the Free Software
-Foundation.  If the Program does not specify a version number of the
-GNU Affero General Public License, you may choose any version ever published
-by the Free Software Foundation.
-
-  If the Program specifies that a proxy can decide which future
-versions of the GNU Affero General Public License can be used, that proxy's
-public statement of acceptance of a version permanently authorizes you
-to choose that version for the Program.
-
-  Later license versions may give you additional or different
-permissions.  However, no additional obligations are imposed on any
-author or copyright holder as a result of your choosing to follow a
-later version.
-
-  15. Disclaimer of Warranty.
-
-  THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
-APPLICABLE LAW.  EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
-HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
-OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
-THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
-PURPOSE.  THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
-IS WITH YOU.  SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
-ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
-
-  16. Limitation of Liability.
-
-  IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
-WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
-THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
-GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
-USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
-DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
-PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
-EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
-SUCH DAMAGES.
-
-  17. Interpretation of Sections 15 and 16.
-
-  If the disclaimer of warranty and limitation of liability provided
-above cannot be given local legal effect according to their terms,
-reviewing courts shall apply local law that most closely approximates
-an absolute waiver of all civil liability in connection with the
-Program, unless a warranty or assumption of liability accompanies a
-copy of the Program in return for a fee.
-
-                     END OF TERMS AND CONDITIONS
-
-            How to Apply These Terms to Your New Programs
-
-  If you develop a new program, and you want it to be of the greatest
-possible use to the public, the best way to achieve this is to make it
-free software which everyone can redistribute and change under these terms.
-
-  To do so, attach the following notices to the program.  It is safest
-to attach them to the start of each source file to most effectively
-state the exclusion of warranty; and each file should have at least
-the "copyright" line and a pointer to where the full notice is found.
-
-    <one line to give the program's name and a brief idea of what it does.>
-    Copyright (C) <year>  <name of author>
-
-    This program is free software: you can redistribute it and/or modify
-    it under the terms of the GNU Affero General Public License as published
-    by the Free Software Foundation, either version 3 of the License, or
-    (at your option) any later version.
-
-    This program is distributed in the hope that it will be useful,
-    but WITHOUT ANY WARRANTY; without even the implied warranty of
-    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-    GNU Affero General Public License for more details.
-
-    You should have received a copy of the GNU Affero General Public License
-    along with this program.  If not, see <https://www.gnu.org/licenses/>.
-
-Also add information on how to contact you by electronic and paper mail.
-
-  If your software can interact with users remotely through a computer
-network, you should also make sure that it provides a way for users to
-get its source.  For example, if your program is a web application, its
-interface could display a "Source" link that leads users to an archive
-of the code.  There are many ways you could offer source, and different
-solutions will be better for different programs; see section 13 for the
-specific requirements.
-
-  You should also get your employer (if you work as a programmer) or school,
-if any, to sign a "copyright disclaimer" for the program, if necessary.
-For more information on this, and how to apply and follow the GNU AGPL, see
-<https://www.gnu.org/licenses/>.

definitions/Setup.hs

@@ -1,2 +0,0 @@
-import Distribution.Simple
-main = defaultMain

definitions/app/api-docs/Main.hs

@@ -1,16 +0,0 @@
-module Main where
-
-import Simplex.Messaging.ServerAPI
-
-import Servant
-import Servant.Docs
-
-apiDocs :: API
-apiDocs = docsWith
-            defaultDocOptions
-            [serverApiIntro]
-            serverApiExtra
-            (Proxy :: Proxy ServerAPI)
-
-main :: IO ()
-main = writeFile "../simplex-messaging-api.md" $ markdown apiDocs

definitions/app/print-scenarios/Main.hs

@@ -1,7 +0,0 @@
-module Main where
-
-import Simplex.Messaging.PrintScenario
-import Simplex.Messaging.Scenarios
-
-main :: IO ()
-main = printScenario establishConnection

definitions/package.yaml

@@ -1,62 +0,0 @@
-name: simplex-definitions
-version: 0.1.0.0
-#synopsis:
-#description:
-homepage: https://github.com/simplex-chat/protocol/blob/master/definitions/readme.md
-license: AGPL-3
-author: Evgeny Poberezkin
-copyright: 2020 Evgeny Poberezkin
-category: Web
-extra-source-files:
-  - readme.md
-
-ghc-options:
-  # - -fplugin=Polysemy.Plugin
-  - -O2
-  - -Wall
-  - -Wcompat
-  - -Werror=incomplete-patterns
-  - -Wredundant-constraints
-  - -Wincomplete-record-updates
-  - -Wincomplete-uni-patterns
-  - -Wunused-type-patterns
-
-dependencies:
-  - aeson
-  - base >= 4.7 && < 5
-  - freer-indexed
-  - polysemy
-  # - polysemy-plugin
-  - lens
-  - mtl
-  - protocol
-  - singletons
-  - servant-docs
-  - servant-server
-  - text
-  - transformers
-
-library:
-  source-dirs: src
-
-executables:
-  api-docs:
-    source-dirs: app/api-docs
-    main: Main.hs
-    ghc-options: -threaded
-    dependencies: simplex-definitions
-
-  print-scenarios:
-    source-dirs: app/print-scenarios
-    main: Main.hs
-    ghc-options: -threaded
-    dependencies: simplex-definitions
-
-tests:
-  simplex-definitions-doctests:
-    source-dirs: tests
-    main: doctest-driver.hs
-    ghc-options: -threaded
-    dependencies:
-      - doctest
-      - doctest-driver-gen

definitions/readme.md

@@ -1,3 +0,0 @@
-# simplex-messaging-api
-
-This package contains [Servant](https://hackage.haskell.org/package/servant-server) API types and is used to generate [API documenation](../simplex-messaging-api.md)

definitions/src/Simplex/Messaging/Broker.hs

@@ -1,53 +0,0 @@
-{-# LANGUAGE DataKinds #-}
-{-# LANGUAGE FlexibleContexts #-}
-{-# LANGUAGE FlexibleInstances #-}
-{-# LANGUAGE GADTs #-}
-{-# LANGUAGE InstanceSigs #-}
-{-# LANGUAGE MultiParamTypeClasses #-}
-{-# LANGUAGE PolyKinds #-}
-{-# OPTIONS_GHC -fno-warn-orphans #-}
-{-# OPTIONS_GHC -fno-warn-unticked-promoted-constructors #-}
-
-module Simplex.Messaging.Broker where
-
-import Control.Monad.Trans.Except
-import Polysemy.Internal
-import Simplex.Messaging.Protocol
-
-instance Monad m => PartyProtocol m Broker where
-  api ::
-    SimplexCommand from '(Broker, s, s') a ->
-    Connection Broker s ->
-    ExceptT String m (a, Connection Broker s')
-  api (CreateConn _) = apiStub
-  api (Subscribe _) = apiStub
-  api (Unsubscribe _) = apiStub
-  api (ConfirmConn _ _) = apiStub
-  api (SecureConn _ _) = apiStub
-  api (SendMsg _ _) = apiStub
-  api (DeleteMsg _ _) = apiStub
-
-  action ::
-    SimplexCommand '(Broker, s, s') to a ->
-    Connection Broker s ->
-    ExceptT String m a ->
-    ExceptT String m (Connection Broker s')
-  action (PushConfirm _ _) = actionStub
-  action (PushMsg _ _) = actionStub
-
-type SimplexBroker = SimplexParty Broker
-
-api' ::
-  Member SimplexBroker r =>
-  SimplexCommand from '(Broker, s, s') a ->
-  Connection Broker s ->
-  Sem r (Either String (a, Connection Broker s'))
-api' cmd conn = send $ Api cmd conn
-
-action' ::
-  Member SimplexBroker r =>
-  SimplexCommand '(Broker, s, s') to a ->
-  Connection Broker s ->
-  Either String a ->
-  Sem r (Either String (Connection Broker s'))
-action' cmd conn res = send $ Action cmd conn res

definitions/src/Simplex/Messaging/Client.hs

@@ -1,84 +0,0 @@
-{-# LANGUAGE DataKinds #-}
-{-# LANGUAGE FlexibleContexts #-}
-{-# LANGUAGE FlexibleInstances #-}
-{-# LANGUAGE GADTs #-}
-{-# LANGUAGE InstanceSigs #-}
-{-# LANGUAGE MultiParamTypeClasses #-}
-{-# LANGUAGE PolyKinds #-}
-{-# OPTIONS_GHC -fno-warn-orphans #-}
-{-# OPTIONS_GHC -fno-warn-unticked-promoted-constructors #-}
-
-module Simplex.Messaging.Client where
-
-import Control.Monad.Trans.Except
-import Polysemy.Internal
-import Simplex.Messaging.Protocol
-
-instance Monad m => PartyProtocol m Recipient where
-  api ::
-    SimplexCommand from '(Recipient, s, s') a ->
-    Connection Recipient s ->
-    ExceptT String m (a, Connection Recipient s')
-  api (PushConfirm _ _) = apiStub
-  api (PushMsg _ _) = apiStub
-
-  action ::
-    SimplexCommand '(Recipient, s, s') to a ->
-    Connection Recipient s ->
-    ExceptT String m a ->
-    ExceptT String m (Connection Recipient s')
-  action (CreateConn _) = actionStub
-  action (Subscribe _) = actionStub
-  action (Unsubscribe _) = actionStub
-  action (SendInvite _) = actionStub
-  action (SecureConn _ _) = actionStub
-  action (DeleteMsg _ _) = actionStub
-
-instance Monad m => PartyProtocol m Sender where
-  api ::
-    SimplexCommand from '(Sender, s, s') a ->
-    Connection Sender s ->
-    ExceptT String m (a, Connection Sender s')
-  api (SendInvite _) = apiStub
-
-  action ::
-    SimplexCommand '(Sender, s, s') to a ->
-    Connection Sender s ->
-    ExceptT String m a ->
-    ExceptT String m (Connection Sender s')
-  action (ConfirmConn _ _) = actionStub
-  action (SendMsg _ _) = actionStub
-
-type SimplexRecipient = SimplexParty Recipient
-
-type SimplexSender = SimplexParty Sender
-
-rApi ::
-  Member SimplexRecipient r =>
-  SimplexCommand from '(Recipient, s, s') a ->
-  Connection Recipient s ->
-  Sem r (Either String (a, Connection Recipient s'))
-rApi cmd conn = send $ Api cmd conn
-
-rAction ::
-  Member SimplexRecipient r =>
-  SimplexCommand '(Recipient, s, s') to a ->
-  Connection Recipient s ->
-  Either String a ->
-  Sem r (Either String (Connection Recipient s'))
-rAction cmd conn res = send $ Action cmd conn res
-
-sApi ::
-  Member SimplexSender r =>
-  SimplexCommand from '(Sender, s, s') a ->
-  Connection Sender s ->
-  Sem r (Either String (a, Connection Sender s'))
-sApi cmd conn = send $ Api cmd conn
-
-sAction ::
-  Member SimplexSender r =>
-  SimplexCommand '(Sender, s, s') to a ->
-  Connection Sender s ->
-  Either String a ->
-  Sem r (Either String (Connection Sender s'))
-sAction cmd conn res = send $ Action cmd conn res

definitions/src/Simplex/Messaging/Connection.hs

@@ -1,70 +0,0 @@
-{-# LANGUAGE DataKinds #-}
-{-# LANGUAGE DuplicateRecordFields #-}
-{-# LANGUAGE GADTs #-}
-{-# LANGUAGE TypeFamilies #-}
-{-# LANGUAGE TypeOperators #-}
-{-# OPTIONS_GHC -fno-warn-unticked-promoted-constructors #-}
-
-module Simplex.Messaging.Connection where
-
-import Data.Kind
-import Data.Text
-import Data.Type.Bool (type (||))
-import Data.Type.Equality (type (==))
-import Simplex.Messaging.Core
-import Simplex.Messaging.Types hiding (Invitation)
-
--- | 'Connection' for all participants
-data Connection (p :: Party) (s :: ConnState) :: Type where
-  -- | no connection with this ID, used by all parties
-  NoConnection ::
-    ConnId ->
-    Connection p None
-  -- | connection created by the broker
-  ConnRcpNew ::
-    (s == New || s == Pending) ~ True =>
-    ClientConn ->
-    SenderConnId ->
-    PrivateKey -> -- key to decrypt messages from sender
-    PublicKey -> -- key for sender to encrypt messages to recipient
-    Connection Recipient s
-  -- | After sender confirmed connection:
-  --     * added sender's key for the broker (inside Conn)
-  --     * removed PublicKey previously sent to sender
-  ConnRcpConfirmed ::
-    ClientConn ->
-    Conn -> -- sender's connection info
-    PrivateKey -> -- key to decrypt messages from the sender
-    Connection Recipient Confirmed
-  -- | Connection is secured and can be used by the sender or it is disabled.
-  -- All sender connection information is removed now.
-  ConnRcp ::
-    (s == Secured || s == Disabled) ~ True =>
-    ClientConn ->
-    PrivateKey -> -- to decrypt messages from sender
-    Connection Recipient s
-  ConnSnd ::
-    (HasState Sender s, (s == None) ~ False) =>
-    ClientConn ->
-    PublicKey -> -- to encrypt messages to recipient
-    Connection Sender s
-  ConnBrkNew ::
-    Conn ->
-    SenderConnId ->
-    Connection Broker New
-  ConnBrk ::
-    (s == Secured || s == Disabled) ~ True =>
-    {recipient :: Conn, sender :: Conn} ->
-    Connection Broker s
-
-data Conn = Conn
-  { connId :: ConnId,
-    brokerVerifyKey :: PublicKey
-  }
-
-data ClientConn = ClientConn
-  { connId :: ConnId,
-    brokerVerifyKey :: PublicKey,
-    brokerKey :: PrivateKey,
-    brokerUri :: Text
-  }

definitions/src/Simplex/Messaging/Core.hs

@@ -1,49 +0,0 @@
-{-# LANGUAGE DataKinds #-}
-{-# LANGUAGE EmptyCase #-}
-{-# LANGUAGE GADTs #-}
-{-# LANGUAGE InstanceSigs #-}
-{-# LANGUAGE PolyKinds #-}
-{-# LANGUAGE ScopedTypeVariables #-}
-{-# LANGUAGE TemplateHaskell #-}
-{-# LANGUAGE TypeApplications #-}
-{-# LANGUAGE TypeFamilies #-}
-{-# LANGUAGE UndecidableInstances #-}
-{-# OPTIONS_GHC -fno-warn-unticked-promoted-constructors #-}
-
-module Simplex.Messaging.Core where
-
-import Data.Kind
-import Data.Singletons.TH
-
-$( singletons
-     [d|
-       data Party = Recipient | Broker | Sender
-         deriving (Show, Eq)
-
-       data ConnState
-         = None -- (all) not available or removed from the broker
-         | New -- (all) connection created (or, for sender, received from recipient)
-         | Pending -- (recipient) sent to sender out-of-band
-         | Confirmed -- (recipient, sender) confirmed by sender with the broker
-         | Secured -- (all) secured with the broker
-         | Disabled -- (broker, recipient) disabled with the broker by recipient
-         deriving (Show, Eq)
-       |]
- )
-
-type family HasState (p :: Party) (s :: ConnState) :: Constraint where
-  HasState Recipient _ = ()
-  HasState Broker None = ()
-  HasState Broker New = ()
-  HasState Broker Secured = ()
-  HasState Broker Disabled = ()
-  HasState Sender None = ()
-  HasState Sender New = ()
-  HasState Sender Confirmed = ()
-  HasState Sender Secured = ()
-
-type family Enabled (rs :: ConnState) (bs :: ConnState) :: Constraint where
-  Enabled New New = ()
-  Enabled Pending New = ()
-  Enabled Confirmed New = ()
-  Enabled Secured Secured = ()

definitions/src/Simplex/Messaging/PrintScenario.hs

@@ -1,70 +0,0 @@
-{-# LANGUAGE AllowAmbiguousTypes #-}
-{-# LANGUAGE DataKinds #-}
-{-# LANGUAGE FlexibleContexts #-}
-{-# LANGUAGE FlexibleInstances #-}
-{-# LANGUAGE GADTs #-}
-{-# LANGUAGE KindSignatures #-}
-{-# LANGUAGE LambdaCase #-}
-{-# OPTIONS_GHC -fno-warn-unticked-promoted-constructors #-}
-
-module Simplex.Messaging.PrintScenario where
-
-import Control.Monad.Writer
-import Control.Protocol (runProtocol)
-import Data.Singletons
-import Simplex.Messaging.Protocol
-import Simplex.Messaging.Types
-
-printScenario :: SimplexProtocol s s' a -> IO ()
-printScenario scn = ps 1 "" $ execWriter $ logScenario scn
-  where
-    ps :: Int -> String -> [(String, String)] -> IO ()
-    ps _ _ [] = return ()
-    ps i p ((p', l) : ls)
-      | p' /= p = part (i + 1) $ show i <> ". " <> p' <> ":\n" <> prefix l
-      | otherwise = part i $ prefix l
-      where
-        part i' s = putStrLn s >> ps i' p' ls
-        prefix s = "   - " <> s
-
-logScenario :: MonadWriter [(String, String)] m => SimplexProtocol s s' a -> m a
-logScenario = runProtocol $ \from to cmd -> do
-  tell [(party from, commandStr cmd <> " " <> party to)]
-  mockCommand cmd
-
-commandStr :: SimplexCommand from to a -> String
-commandStr = \case
-  CreateConn _ -> "creates connection in"
-  Subscribe cid -> "subscribes to connection " <> show cid <> " in"
-  Unsubscribe cid -> "unsubscribes from connection " <> show cid <> " in"
-  SendInvite _ -> "sends out-of band invitation to "
-  ConfirmConn cid _ -> "confirms connection " <> show cid <> " in"
-  PushConfirm cid _ -> "pushes confirmation for " <> show cid <> " to"
-  SecureConn cid _ -> "secures connection " <> show cid <> " in"
-  SendMsg cid _ -> "sends message to connection " <> show cid <> " in"
-  PushMsg cid _ -> "pushes message from connection " <> show cid <> " to"
-  DeleteMsg cid _ -> "deletes message from connection " <> show cid <> " in"
-
-mockCommand :: Monad m => SimplexCommand from to a -> m a
-mockCommand = \case
-  (CreateConn _) ->
-    return
-      CreateConnResponse
-        { recipientId = "Qxz93A",
-          senderId = "N9pA3g"
-        }
-  Subscribe _ -> return ()
-  Unsubscribe _ -> return ()
-  SendInvite _ -> return ()
-  ConfirmConn _ _ -> return ()
-  PushConfirm _ _ -> return ()
-  SecureConn _ _ -> return ()
-  SendMsg _ _ -> return ()
-  PushMsg _ _ -> return ()
-  DeleteMsg _ _ -> return ()
-
-party :: Sing (p :: Party) -> String
-party = \case
-  SRecipient -> "Alice (recipient)"
-  SBroker -> "Alice's server (broker)"
-  SSender -> "Bob (sender)"

definitions/src/Simplex/Messaging/Protocol.hs

@@ -1,133 +0,0 @@
-{-# LANGUAGE ConstraintKinds #-}
-{-# LANGUAGE DataKinds #-}
-{-# LANGUAGE GADTs #-}
-{-# LANGUAGE MultiParamTypeClasses #-}
-{-# LANGUAGE OverloadedStrings #-}
-{-# LANGUAGE PolyKinds #-}
-{-# LANGUAGE TypeFamilies #-}
-{-# LANGUAGE TypeOperators #-}
-{-# LANGUAGE UndecidableInstances #-}
-{-# OPTIONS_GHC -fno-warn-unticked-promoted-constructors #-}
-
-module Simplex.Messaging.Protocol
-  ( module Simplex.Messaging.Core,
-    module Simplex.Messaging.Protocol,
-  )
-where
-
-import Control.Monad.Trans.Except
-import Control.Protocol
-import Data.Kind
-import Data.Type.Bool (type (||))
-import Data.Type.Equality (type (==))
-import Simplex.Messaging.Core
-import Simplex.Messaging.Types
-
-type SimplexProtocol = Protocol SimplexCommand '[Recipient, Broker, Sender]
-
-data SimplexCommand :: Command Party ConnState where
-  CreateConn ::
-    PublicKey ->
-    SimplexCommand
-      '(Recipient, None, New)
-      '(Broker, None, New)
-      CreateConnResponse
-  Subscribe ::
-    Enabled rs bs =>
-    ConnId ->
-    SimplexCommand
-      '(Recipient, rs, rs)
-      '(Broker, bs, bs)
-      ()
-  Unsubscribe ::
-    Enabled rs bs =>
-    ConnId ->
-    SimplexCommand
-      '(Recipient, rs, rs)
-      '(Broker, bs, bs)
-      ()
-  SendInvite ::
-    Invitation ->
-    SimplexCommand
-      '(Recipient, New, Pending)
-      '(Sender, None, New)
-      ()
-  ConfirmConn ::
-    SenderConnId ->
-    Encrypted ->
-    SimplexCommand
-      '(Sender, New, Confirmed)
-      '(Broker, New, New)
-      ()
-  PushConfirm ::
-    ConnId ->
-    Message ->
-    SimplexCommand
-      '(Broker, New, New)
-      '(Recipient, Pending, Confirmed)
-      ()
-  SecureConn ::
-    ConnId ->
-    PublicKey ->
-    SimplexCommand
-      '(Recipient, Confirmed, Secured)
-      '(Broker, New, Secured)
-      ()
-  SendMsg ::
-    (ss == Confirmed || ss == Secured) ~ True =>
-    SenderConnId ->
-    Encrypted ->
-    SimplexCommand
-      '(Sender, ss, Secured)
-      '(Broker, Secured, Secured)
-      ()
-  PushMsg ::
-    ConnId ->
-    Message ->
-    SimplexCommand
-      '(Broker, Secured, Secured)
-      '(Recipient, Secured, Secured)
-      ()
-  DeleteMsg ::
-    ConnId ->
-    MessageId ->
-    SimplexCommand
-      '(Recipient, Secured, Secured)
-      '(Broker, Secured, Secured)
-      ()
-
--- connection type stub for all participants, TODO move from idris
-data
-  Connection
-    (p :: Party)
-    (s :: ConnState) :: Type
-  where
-  Connection :: String -> Connection p s -- TODO replace with real type definition
-
-class Monad m => PartyProtocol m (p :: Party) where
-  api ::
-    SimplexCommand from '(p, s, s') a ->
-    Connection p s ->
-    ExceptT String m (a, Connection p s')
-  action ::
-    SimplexCommand '(p, s, s') to a ->
-    Connection p s ->
-    ExceptT String m a ->
-    ExceptT String m (Connection p s')
-
-apiStub :: Monad m => Connection p s -> ExceptT String m (a, Connection p s')
-apiStub _ = throwE "api not implemented"
-
-actionStub :: Monad m => Connection p s -> ExceptT String m a -> ExceptT String m (Connection p s')
-actionStub _ _ = throwE "action not implemented"
-
-data SimplexParty (p :: Party) m a where
-  Api ::
-    SimplexCommand from '(p, s, s') x ->
-    Connection p s ->
-    SimplexParty p m (Either String (x, Connection p s'))
-  Action ::
-    SimplexCommand '(p, s, s') to x ->
-    Connection p s ->
-    Either String x ->
-    SimplexParty p m (Either String (Connection p s'))

definitions/src/Simplex/Messaging/Scenarios.hs

@@ -1,45 +0,0 @@
-{-# LANGUAGE DataKinds #-}
-{-# LANGUAGE DuplicateRecordFields #-}
-{-# LANGUAGE OverloadedStrings #-}
-{-# LANGUAGE RebindableSyntax #-}
-{-# OPTIONS_GHC -fno-warn-missing-fields #-}
-{-# OPTIONS_GHC -fno-warn-unticked-promoted-constructors #-}
-{-# OPTIONS_GHC -fno-warn-unused-do-bind #-}
-
-module Simplex.Messaging.Scenarios where
-
-import Control.Protocol
-import Control.XMonad.Do
-import Data.Singletons
-import Data.String
-import Simplex.Messaging.Protocol
-import Simplex.Messaging.Types
-import Prelude hiding ((>>), (>>=))
-
-r :: Sing Recipient
-r = SRecipient
-
-b :: Sing Broker
-b = SBroker
-
-s :: Sing Sender
-s = SSender
-
--- Establish simplex messaging connection and send first message
-establishConnection :: SimplexProtocol '[None, None, None] '[Secured, Secured, Secured] ()
-establishConnection = do
-  r ->: b $ CreateConn "BODbZxmtKUUF1l8pj4nVjQ"
-  r ->: b $ Subscribe "RU"
-  r ->: s $ SendInvite Invitation {connId = "SU"}
-  s ->: b $ ConfirmConn "SU" "encrypted"
-  b ->: r $ PushConfirm "RU" Message {msgId = "abc", msg = "XPaVEVNunkYKqqK0dnAT5Q"}
-  r ->: b $ SecureConn "RU" "XPaVEVNunkYKqqK0dnAT5Q"
-  r ->: b $ DeleteMsg "RU" "abc"
-  s ->: b $ SendMsg "SU" "welcome" -- welcome message
-  b ->: r $ PushMsg "RU" Message {msgId = "def", msg = "welcome"}
-  r ->: b $ DeleteMsg "RU" "def"
-  -- The connection is established ("Secured"), sending the message
-  s ->: b $ SendMsg "SU" "hello there"
-  b ->: r $ PushMsg "RU" Message {msgId = "ghi", msg = "hello there"}
-  r ->: b $ DeleteMsg "RU" "ghi"
-  r ->: b $ Unsubscribe "RU"

definitions/src/Simplex/Messaging/ServerAPI.hs

@@ -1,155 +0,0 @@
-{-# LANGUAGE DataKinds #-}
-{-# LANGUAGE DuplicateRecordFields #-}
-{-# LANGUAGE FlexibleInstances #-}
-{-# LANGUAGE OverloadedStrings #-}
-{-# LANGUAGE TypeOperators #-}
-{-# OPTIONS_GHC -fno-warn-orphans #-}
-
-module Simplex.Messaging.ServerAPI
-  ( ServerAPI,
-    serverApiIntro,
-    serverApiExtra,
-  )
-where
-
-import Control.Lens
-import Data.Function ()
-import Servant
-import Servant.Docs
-import Simplex.Messaging.Types
-
-type ServerAPI =
-  CreateConnection
-    :<|> SecureConnection
-    :<|> DeleteConnection
-    :<|> GetMessages
-    :<|> DeleteMessage
-    :<|> SendMessage
-
-type CreateConnection =
-  "connection" :> ReqBody '[JSON] CreateConnRequest
-    :> PostCreated '[JSON] CreateConnResponse
-
-type SecureConnection =
-  "connection" :> Capture "connectionId" Base64EncodedString
-    :> ReqBody '[JSON] SecureConnRequest
-    :> Put '[JSON] NoContent
-
-type DeleteConnection =
-  "connection" :> Capture "connectionId" Base64EncodedString
-    :> Delete '[JSON] NoContent
-
-type GetMessages =
-  "connection" :> Capture "connectionId" Base64EncodedString
-    :> "messages"
-    :> QueryParam "fromMessageId" (Maybe Base64EncodedString)
-    :> Get '[JSON] MessagesResponse
-
-type DeleteMessage =
-  "connection" :> Capture "connectionId" Base64EncodedString
-    :> "messages"
-    :> Capture "messageId" Base64EncodedString
-    :> Delete '[JSON] NoContent
-
-type SendMessage =
-  "connection" :> Capture "senderConnectionId" Base64EncodedString
-    :> "messages"
-    :> ReqBody '[JSON] SendMessageRequest
-    :> PostCreated '[JSON] NoContent
-
--- API docs
-serverApiIntro :: DocIntro
-serverApiIntro =
-  DocIntro
-    "Simplex messaging protocol REST API"
-    [ "This document lists all required REST endpoints of simplex messaging API.",
-      "Also see [Simplex messaging protocol implementation](simplex-messaging-implementation.md) for more details."
-    ]
-
-serverApiExtra :: ExtraInfo ServerAPI
-serverApiExtra =
-  info
-    (Proxy :: Proxy CreateConnection)
-    "Create connection"
-    []
-    <> info
-      (Proxy :: Proxy SecureConnection)
-      "Secure connection"
-      []
-    <> info
-      (Proxy :: Proxy DeleteConnection)
-      "Delete connection"
-      []
-    <> info
-      (Proxy :: Proxy GetMessages)
-      "Get messages"
-      []
-    <> info
-      (Proxy :: Proxy DeleteMessage)
-      "Delete message"
-      []
-    <> info
-      (Proxy :: Proxy SendMessage)
-      "Send message"
-      []
-  where
-    info p title comments =
-      extraInfo p $ defAction & notes <>~ [DocNote title comments]
-
-instance ToCapture (Capture "connectionId" String) where
-  toCapture _ =
-    DocCapture
-      "connectionId"
-      "Recipient connection ID - unique connection ID to be used by connection recipient"
-
-instance ToCapture (Capture "senderConnectionId" String) where
-  toCapture _ =
-    DocCapture
-      "senderConnectionId"
-      "Sender connection ID - unique connection ID to be used by connection sender"
-
-instance ToCapture (Capture "messageId" String) where
-  toCapture _ =
-    DocCapture
-      "messageId"
-      "Message ID - unique message ID to be used by connection recipient"
-
-instance ToParam (QueryParam "fromMessageId" (Maybe Base64EncodedString)) where
-  toParam _ =
-    DocQueryParam
-      "fromMessageId"
-      ["message ID, e.g., `p8PCiGPZ`"]
-      "if set, the server will respond with the messages received starting from the message with server message ID (unique per server) passed in this parameter."
-      Normal
-
-instance ToSample CreateConnRequest where
-  toSamples _ = singleSample $ CreateConnRequest "BODbZxmtKUUF1l8pj4nVjQ"
-
-instance ToSample CreateConnResponse where
-  toSamples _ = singleSample $ CreateConnResponse "Qxz93A" "N9pA3g"
-
-instance ToSample SecureConnRequest where
-  toSamples _ = singleSample $ SecureConnRequest "XPaVEVNunkYKqqK0dnAT5Q"
-
-dummyMessage :: Message
-dummyMessage =
-  Message
-    { msgId = "p8PCiGPZ",
-      ts = "2020-03-15T19:58:33.695Z",
-      msg = "OQLMXoEA4iv-aR46puPJuY1Rdoc1KY0gfq8oElJwtAs"
-    }
-
-instance ToSample MessagesResponse where
-  toSamples _ =
-    singleSample $
-      MessagesResponse
-        { messages = [dummyMessage],
-          nextMessageId = Nothing
-        }
-
-instance ToSample SendMessageRequest where
-  toSamples _ =
-    singleSample $
-      SendMessageRequest
-        { msg = "OQLMXoEA4iv-aR46puPJuY1Rdoc1KY0gfq8oElJwtAs"
-        }

definitions/src/Simplex/Messaging/Types.hs

@@ -1,77 +0,0 @@
-{-# LANGUAGE DeriveAnyClass #-}
-{-# LANGUAGE DeriveGeneric #-}
-{-# LANGUAGE DuplicateRecordFields #-}
-
-module Simplex.Messaging.Types where
-
-import Data.Aeson
-import Data.String
-import Data.Text
-import GHC.Generics
-
-newtype CreateConnRequest = CreateConnRequest
-  { recipientKey :: Key
-  }
-  deriving (Eq, Show, Generic, ToJSON, FromJSON)
-
-instance IsString CreateConnRequest where
-  fromString = CreateConnRequest
-
-data CreateConnResponse = CreateConnResponse
-  { recipientId :: String,
-    senderId :: String
-  }
-  deriving (Show, Generic, ToJSON, FromJSON)
-
-newtype SecureConnRequest = SecureConnRequest
-  { senderKey :: Key
-  }
-  deriving (Show, Generic, ToJSON, FromJSON)
-
-instance IsString SecureConnRequest where
-  fromString = SecureConnRequest
-
-data Message = Message
-  { msgId :: MessageId,
-    ts :: TimeStamp,
-    msg :: Encrypted -- TODO make it Text
-  }
-  deriving (Show, Generic, ToJSON, FromJSON)
-
-data MessagesResponse = MessagesResponse
-  { messages :: [Message],
-    nextMessageId :: Maybe Base64EncodedString
-  }
-  deriving (Show, Generic, ToJSON, FromJSON)
-
-newtype SendMessageRequest = SendMessageRequest
-  { msg :: Base64EncodedString
-  }
-  deriving (Show, Generic, ToJSON, FromJSON)
-
-instance IsString SendMessageRequest where
-  fromString = SendMessageRequest
-
-data Invitation = Invitation
-  { connId :: ConnId,
-    brokerUri :: Text,
-    encryptKey :: PublicKey
-  }
-
-type Key = Base64EncodedString -- deprecated, not to be used
-
-type PublicKey = Base64EncodedString
-
-type PrivateKey = Base64EncodedString
-
-type ConnId = Base64EncodedString
-
-type SenderConnId = Base64EncodedString
-
-type MessageId = Base64EncodedString
-
-type Encrypted = Base64EncodedString
-
-type Base64EncodedString = String
-
-type TimeStamp = String

definitions/stack.yaml

@@ -1,67 +0,0 @@
-# This file was automatically generated by 'stack init'
-#
-# Some commonly used options have been documented as comments in this file.
-# For advanced use and comprehensive documentation of the format, please see:
-# https://docs.haskellstack.org/en/stable/yaml_configuration/
-
-# Resolver to choose a 'specific' stackage snapshot or a compiler version.
-# A snapshot resolver dictates the compiler version and the set of packages
-# to be used for project dependencies. For example:
-#
-# resolver: lts-3.5
-# resolver: nightly-2015-09-21
-# resolver: ghc-7.10.2
-#
-# The location of a snapshot can be provided as a file or url. Stack assumes
-# a snapshot provided as a file might change, whereas a url resource does not.
-#
-# resolver: ./custom-snapshot.yaml
-# resolver: https://example.com/snapshots/2018-01-01.yaml
-resolver: lts-15.11
-
-# User packages to be built.
-# Various formats can be used as shown in the example below.
-#
-# packages:
-# - some-directory
-# - https://example.com/foo/bar/baz-0.0.2.tar.gz
-#   subdirs:
-#   - auto-update
-#   - wai
-packages:
-  - .
-# Dependency packages to be pulled from upstream that are not in the resolver.
-# These entries can reference officially published versions as well as
-# forks / in-progress versions pinned to a git hash. For example:
-#
-# extra-deps:
-# - acme-missiles-0.3
-# - git: https://github.com/commercialhaskell/stack.git
-#   commit: e7b331f14bcffb8367cd58fbfc8b40ec7642100a
-#
-extra-deps:
-  - freer-indexed-0.1.0.0@sha256:b247be91b8ad2154fe1a514dec7c6a2553281d89325f0bc213d1d832d4c1a0e9,3007
-  - protocol-0.1.0.1@sha256:1e95952ba8fc17bbd6c1e4cf1e2993590a90ab938c30c6e530ad0f3ba4ec1a8c,1598
-# Override default flag values for local packages and extra-deps
-# flags: {}
-
-# Extra package databases containing global packages
-# extra-package-dbs: []
-
-# Control whether we use the GHC we find on the path
-# system-ghc: true
-#
-# Require a specific version of stack, using version ranges
-# require-stack-version: -any # Default
-# require-stack-version: ">=2.1"
-#
-# Override the architecture used by stack, especially useful on Windows
-# arch: i386
-# arch: x86_64
-#
-# Extra directories used by stack for building
-# extra-include-dirs: [/path/to/dir]
-# extra-lib-dirs: [/path/to/dir]
-#
-# Allow a newer minor version of GHC than the snapshot specifies
-# compiler-check: newer-minor

definitions/tests/doctest-driver.hs

@@ -1 +0,0 @@
-{-# OPTIONS_GHC -F -pgmF doctest-driver-gen -optF src -optF -XBlockArguments -optF -XDuplicateRecordFields -optF -XLambdaCase -optF -XNamedFieldPuns -optF -XOverloadedStrings -optF -XRecordWildCards -optF -XAllowAmbiguousTypes -optF -XConstraintKinds -optF -XDeriveAnyClass -optF -XEmptyCase -optF -XFlexibleContexts -optF -XFlexibleInstances -optF -XGADTs -optF -XInstanceSigs -optF -XMultiParamTypeClasses -optF -XNoStarIsType -optF -XScopedTypeVariables -optF -XStandaloneDeriving -optF -XTemplateHaskell -optF -XTypeApplications -optF -XTypeFamilies -optF -XTypeInType -optF -XTypeOperators -optF -XUndecidableInstances #-}

diagrams/duplex-messaging/duplex-creating.mmd

@@ -1,71 +0,0 @@
-sequenceDiagram
-  participant A as Alice
-  participant AA as Alice's<br>agent
-  participant AS as Alice's<br>server
-  participant BS as Bob's<br>server
-  participant BA as Bob's<br>agent
-  participant B as Bob
-
-  note over AA, BA: status (receive/send): NONE/NONE
-
-  note over A, AA: 1. request connection from agent
-  A ->> AA: NEW: create<br>duplex connection
- 
-  note over AA, AS: 2. create Alice's SMP queue
-  AA ->> AS: NEW: create SMP queue
-  AS ->> AA: IDS: SMP queue IDs
-  note over AA: status: NEW/NONE
-
-  AA ->> A: INV: invitation<br>to connect
-  note over AA: status: PENDING/NONE
-
-  note over A, B: 3. out-of-band invitation
-  A ->> B: OOB: invitation to connect
-
-  note over BA, B: 4. accept connection
-  B ->> BA: JOIN:<br>via invitation info
-  note over BA: status: NONE/NEW
-
-  note over BA, AA: 5. establish Alice's SMP queue
-  BA ->> AS: SEND: KEY: sender's server key
-  note over BA: status: NONE/CONFIRMED
-  activate BA
-  AS ->> AA: MSG: KEY: sender's server key
-  note over AA: status: CONFIRMED/NONE
-  AA ->> AS: KEY: secure queue
-  note over AA: status: SECURED/NONE
-
-  BA ->> AS: SEND: HELLO: try sending until successful
-  deactivate BA
-  note over BA: status: NONE/ACTIVE
-  AS ->> AA: MSG: HELLO: Alice's agent<br>knows Bob can send
-  note over AA: status: ACTIVE/NONE
-
-  note over BA, BS: 6. create Bob's SMP queue
-  BA ->> BS: NEW: create SMP queue
-  BS ->> BA: IDS: SMP queue IDs
-  note over BA: status: NEW/ACTIVE
-
-  note over AA, BA: 7. establish Bob's SMP queue
-  BA ->> AS: SEND: REPLY: invitation to the connect
-  note over BA: status: PENDING/ACTIVE
-  AS ->> AA: MSG: REPLY: invitation<br>to connect
-  note over AA: status: ACTIVE/NEW
-
-  AA ->> BS: SEND: KEY: sender's server key
-  note over AA: status: ACTIVE/CONFIRMED
-  activate AA
-  BS ->> BA: MSG: KEY: sender's server key
-  note over BA: status: CONFIRMED/ACTIVE
-  BA ->> BS: KEY: secure queue
-  note over BA: status: SECURED/ACTIVE
-
-  AA ->> BS: SEND: HELLO: try sending until successful
-  deactivate AA
-  note over AA: status: ACTIVE/ACTIVE
-  BS ->> BA: MSG: HELLO: Bob's agent<br>knows Alice can send
-  note over BA: status: ACTIVE/ACTIVE
-
-  note over A, B: 8. notify users about connection success
-  AA ->> A: CON: connected
-  BA ->> B: CON: connected

diagrams/graph-chat/duplex-creating.mmd

@@ -1,40 +0,0 @@
-sequenceDiagram
-  participant A as Alice's app
-  participant SA as Alice's servers <br> (connections CAi)
-  participant SB as Bob's servers <br> (connections CBi)
-  participant B as Bob's app
-
-  note over A, SA: 1. initiate duplex connection
-  A ->> SA: create n connections CAi (send RKAi)
-  SA ->> A: receive connections URIs (RUAi and SUAi)
-  A ->> SA: subscribe to messages from CAi
-
-  A -->> B: 2. send secure message with connections "public" keys (EKAi) and URIs (SUAi)
-
-  note over B: 3. accept Alice's <br> duplex connection
-  B ->> SB: create n connections CBi (send RKBi)
-  SB ->> B: receive connections URIs (RUBi and SUBi)
-  B ->> SB: subscribe to messages from CBi
-  note over B: prepare msgs with: <br> - sender key SKAi <br> - Bob's profile <br> - EKBi and SUBi
-  B ->> SA: accept CAi (send EKAi-encrypted messages to CAi)
-
-  note over A: 4. Add Bob's duplex <br> connection
-  SA ->> A: receive Bob's app messages
-  note over A: identify Bob and <br> confirm connection
-  A ->> SA: secure connections CAi with Bob's SKAi
-  note over A: prepare msgs with: <br> - sender key SKBi <br> - Alice's profile <br> - CAi confirmation
-  A ->> SB: accept CBi (send EKBi-encrypted messages to CBi)
-  note over A: Bob's duplex conn. <br> "pending"
-
-  note over B: 5. Add Alice's duplex <br> connection
-  SB ->> B: receive Alice's app messages
-  B ->> SB: secure connections CBi with Alice's SKBi
-  B ->> SA: send "welcome" message to duplex connection (via all CAi)
-  note over B: Alice's duplex conn. <br> "pending"
-
-  SA ->> A: 6. receive Bob's "welcome" message
-  note over A: Bob's duplex conn. <br> "established"
-  A ->> SB: send "welcome" message to duplex connection (via all CBi)
-
-  SB ->> B: 7. receive Alice's "welcome" message
-  note over B: Alice's duplex conn. <br> "established"

diagrams/graph-chat/duplex-using.mmd

@@ -1,21 +0,0 @@
-sequenceDiagram
-  participant A as Alice's app
-  participant SA as Alice's servers <br> (connections CAi)
-  participant SB as Bob's servers <br> (connections CBi)
-  participant B as Bob's app
-
-  note over A: 1. Alice writes msg <br> to Bob in the app: <br> - message ID <br> - timestamp <br> - message body
-  note over A: sign and encrypt <br> message version <br> for each conn. CBi
-  A ->> SB: send message versions to connections CBi on Bob's servers
-
-  SB ->> B: 2. retrive message versions from CBi
-  note over B: decrypt and verify <br> message versions
-  note over B: discard duplicates <br> and show in chat <br> with Alice
-
-  note over B: 3. prepare msg <br> "message received": <br> - new message ID <br> - msg correlation ID <br> - receipt timestamp
-  note over B: sign and encrypt <br> message version <br> for each conn. CAi
-  B ->> SA: send message versions to connections CAi on Alice's servers
-
-  SA ->> A: 4. retrive message versions from CAi
-  note over A: decrypt and verify <br> message versions
-  note over A: discard duplicates <br> and show sent msg <br> as delivered <br> in chat with Bob

diagrams/simplex-messaging/simplex-creating.mmd

@@ -1,20 +0,0 @@
-sequenceDiagram
-  participant B as Bob (sender)
-  participant S as server (queue RID)
-  participant A as Alice (recipient)
-
-  note over A: creating queue<br>("public" key RK<br>for msg retrieval)
-  A ->> S: 1. create queue ("NEW")
-  S ->> A: respond with queue RID and SID ("IDS")
-
-  note over A: out-of-band msg<br>(sender's queue SID<br>and "public" key EK<br>to encrypt msgs)
-  A -->> B: 2. send out-of-band message
-
-  note over B: confirm queue<br>("public" key SK for<br>sending messages<br>and any optional<br>info encrypted with<br>"public" key EK)
-  B ->> S: 3. confirm queue ("SEND" command not signed)
-
-  S ->> A: 4. deliver Bob's message
-  note over A: decrypt message<br>("private" key EK)
-  A ->> S: 5. secure queue ("KEY", RK-signed)
-
-  note over S: 6. simplex<br>queue RID<br>is ready to use!

diagrams/simplex-messaging/simplex-op.mmd

@@ -1,26 +0,0 @@
-sequenceDiagram
-  participant S as sender (client)
-  participant A as transport
-  participant Q as pubsub
-  participant P as transport
-  participant R as receiver (client)
-  note over R: sign subscription (1)
-  R ->> P: subscribe to messages
-  note over P: verify subscriber (1)
-  alt subscriber verified?
-    P -->> Q: subscribe
-  else
-    P ->> R: reject subscription
-  end
-  note over S: sign message (2)
-  S ->> A: send message
-  note over A: verify sender (2)
-  alt sender verified?
-    A -->> Q: queue message
-    activate Q
-  else
-    A ->> S: reject message
-  end
-  Q -->> P: take message
-  deactivate Q
-  P ->> R: deliver message

diagrams/simplex-messaging/simplex-using.mmd

@@ -1,10 +0,0 @@
-sequenceDiagram
-  participant B as Bob (sender)
-  participant S as server (queue RID)
-  participant A as Alice (recipient)
-
-  note over B: encrypt message<br>("public" key EK)
-  B ->> S: 1. send message to SID (SK-signed command)
-
-  S ->> A: 2. receive messages from RID (RK-signed subscription)
-  note over A: decrypt message<br>("private" key EK)

diagrams/simplex-messaging/simplex.mmd

@@ -1,11 +0,0 @@
-graph LR
-  VS{{"verify sender (SK)"}}
-  VR{{"verify recipient (RK)"}}
-
-  S(sender) -->|msg| VS
-  subgraph "server (queue RID, SID)"
-    VS --> DB[("storage")]
-    DB --> VR
-  end
-  R(recipient) -->|"1) sub"| VR
-  VR -->|"2) msg"| R

graph-chat.md

@@ -1,214 +0,0 @@
-# Graph-chat protocol
-
-A generic chat protocol for client applications that communicate via simplex messaging protocol
-
-
-## Problems of the existing chat platforms and protocols
-
-- Dependency on a single company/server to access and use chat. That creates implications for chat privacy, user profile resilience and data ownership.
-- Visiblity of user profile information to chat system and other chat users. Chat users have limited control as to who can see and find their profile.
-- Visibility of user contacts graph to chat server.
-- [E2EE][1] can be compromised by [MITM][2] attack (see [simplex messaging protocol][3]).
-- Identity related problems (also see [simplex messaging protocol][3]).
-
-
-## Graph-chat protocol abstract
-
-TODO
-
-
-## Duplex connection
-
-Majority of chat scenarios requires duplex (bi-directional) connections between participants. Graph-chat protocol uses multiple simplex (unidirectional) connections created on multiple simplex messaging servers to implement duplex connections.
-
-Each duplex connection consists of one or multiple, for redundancy, pairs of simplex connections to connect chat participants or devices of the same participant - it is used for "contacts", "devices", "group participants", etc. For practical purposes of redundancy, chat clients can use 2-4 pairs of simplex connections.
-
-The process described below establishes a duplex connection between Alice and Bob that has `n` simplex connections `CAi` (where `1 <= i <= n`) from Bob to Alice (created by Alice on her servers) and `n` simplex connections `CBi` (where `1 <= i <= n`) from Alice to Bob (created by Bob on his servers).
-
-The following symbols are used below:
-- simplex connections:
-  - `CAi` - Alice's simplex connection number `i` (out of `n`) allowing Bob to send messages to Alice.
-  - `CBi` - Bob's simplex connection number `i` (out of `n`) allowing Alice to send messages to Bob.
-- keys created for Alice's connections:
-  - `RUAi` - server-generated recipient connection URI of `CAi` (to be used by Alice to retrieve messages).
-  - `EKAi` - Alice's assymetric key pair used:
-    - by Bob to encrypt and by Alice to decrypt messages from Bob sent via `CAi`.
-    - by Alice to sign and Bob to verify messages from Alice sent via `CBi`.
-  - `SUAi` - server-generated sender connection URI of `CAi` (to be used by Bob to send messages).
-  - `RKAi` - Alice's recipient key of `CAi`.
-  - `SKAi` - Bob's sender key of `CAi`.
-- keys created for Bob's connections
-  - `RUBi` - server-generated recipient connection URI of `CBi` (to be used by Bob to retrive messages).
-  - `EKBi` - Bob's assymetric key pair used for:
-    - Alice to encrypt and Bob to decrypt messages from Alice sent via `CBi`.
-    - Bob to sign and Alice to verify messages from Bob sent via `CAi`.
-  - `SUBi` - server-generated sender connection URI of `CBi` (to be used by Alice to send messages).
-  - `RKBi` - Bob's recipient key of `CBi`.
-  - `SKBi` - Alice's sender key of `CBi`.
-
-
-### Creating duplex connection
-
-To create a duplex connection initiated by Alice, Alice's and Bob's apps follow these steps:
-
-1. Alice's app initiates duplex connection:
-   1. it creates `n` simplex connections `CAi` (step 1 in [simplex messaging][3]) that are defined by recipient URIs `RUAi` and have:
-      - client-generated:
-         - Alice's asymmetric key pairs `EKAi` to encrypt messages.
-         - recipient keys `RKAi`.
-      - server-generated:
-         - sender URIs `SUAi`.
-   2. optionally, Alice's app subsribes to receive messages from these new connections `CAi` ([simplex messaging protocol implementation][4] defines protocol to be used for subscriptions).
-2. Alice's app sends a secure message to Bob's app (step 2 in [simplex messaging][3]):
-   1. it prepares the message with the information needed to establish all connections `CAi`, including all encryption keys `EKAi` and sender connection URIs (`SUAi`).
-   2. depending on the communication scenario, Alice's app sends this message to Bob's app in one of available secure ways (either out-of-band or via available secure duplex connection(s) - see [Duplex connection security level](#TODO)), depending on communication scenario:
-      - if Bob is a new contact, the information is presented as a visual code (e.g. QR code(s)) to Bob's app - as out-of-band message needed to establish connections `CAi` (see [simplex messaging protocol][3] and [Adding direct contact](#adding-direct-contact)).
-      - if Bob is added to group chat with Alice by some contact of Alice or Bob, this information can be passed through all possible chains of contacts between Alice and Bob in the group (to minimise the risk of MITM attack) (see [Group chat](#TODO)).
-      - if Alice adds Bob as a contact via Bob's trusted contact John, who also has Alice as a trusted contact, this information will be passed via John (who has secure duplex connections with both Alice and Bob), in the same way as with Group chat (see [Trusted contacts](#TODO)).
-      - if Alice already has Bob as a contact, and she wants to create a separate off-the-record chat with him, this information will be passed via their existing connection (see [Off-the-record chat](#TODO)).
-      - etc. In all cases, the protocol defines the most secure possible way to pass the out-of-band (from the point of view of the new connections) message required by [simplex messaging protocol][3] to create simplex connections.
-3. Bob's app accepts duplex connection with Alice:
-   1. it receives the message from Alice's app, in a way defined by a specific chat scenario.
-   2. it interprets the received information as simplex messaging protocol out-of-band messages to accept all simplex connections `CAi`.
-   3. it creates `n` new simplex connections `CBi` on Bob's servers defined by recipient URIs `RUBi` and have:
-      - client-generated:
-         - Bob's asymmetric key pairs `EKBi` to encrypt messages.
-         - recipient keys `RKBi`.
-      - server-generated:
-         - sender URIs `SUBi`.
-   4. optionally, Bob's app subsribes to receive messages from these new connection `CBi` (see [simplex messaging protocol implementation][4]).
-   5. it proceeds with accepting Alice's app connections `CAi` (step 3 in [simplex messaging protocol][3]).
-   6. in response to each connection `CAi`, as optional information, Bob's app includes:
-      - Bob's user profile (that is only stored in graph-chat client and not visible to any server)
-      - information to establish connection `CBi`:
-         - encryption key `EKBi`.
-         - sender connection URIs (`SUBi`).
-   7. his app now sends the unsigned requests to Alice's connections `CAi` (step 3.4 in [simplex messaging protocol][3]), to both confirm the connections `CAi` and to propose the new connections `CBi`. As each message is encrypted by the key `EKAi` of the connection `CAi` that only Alice can decrypt, it is safe to send it - from simplex messaging server point of view it is an out-of-band message.
-4. Alice's app adds Bob's duplex connection:
-   1. it receives the messages from Bob via connections `CAi` (step 4 in [simplex messaging protocol][3]).
-   2. depending on chat scenario, Bob is identified and confirmed:
-      - for new contact, Alice may visually identify Bob's user profile and accepts Bob as a contact.
-      - for group participant, Alice's app will match known Bob's user profile ID with received user profile ID (that is only visible to the clients apps that have this profile).
-   3. it secures the connections `CAi` with keys `SKAi` received from Bob - the connections are now established (step 5 in [simplex messaging protocol][3]).
-   4. it accepts the connections `CBi`, including in the response to Bob's server Alice's user profile and (as the additional information) the confirmation that the connections `CAi` are secured and can be used (step 3 in [simplex messaging protocol][3]).
-   5. it sends the unsigned messages via connections `CBi`.
-   6. it adds Bob's duplex connection to the list of available duplex connections as "pending" (Alice cannot yet send messages to Bob, but Bob already can send messages to Alice). Possibly, the status of duplex connection can indicate that Bob already can send messages to Alice.
-5. Bob's app adds duplex connection with Alice:
-   1. it receives the initial messages via connections `CBi`.
-   2. it secures the connections `CBi` - they are now established as well (step 5 in [simplex messaging protocol][3]).
-   3. it adds duplex connection with Alice in the list of available duplex connections as "pending" (to indicate that Alice cannot yet send messages). Possibly, the status of duplex connection can indicate that Bob already can send messages to Alice.
-   4. it sends a special message (message type "welcome") to Alice's app via duplex connection (i.e., via all connections `CAi`) to confirm that adding Alice's contact is completed (see [Sending messages via duplex connection](#TODO)).
-6. Alice's app finalises adding duplex connection with Bob:
-   1. it receives "welcome" message from Bob's app via duplex connection.
-   2. it changes Bob's duplex connection status to "established".
-   3. it sends a special "welcome" message to Bob's app via duplex connection.
-7. Bob's app finalises adding duplex connection with Alice:
-   1. it receives "welcome" message from Alice's app via duplex connection.
-   2. it changes Alice's duplex connection status "established".
-
-**Creating duplex connection between Alice and Bob:**
-
-![Creating connection](/diagrams/graph-chat/duplex-creating.svg)
-
-
-### Sending message via duplex connection
-
-When Alice sends the message to Bob via the duplex connection, they follow these steps:
-
-1. Alice sends the message to Bob in the app:
-    1. her app prepares the message, including:
-       - client-generated message ID (unique per duplex connection).
-       - client timestamp.
-       - message body.
-    2. her app prepares a separate version of the message for each connection `CBi`, by signing each version of the message with the corresponding key `EKAi` and encrypting it with `EKBi`.
-    3. her app sends all copies of the message to corresponding connections `CBi`.
-2. Bob's app receives the message from Alice:
-    1. it retrieves all versions of the message via all connections `CBi` (they can arrive at different time and possibly out of order with other messages).
-    2. it decrypts each version with `EKBi` and verifies the signature with `EKAi`.
-    3. the first successfully decrypted message version is added to the Bob's chat with Alice and notification can be shown to Bob; if message verification failed it is marked as "unverified" in the chat.
-    4. all subsequently decrypted and verified copies are discarded.
-       - if some of the decryptions or verifications fail, the corresponding connection is marked as "possibly compromised" and has to be replaced by the app with another connection.
-       - if decryption or verification fails for all connections `CBi`, Alice's contact is marked as "compromised".
-3. Bob's app sends "message received" to Alice's app:
-    1. it prepares a special "message received" message, including:
-       - client-generated message ID (unique per duplex connection).
-       - correlation ID of the message received from Alice.
-       - client timestamp of the message reception.
-    2. it prepares a separate version of the message for each connection `CAi`, by signing each version of the message with the corresponding key `EKBi` and encrypting it with `EKAi`.
-    3. it sends all copies of the message to corresponding connections `CAi`.
-4. Alice's app receives "message received" from Bob's app:
-    1. it retrieves all versions of the "message received" message via all connections `CAi` (they can arrive at different time and possibly out of order with other messages).
-    2. it decrypts each version with `EKAi` and verifies the signature with `EKBi`.
-    3. once the first version is successfully decrypted and verified, the previosly sent message in the Alice's chat with Bob is marked as delivered.
-    4. all subsequently decrypted and verified copies are discarded.
-       - if some of the decryptions or verifications fail, the corresponding connection is marked as "possibly compromised" and has to be replaced by the app with another connection.
-       - if decryption or verification fails for all connections `CAi`, Bob's contact is marked as "compromised".
-    
-**Sending message from Alice to Bob via duplex connection:**
-
-![Sending message](/diagrams/graph-chat/duplex-using.svg)
-
-
-## Adding direct contact
-
-"Direct contact" is a term used for a duplex connection with another chat user established directly, not via another user. Establishing contact requires sending an out-of-band message - "visual code" (e.g. QR code(s)) is used for this purpose. 
-
-For example, Alice and Bob want to have a conversation in chat app, exchanging messages with each other. They both have graph-chat client and access to several simplex messaging servers (see [simplex messaging protocol][3]) that they can use to receive messages, and their graph-chat clients are configured to use these servers.
-
-To chat in the app Alice needs to add Bob as "direct contact" to her contacts in the app.
-
-1. Alice initiates adding "direct contact" in her graph-chat client app.
-2. Alice shares "visual code" (e.g. QR code(s)) with Bob:
-   1. her app prepares secure message to establish duplex connection with Bob's app (step 1 in [Creating duplex connection](#creating-duplex-connection)).
-   2. her app prepares and displays a visual code with this message (step 2 in [Creating duplex connection](#creating-duplex-connection)).
-   3. Alice now can share this visual code with Bob, either in person or via a video call (see [simplex messaging protocol][3]).
-3. Bob reads "visual code" that Alice prepared via his app:
-   1. he initiates adding "direct contact" via visual code in his graph-chat client app.
-   2. his app interprets this visual code as secure message required to accept duplex connection with Alice and proceeds with creating duplex connection (step 3 in [Creating duplex connection](#creating-duplex-connection)).
-4. Alice's app adds Bob as "direct contact":
-   1. it proceeds with creating duplex connection.
-   2. once "pending" duplex connection with Bob's app is created in Alice's app, Bob is added as "direct contact" with the status "pending" in Alice's app (step 4 in [Creating duplex connection](#creating-duplex-connection)).
-5. Bob's app adds Alice as "direct contact":
-   1. it proceeds with creating duplex connection.
-   2. once "pending" duplex connection with Alice's app is created in Bob's app, Alice is added as "direct contact" with the status "pending" in Bob's app (step 5 in [Creating duplex connection](#creating-duplex-connection)).
-6. Alice's app finalises adding Bob as "direct contact":
-   1. it finalises adding duplex connection with Bob's app (step 6 in [Creating duplex connection](#creating-duplex-connection)).
-   2. it changes Bob's "direct contact" status to "established".
-7. Bob's app finalises adding Alice as "direct contact":
-   1. it finalises adding duplex connection with Alice's app (step 7 in [Creating duplex connection](#creating-duplex-connection)).
-   2. it changes Alice's "direct contact" status to "established".
-
-
-## Connection and message types used in graph-chat protocol
-
-##### Duplex connections
-
-TODO
-
-- "contact" - can be of type "person", "bot", "device", "organisation".
-- "group-participant" - connection to another group participant that was established via the chain of other contacts in the group.
-- "broadcast" - duplex connection from broadcast subscriber to publisher; client apps should only allow to receive messages and send back control messages, but not the content messages. Profile of the subscriber is not shared with the publisher.
-- "broadcast-subscriber" - duplex connection from broadcast publisher to subscriber.
-
-
-##### Messages
-
-Control messages:
-
-- "welcome" - sent and recieved by both participants when duplex connection is being established.
-- "receipt" - acknowledging message receipt.
-- "contact: update" - changing "contact" profile .
-- "profile" - changing "contact 
-- "gm: add" - sent to add simplex connection to graph-chat duplex connection.
-- "gm: remove" - sent to remove simplex connection from graph-chat duplex connection.
-
-Content messages:
-
-- "text"
-- "image"
-
-
-[1]: https://en.wikipedia.org/wiki/Man-in-the-middle_attack
-[2]: https://en.wikipedia.org/wiki/End-to-end_encryption
-[3]: simplex-messaging.md
-[4]: simplex-messaging-implementation.md

images/logo.svg

@@ -0,0 +1,12 @@
+<svg width="815" height="233" viewBox="0 0 815 233" fill="none" xmlns="http://www.w3.org/2000/svg">
+<g clip-path="url(#clip0)">
+<path fill-rule="evenodd" clip-rule="evenodd" d="M40.5683 66.9141C43.9954 66.9141 47.376 67.2323 50.7104 67.8687C54.0448 68.5051 57.2403 69.3234 60.2968 70.3235C63.3533 71.3236 66.2014 72.4601 68.8411 73.7329C71.4809 75.0058 73.9122 76.2786 76.1351 77.5515L66.132 96.0987L65.9952 95.9474C65.6001 95.5596 64.6732 94.9282 63.2144 94.0531C61.5472 93.053 59.4632 92.0074 56.9624 90.9164C54.4616 89.8254 51.7293 88.8708 48.7654 88.0525C45.8015 87.2342 42.8376 86.8251 39.8737 86.8251C31.723 86.8251 27.6476 89.5072 27.6476 94.8713C27.6476 96.5079 28.0876 97.8716 28.9675 98.9627C29.8474 100.054 31.1441 101.031 32.8576 101.895C34.5711 102.758 36.7245 103.554 39.318 104.281C41.9114 105.009 44.9216 105.827 48.3486 106.736C53.0723 108.009 57.3329 109.395 61.1304 110.896C64.9279 112.396 68.1465 114.26 70.7862 116.487C73.4259 118.715 75.4636 121.419 76.8993 124.602C78.3349 127.784 79.0527 131.602 79.0527 136.057C79.0527 141.512 78.0107 146.126 75.9267 149.899C73.8427 153.673 71.0872 156.718 67.6602 159.037C64.2332 161.355 60.2968 163.037 55.8509 164.083C51.4051 165.128 46.8203 165.651 42.0966 165.651C38.4844 165.651 34.7795 165.378 30.982 164.833C27.1845 164.287 23.4796 163.492 19.8674 162.446C16.2551 161.401 12.7587 160.15 9.37796 158.696C5.99726 157.241 2.87128 155.559 0 153.65L10.0032 134.148L10.1812 134.338C10.6877 134.825 11.8324 135.625 13.6154 136.739C15.6531 138.012 18.177 139.285 21.1872 140.558C24.1974 141.83 27.555 142.967 31.2599 143.967C34.9647 144.967 38.7159 145.467 42.5134 145.467C50.5715 145.467 54.6006 143.058 54.6006 138.239C54.6006 136.421 53.9985 134.921 52.7944 133.739C51.5903 132.557 49.9231 131.489 47.7928 130.534C45.6625 129.579 43.1386 128.693 40.221 127.875C37.3034 127.056 34.1311 126.147 30.7041 125.147C26.1657 123.783 22.2292 122.306 18.8948 120.715C15.5605 119.124 12.805 117.283 10.6284 115.192C8.45174 113.1 6.83086 110.691 5.76571 107.964C4.70056 105.236 4.16798 102.054 4.16798 98.4171C4.16798 93.3257 5.14051 88.8253 7.08557 84.9158C9.03063 81.0063 11.6703 77.7106 15.0047 75.0285C18.3391 72.3464 22.2061 70.3235 26.6056 68.9597C31.0051 67.5959 35.6594 66.9141 40.5683 66.9141ZM131.47 67.7323V164.56H108.685V67.7323H131.47ZM191.39 67.7323L216.954 118.328L242.657 67.7323H267.248V164.56H244.463V106.6L223.067 148.74H210.841L189.445 106.6V164.56H166.66V67.7323H191.39ZM419.925 67.7323V144.922H467.718V164.56H397.14V67.7323H419.925ZM560.072 67.7323V87.3706H514.78V106.191H553.681V124.329H514.78V144.922H561.323V164.56H491.995V67.7323H560.072ZM341.818 67.7323L343.378 67.7677C347.5 67.9563 351.31 68.8991 354.809 70.5962C358.745 72.5055 362.126 75.0058 364.951 78.097C367.776 81.1882 369.999 84.6885 371.62 88.598C373.24 92.5075 374.051 96.4624 374.051 100.463C374.051 104.736 373.287 108.827 371.758 112.737C370.23 116.646 368.1 120.147 365.368 123.238C362.635 126.329 359.324 128.784 355.434 130.602C351.976 132.218 348.189 133.116 344.072 133.296L342.513 133.33H322.507V164.56H299.722V67.7323H341.818ZM340.29 87.3706H322.507V113.828H341.124C343.81 113.828 346.125 112.691 348.07 110.418C350.016 108.145 350.988 104.827 350.988 100.463C350.988 98.1898 350.687 96.2351 350.085 94.5986C349.483 92.9621 348.672 91.5983 347.654 90.5073C346.635 89.4163 345.477 88.6207 344.18 88.1207C342.884 87.6206 341.587 87.3706 340.29 87.3706Z" fill="#062D56"/>
+<path fill-rule="evenodd" clip-rule="evenodd" d="M642.628 136.08L680.309 173.782L699.513 154.567L699.506 154.561L737.917 116.134L700.236 78.4367L700.243 78.4334L681.404 59.5826L642.993 98.014L642.99 98.0104L681.401 59.5829L643.725 21.881L662.929 2.6652L700.605 40.3673L739.016 1.93511L757.855 20.7859L719.443 59.2176L757.121 96.918L795.533 58.4875L814.373 77.3382L775.959 115.768L813.643 153.471L794.439 172.687L756.756 134.984L718.348 173.415L756.031 211.119L736.827 230.335L699.144 192.63L660.74 231.065L641.901 212.214L680.306 173.78L642.625 136.083L642.628 136.08Z" fill="#07B4B9"/>
+<path fill-rule="evenodd" clip-rule="evenodd" d="M604.77 59.7651L642.446 97.4664L680.856 59.035L699.696 77.8858L661.285 116.317L698.966 154.019L679.762 173.235L642.081 135.532L603.675 173.965L584.836 155.114L623.243 116.682L585.566 78.9809L604.77 59.7651Z" fill="#062D56"/>
+</g>
+<defs>
+<clipPath id="clip0">
+<rect width="815" height="233" fill="white"/>
+</clipPath>
+</defs>
+</svg>

migrations/20210612_initial.sql

@@ -0,0 +1,278 @@
+CREATE TABLE contact_profiles ( -- remote user profile
+  contact_profile_id INTEGER PRIMARY KEY,
+  display_name TEXT NOT NULL, -- contact name set by remote user (not unique), this name must not contain spaces
+  full_name TEXT NOT NULL,
+  properties TEXT NOT NULL DEFAULT '{}' -- JSON with contact profile properties
+);
+
+CREATE INDEX contact_profiles_index ON contact_profiles (display_name, full_name);
+
+CREATE TABLE users (
+  user_id INTEGER PRIMARY KEY,
+  contact_id INTEGER NOT NULL UNIQUE REFERENCES contacts ON DELETE CASCADE
+    DEFERRABLE INITIALLY DEFERRED,
+  local_display_name TEXT NOT NULL UNIQUE,
+  active_user INTEGER NOT NULL DEFAULT 0, -- 1 for active user
+  FOREIGN KEY (user_id, local_display_name)
+    REFERENCES display_names (user_id, local_display_name)
+    ON DELETE RESTRICT
+    ON UPDATE CASCADE
+    DEFERRABLE INITIALLY DEFERRED
+);
+
+CREATE TABLE display_names (
+  user_id INTEGER NOT NULL REFERENCES users,
+  local_display_name TEXT NOT NULL,
+  ldn_base TEXT NOT NULL,
+  ldn_suffix INTEGER NOT NULL DEFAULT 0,
+  PRIMARY KEY (user_id, local_display_name) ON CONFLICT FAIL,
+  UNIQUE (user_id, ldn_base, ldn_suffix) ON CONFLICT FAIL
+) WITHOUT ROWID;
+
+CREATE TABLE contacts (
+  contact_id INTEGER PRIMARY KEY,
+  contact_profile_id INTEGER REFERENCES contact_profiles, -- NULL if it's an incognito profile
+  user_id INTEGER NOT NULL REFERENCES users,
+  local_display_name TEXT NOT NULL,
+  is_user INTEGER NOT NULL DEFAULT 0, -- 1 if this contact is a user
+  via_group INTEGER REFERENCES groups (group_id) ON DELETE SET NULL,
+  created_at TEXT NOT NULL DEFAULT (datetime('now')),
+  FOREIGN KEY (user_id, local_display_name)
+    REFERENCES display_names (user_id, local_display_name)
+    ON DELETE RESTRICT
+    ON UPDATE CASCADE,
+  UNIQUE (user_id, local_display_name),
+  UNIQUE (user_id, contact_profile_id)
+);
+
+CREATE TABLE sent_probes (
+  sent_probe_id INTEGER PRIMARY KEY,
+  contact_id INTEGER NOT NULL UNIQUE REFERENCES contacts ON DELETE CASCADE,
+  probe BLOB NOT NULL,
+  user_id INTEGER NOT NULL REFERENCES users,
+  UNIQUE (user_id, probe)
+);
+
+CREATE TABLE sent_probe_hashes (
+  sent_probe_hash_id INTEGER PRIMARY KEY,
+  sent_probe_id INTEGER NOT NULL REFERENCES sent_probes ON DELETE CASCADE,
+  contact_id INTEGER NOT NULL REFERENCES contacts ON DELETE CASCADE,
+  user_id INTEGER NOT NULL REFERENCES users,
+  UNIQUE (sent_probe_id, contact_id)
+);
+
+CREATE TABLE received_probes (
+  received_probe_id INTEGER PRIMARY KEY,
+  contact_id INTEGER NOT NULL REFERENCES contacts ON DELETE CASCADE,
+  probe BLOB,
+  probe_hash BLOB NOT NULL,
+  user_id INTEGER NOT NULL REFERENCES users
+);
+
+CREATE TABLE known_servers(
+  server_id INTEGER PRIMARY KEY,
+  host TEXT NOT NULL,
+  port TEXT NOT NULL,
+  key_hash BLOB,
+  user_id INTEGER NOT NULL REFERENCES users,
+  UNIQUE (user_id, host, port)
+) WITHOUT ROWID;
+
+CREATE TABLE group_profiles ( -- shared group profiles
+  group_profile_id INTEGER PRIMARY KEY,
+  display_name TEXT NOT NULL, -- this name must not contain spaces
+  full_name TEXT NOT NULL,
+  properties TEXT NOT NULL DEFAULT '{}' -- JSON with user or contact profile
+);
+
+CREATE TABLE groups (
+  group_id INTEGER PRIMARY KEY, -- local group ID
+  user_id INTEGER NOT NULL REFERENCES users,
+  local_display_name TEXT NOT NULL, -- local group name without spaces
+  group_profile_id INTEGER REFERENCES group_profiles, -- shared group profile
+  inv_queue_info BLOB,
+  FOREIGN KEY (user_id, local_display_name)
+    REFERENCES display_names (user_id, local_display_name)
+    ON DELETE RESTRICT
+    ON UPDATE CASCADE,
+  UNIQUE (user_id, local_display_name),
+  UNIQUE (user_id, group_profile_id)
+);
+
+CREATE TABLE group_members ( -- group members, excluding the local user
+  group_member_id INTEGER PRIMARY KEY,
+  group_id INTEGER NOT NULL REFERENCES groups ON DELETE RESTRICT,
+  member_id BLOB NOT NULL, -- shared member ID, unique per group
+  member_role TEXT NOT NULL, -- owner, admin, member
+  member_category TEXT NOT NULL, -- see GroupMemberCategory
+  member_status TEXT NOT NULL, -- see GroupMemberStatus
+  invited_by INTEGER REFERENCES contacts (contact_id) ON DELETE RESTRICT, -- NULL for the members who joined before the current user and for the group creator
+  group_queue_info BLOB,
+  direct_queue_info BLOB,
+  user_id INTEGER NOT NULL REFERENCES users,
+  local_display_name TEXT NOT NULL, -- should be the same as contact
+  contact_profile_id INTEGER NOT NULL REFERENCES contact_profiles ON DELETE RESTRICT,
+  contact_id INTEGER REFERENCES contacts ON DELETE RESTRICT,
+  FOREIGN KEY (user_id, local_display_name)
+    REFERENCES display_names (user_id, local_display_name)
+    ON DELETE RESTRICT
+    ON UPDATE CASCADE,
+  UNIQUE (group_id, member_id)
+);
+
+CREATE TABLE group_member_intros (
+  group_member_intro_id INTEGER PRIMARY KEY,
+  re_group_member_id INTEGER NOT NULL REFERENCES group_members (group_member_id) ON DELETE CASCADE,
+  to_group_member_id INTEGER NOT NULL REFERENCES group_members (group_member_id) ON DELETE CASCADE,
+  group_queue_info BLOB,
+  direct_queue_info BLOB,
+  intro_status TEXT NOT NULL, -- see GroupMemberIntroStatus
+  UNIQUE (re_group_member_id, to_group_member_id)
+);
+
+CREATE TABLE files (
+  file_id INTEGER PRIMARY KEY,
+  contact_id INTEGER REFERENCES contacts ON DELETE RESTRICT,
+  group_id INTEGER REFERENCES groups ON DELETE RESTRICT,
+  file_name TEXT NOT NULL,
+  file_path TEXT,
+  file_size INTEGER NOT NULL,
+  chunk_size INTEGER NOT NULL,
+  created_at TEXT NOT NULL DEFAULT (datetime('now')),
+  user_id INTEGER NOT NULL REFERENCES users
+);
+
+CREATE TABLE snd_files (
+  file_id INTEGER NOT NULL REFERENCES files ON DELETE RESTRICT,
+  connection_id INTEGER NOT NULL REFERENCES connections ON DELETE RESTRICT,
+  file_status TEXT NOT NULL, -- new, accepted, connected, completed
+  group_member_id INTEGER REFERENCES group_members ON DELETE RESTRICT,
+  PRIMARY KEY (file_id, connection_id)
+) WITHOUT ROWID;
+
+CREATE TABLE rcv_files (
+  file_id INTEGER PRIMARY KEY REFERENCES files ON DELETE RESTRICT,
+  file_status TEXT NOT NULL, -- new, accepted, connected, completed
+  group_member_id INTEGER REFERENCES group_members ON DELETE RESTRICT,
+  file_queue_info BLOB
+);
+
+CREATE TABLE snd_file_chunks (
+  file_id INTEGER NOT NULL,
+  connection_id INTEGER NOT NULL,
+  chunk_number INTEGER NOT NULL,
+  chunk_agent_msg_id INTEGER,
+  chunk_sent INTEGER NOT NULL DEFAULT 0, -- 0 (sent to agent), 1 (sent to server)
+  FOREIGN KEY (file_id, connection_id) REFERENCES snd_files ON DELETE CASCADE,
+  PRIMARY KEY (file_id, connection_id, chunk_number)
+) WITHOUT ROWID;
+
+CREATE TABLE rcv_file_chunks (
+  file_id INTEGER NOT NULL REFERENCES rcv_files,
+  chunk_number INTEGER NOT NULL,
+  chunk_agent_msg_id INTEGER NOT NULL,
+  chunk_stored INTEGER NOT NULL DEFAULT 0, -- 0 (received), 1 (appended to file)
+  PRIMARY KEY (file_id, chunk_number)
+) WITHOUT ROWID;
+
+CREATE TABLE connections ( -- all SMP agent connections
+  connection_id INTEGER PRIMARY KEY,
+  agent_conn_id BLOB NOT NULL UNIQUE,
+  conn_level INTEGER NOT NULL DEFAULT 0,
+  via_contact INTEGER REFERENCES contacts (contact_id),
+  conn_status TEXT NOT NULL,
+  conn_type TEXT NOT NULL, -- contact, member, rcv_file, snd_file
+  contact_id INTEGER REFERENCES contacts ON DELETE RESTRICT,
+  group_member_id INTEGER REFERENCES group_members ON DELETE RESTRICT,
+  snd_file_id INTEGER,
+  rcv_file_id INTEGER REFERENCES rcv_files (file_id) ON DELETE RESTRICT,
+  created_at TEXT NOT NULL DEFAULT (datetime('now')),
+  user_id INTEGER NOT NULL REFERENCES users,
+  FOREIGN KEY (snd_file_id, connection_id)
+    REFERENCES snd_files (file_id, connection_id)
+    ON DELETE RESTRICT
+    DEFERRABLE INITIALLY DEFERRED
+);
+
+CREATE TABLE events ( -- messages received by the agent, append only
+  event_id INTEGER PRIMARY KEY,
+  agent_msg_id INTEGER NOT NULL, -- internal message ID
+  external_msg_id INTEGER NOT NULL, -- external message ID (sent or received)
+  agent_meta TEXT NOT NULL, -- JSON with timestamps etc. sent in MSG
+  connection_id INTEGER NOT NULL REFERENCES connections,
+  received INTEGER NOT NULL, -- 0 for received, 1 for sent
+  chat_event_id INTEGER,
+  continuation_of INTEGER, -- references chat_event_id, but can be incorrect
+  event_type TEXT NOT NULL, -- event type - see protocol/types.ts
+  event_encoding INTEGER NOT NULL, -- format of event_body: 0 - binary, 1 - text utf8, 2 - JSON (utf8)
+  content_type TEXT NOT NULL, -- content type - see protocol/types.ts
+  event_body BLOB, -- agent message body as sent
+  event_hash BLOB NOT NULL,
+  integrity TEXT NOT NULL DEFAULT '',
+  created_at TEXT NOT NULL DEFAULT (datetime('now'))
+);
+
+CREATE INDEX events_external_msg_id_index ON events (connection_id, external_msg_id);
+
+CREATE TABLE event_body_parts (
+  event_body_part_id INTEGER PRIMARY KEY,
+  event_id REFERENCES events,
+  full_size INTEGER NOT NULL,
+  part_status TEXT, -- full, partial
+  content_type TEXT NOT NULL,
+  event_part BLOB
+);
+
+CREATE TABLE contact_profile_events (
+  event_id INTEGER NOT NULL UNIQUE REFERENCES events,
+  contact_profile_id INTEGER NOT NULL REFERENCES contact_profiles
+);
+
+CREATE TABLE group_profile_events (
+  event_id INTEGER NOT NULL UNIQUE REFERENCES events,
+  group_profile_id INTEGER NOT NULL REFERENCES group_profiles
+);
+
+CREATE TABLE group_events (
+  event_id INTEGER NOT NULL UNIQUE REFERENCES events,
+  group_id INTEGER NOT NULL REFERENCES groups ON DELETE RESTRICT,
+  group_member_id INTEGER REFERENCES group_members -- NULL for current user
+);
+
+CREATE TABLE group_event_parents (
+  group_event_parent_id INTEGER PRIMARY KEY,
+  event_id INTEGER NOT NULL REFERENCES group_events (event_id),
+  parent_group_member_id INTEGER REFERENCES group_members (group_member_id), -- can be NULL if parent_member_id is incorrect
+  parent_member_id BLOB, -- shared member ID, unique per group
+  parent_event_id INTEGER REFERENCES events (event_id) ON DELETE CASCADE, -- this can be NULL if received event references another event that's not received yet
+  parent_chat_event_id INTEGER NOT NULL,
+  parent_event_hash BLOB NOT NULL
+);
+
+CREATE INDEX group_event_parents_parent_chat_event_id_index
+  ON group_event_parents (parent_member_id, parent_chat_event_id);
+
+CREATE TABLE messages ( -- mutable messages presented to user
+  message_id INTEGER PRIMARY KEY,
+  contact_id INTEGER NOT NULL REFERENCES contacts ON DELETE RESTRICT, -- 1 for sent messages
+  group_id INTEGER REFERENCES groups ON DELETE RESTRICT, -- NULL for direct messages
+  deleted INTEGER NOT NULL, -- 1 for deleted
+  msg_type TEXT NOT NULL,
+  content_type TEXT NOT NULL,
+  msg_text TEXT NOT NULL, -- textual representation
+  msg_props TEXT NOT NULL -- JSON
+);
+
+CREATE TABLE message_content (
+  message_content_id INTEGER PRIMARY KEY,
+  message_id INTEGER REFERENCES messages ON DELETE CASCADE,
+  content_type TEXT NOT NULL,
+  content_size INTEGER, -- full expected content size
+  content_status TEXT, -- empty, part, full
+  content BLOB NOT NULL
+);
+
+CREATE TABLE message_events (
+  event_id INTEGER NOT NULL UNIQUE REFERENCES events,
+  message_id INTEGER NOT NULL REFERENCES messages
+);

package.yaml

@@ -1,5 +1,5 @@
 name: simplex-chat
-version: 0.3.0
+version: 0.4.2
 #synopsis:
 #description:
 homepage: https://github.com/simplex-chat/simplex-chat#readme
@@ -11,29 +11,56 @@ category: Web, System, Services, Cryptography
 extra-source-files:
   - README.md
 
+dependencies:
+  - aeson == 1.5.*
+  - ansi-terminal == 0.10.*
+  - attoparsec == 0.13.*
+  - base >= 4.7 && < 5
+  - base64-bytestring >= 1.0 && < 1.3
+  - bytestring == 0.10.*
+  - composition == 1.0.*
+  - containers == 0.6.*
+  - cryptonite >= 0.27 && < 0.30
+  - directory == 1.3.*
+  - exceptions == 0.10.*
+  - file-embed == 0.0.14.*
+  - filepath == 1.4.*
+  - mtl == 2.2.*
+  - optparse-applicative == 0.15.*
+  - process == 1.6.*
+  - simple-logger == 0.1.*
+  - simplexmq == 0.4.*
+  - sqlite-simple == 0.4.*
+  - stm == 2.5.*
+  - terminal == 0.2.*
+  - text == 1.2.*
+  - time == 1.9.*
+  - unliftio == 0.2.*
+  - unliftio-core == 0.2.*
+
+library:
+  source-dirs: src
+
 executables:
-  dog-food:
-    source-dirs: apps/dog-food
+  simplex-chat:
+    source-dirs: apps/simplex-chat
     main: Main.hs
     dependencies:
-      - base == 4.13.*
-      - ansi-terminal == 0.10.*
-      - async == 2.2.*
-      - attoparsec == 0.13.*
-      - bytestring == 0.10.*
-      - directory == 1.3.*
-      - filepath == 1.4.*
-      - mtl == 2.2.*
-      - optparse-applicative == 0.15.*
-      - simple-logger == 0.1.*
-      - simplexmq == 0.3.*
-      - stm == 2.5.*
-      - terminal == 0.2.*
-      - text == 1.2.*
-      - unliftio == 0.2.*
+      - simplex-chat
     ghc-options:
       - -threaded
 
+tests:
+  simplex-chat-test:
+    source-dirs: tests
+    main: Test.hs
+    dependencies:
+      - simplex-chat
+      - async == 2.2.*
+      - hspec == 2.7.*
+      - network == 3.1.*
+      - stm == 2.5.*
+
 ghc-options:
   # - -haddock
   - -Wall

protocol/types.ts

@@ -0,0 +1,74 @@
+// x. namespace is for chat messages transmitted inside SMP agent MSG
+type MemberMessageType =
+  | "x.grp.info" // group profile information or update
+  | "x.grp.off" // disable group
+  | "x.grp.del" // group deleted
+  | "x.grp.mem.new" // new group member
+  | "x.grp.mem.acl" // group member permissions (ACL)
+  | "x.grp.mem.leave" // group member left
+  | "x.grp.mem.off" // suspend group member
+  | "x.grp.mem.on" // enable group member
+  | "x.grp.mem.del" // group member removed
+
+type ProfileMessageType =
+  | "x.info" // profile information or update
+  | "x.info.grp" // information about group in profile
+  | "x.info.con" // information about contact in profile
+
+type NotificationMessageType = "x.msg.read"
+
+type OpenConnMessageType =
+  | "x.open.grp" // open invitation to the group
+  | "x.open.con" // open invitation to the contact
+
+type ContentMessageType =
+  | "x.msg.new" // new message
+  | "x.msg.append" // additional part of the message
+  | "x.msg.del" // delete message
+  | "x.msg.update" // update message
+  | "x.msg.fwd" // forward message
+  | "x.msg.reply" // reply to message
+
+// TODO namespace for chat messages transmitted as other agent messages
+
+type DirectMessageType =
+  | ProfileMessageType
+  | NotificationMessageType
+  | OpenConnMessageType
+  | ContentMessageType
+
+type GroupMessageType = MemberMessageType | DirectMessageType
+
+type ContentType =
+  | "c.text"
+  | "c.html"
+  | "c.image"
+  | "c.audio"
+  | "c.video"
+  | "c.doc"
+  | "c.sticker"
+  | "c.file"
+  | "c.link"
+  | "c.form"
+  | "c.poll"
+  | "c.applet"
+
+// the type of message data transmitted inside SMP agent MSG
+interface MessageData<T extends GroupMessageType> {
+  type: T
+  sent: Date
+  data: unknown
+}
+
+interface DirectMessageData<T extends DirectMessageType> extends MessageData<T> {}
+
+interface GroupMessageData<T extends GroupMessageType> extends MessageData<T> {
+  msgId: number
+  parents: ParentMessage[]
+}
+
+interface ParentMessage {
+  memberId: Uint8Array
+  msgId: number
+  msgHash: Uint8Array
+}

simplex-messaging.md

@@ -1,624 +0,0 @@
-# Simplex Messaging Protocol (SMP)
-
-## Table of contents
-
-- [Abstract](#abstract)
-- [Introduction](#introduction)
-- [SMP Model](#smp-model)
-- [Out-of-band messages](#out-of-band-messages)
-- [Simplex queue](#simplex-queue)
-- [SMP procedure](#smp-procedure)
-- [SMP qualities and features](#smp-qualities-and-features)
-- [Cryptographic algorithms](#cryptographic-algorithms)
-- [Simplex queue IDs](#simplex-queue-ids)
-- [Server privacy requirements](#server-privacy-requirements)
-- [SMP commands](#smp-commands)
-  - [Correlating responses with commands](#correlating-responses-with-commands)
-  - [Command authentication](#command-authentication)
-  - [Keep-alive command](#keep-alive-command)
-  - [Recipient commands](#recipient-commands)
-    - [Create queue command](#create-queue-command)
-    - [Subscribe to queue](#subscribe-to-queue)
-    - [Secure queue command](#secure-queue-command)
-    - [Acknowledge message delivery](#acknowledge-message-delivery)
-    - [Suspend queue](#suspend-queue)
-    - [Delete queue](#delete-queue)
-  - [Sender commands](#sender-commands)
-    - [Send message](#send-message)
-  - [Server messages](#server-messages)
-    - [Queue IDs response](#queue-ids-response)
-    - [Deliver queue message](#deliver-queue-message)
-    - [Subscription END notification](#subscription-end-notification)
-    - [Error responses](#error-responses)
-    - [OK response](#ok-response)
-- [Appendices](#appendices)
-  - [Appendix A. Transport connection with the SMP server](#appendix-a)
-
-## Abstract
-
-Simplex Messaging Protocol is a transport agnostic client-server protocol for asynchronous distributed secure unidirectional message transmission via persistent simplex message queues.
-
-It's designed with the focus on communication security and integrity, under the assumption that any part of the message transmission network can be compromised.
-
-It is designed as a low level protocol for other application protocols to solve the problem of secure and private message transmission, making [MITM attack][1] very difficult at any part of the message transmission system.
-
-## Introduction
-
-The objective of Simplex Messaging Protocol (SMP) is to facilitate the secure and private unidirectional transfer of messages from senders to recipients via persistent simplex queues managed by the message broker (server).
-
-SMP is independent of any particular transmission system and requires only a reliable ordered data stream channel. While this document describes transport over TCP, other transports are also possible.
-
-The protocol describes the set of commands that recipients and senders can exchange with SMP servers to create and to operate unidirectional "queues" (a data abstraction identifying one of many communication channels managed by the server) and to send messages from the sender to the recipient via the SMP server.
-
-More complex communication scenarios can be designed using multiple queues - for example, a duplex communication channel can be made of 2 simplex queues.
-
-The protocol is designed with the focus on privacy and security, to some extent deprioritizing reliability by requiring that SMP servers only store messages until they are acknowledged by the recipients and, in any case, for a limited period of time. For communication scenarios requiring more reliable transmission the users should use several SMP servers to pass each message and implement some additional protocol to ensure that messages are not removed, inserted or changed - this is out of scope of this document.
-
-SMP does not use any form of participants' identities and provides [E2EE][2] without the possibility of [MITM attack][1] relying on two pre-requisites:
-
-- the users can establish a secure encrypted transport connection with the SMP server. [Appendix A](#appendix-a) describes SMP transport protocol of such connection over TCP, but any other transport connection protocol can be used.
-
-- the recipient can pass a single message to the sender via a pre-existing secure and private communication channel (out-of-band message) - the information in this message is used to encrypt messages and to establish connection with SMP server.
-
-## SMP Model
-
-The SMP model has three communication participants: the recipient, the message broker (SMP server) that is chosen and, possibly, controlled by the recipient, and the sender.
-
-SMP server manages multiple "simplex queues" - data records on the server that identify communication channels from the senders to the recipients. The same communicating party that is the sender in one queue, can be the recipient in another - without exposing this fact to the server.
-
-The queue record consists of 2 unique random IDs generated by the server, one for the recipient and another for the sender, and 2 keys to authenticate the recipient and the sender respectively, provided by the client. The users of SMP protocol must use a unique key for each queue, to avoid the possibility of aggregating and analysing their queues in case SMP server is compromised.
-
-Creating and using the queue requires sending commands to the SMP server from the recipient and the sender - they are described in detail in [SMP commands](#smp-commands) section.
-
-## Out-of-band messages
-
-The out-of-band message with the queue information is sent via some trusted alternative channel from the recipient to the sender. This message is used to share the encryption (a.k.a. "public") key that the sender will use to encrypt the messages (to be decrypted by the recipient), sender queue ID, server hostname and any other information necessary to establish secure encrypted connection with SMP server (see [Appendix A](#appendix-a) for SMP transport protocol).
-
-The [ABNF][8]  syntax of the message is:
-
-```abnf
-outOfBandMsg = "smp::" server "::" queueId "::" encryptionKey
-server = <hostname> [":" port] ["#" serverKeyHash]
-port = 1*DIGIT
-serverKeyHash = encoded
-queueId = encoded
-encryptionKey = %s"rsa:" x509encoded ; the recipient's RSA public key for sender to encrypt messages
-x509encoded = <base64 X509 key encoding>
-encoded = <base64 encoded binary>
-```
-
-`hostname` can be IP address or domain name, as defined in RFC 1123, section 2.1.
-
-`port` is optional, the default TCP port for SMP protocol is 5223.
-
-`serverKeyHash` is an optional hash of the server transport key used during transport handshake (see [Appendix A](#appendix-a)).
-
-Encryption keys are encoded using [X509][11] specification.
-
-Defining the approach to out-of-band message passing is out of scope of this protocol.
-
-## Simplex queue
-
-The simplex queue is the main unit of SMP protocol. It is used by:
-
-- Sender of the queue (who received out-of-band message) to send messages to the server using sender's queue ID, signed by sender's key.
-
-- Recipient of the queue (who created the queue and sent out-of-band message) will use it to retrieve messages from the server, signing the commands by the recipient key.
-
-- Participant identities are not shared with the server - new unique keys and queue IDs are used for each queue.
-
-This simplex queue can serve as a building block for more complex communication network. For example, two (or more, for redundancy) simplex queues can be used to create a duplex communication channel. Higher level primitives that are only known to system participants in their client applications can be created as well - e.g., contacts, conversations, groups and broadcasts. Simplex messaging servers only have the information about the low-level simplex queues. In this way a high level of privacy and security of the communication is provided. Application level primitives are not in scope of this protocol.
-
-This approach is based on the concept of [unidirectional networks][4] that are used for applications with high level of information security.
-
-Access to each queue is controlled with unique (not shared with other queues) asymmetric key pairs, separate for the sender and the recipient. The sender and the receiver have private keys, and the server has associated public keys to authenticate participants' commands by verifying cryptographic signatures.
-
-The messages sent over the queue are encrypted and decrypted using another key pair that was shared via out-of-band message - the recipient has the private key and the sender has the associated public key.
-
-**Simplex queue diagram:**
-
-![Simplex queue](/diagrams/simplex-messaging/simplex.svg)
-
-Queue is defined by recipient ID `RID` and sender ID `SID`, unique for the server. Sender key (`SK`) is used by the server to verify sender's commands (identified by `SID`) to send messages. Recipient key (`RK`) is used by the server to verify recipient's commands (identified by `RID`) to retrieve messages.
-
-The protocol uses different IDs for sender and recipient in order to provide an additional privacy by preventing the correlation of senders and recipients commands sent over the network - in case the encrypted transport is compromised, it would still be difficult to correlate senders and recipients without access to the queue records on the server.
-
-## SMP procedure
-
-The SMP procedure of creating a simplex queue on SMP server is explained using participants Alice (the recipient) who wants to receive messages from Bob (the sender).
-
-To create and start using a simplex queue Alice and Bob follow these steps:
-
-1. Alice creates a simplex queue on the server:
-
-    1. Decides which SMP server to use (can be the same or different server that Alice uses for other queues) and opens secure encrypted transport connection to the chosen SMP server (see [Appendix A](#appendix-a)).
-
-    2. Generates a new random public/private key pair (encryption key - `EK`) that she did not use before for Bob to encrypt the messages.
-
-    3. Generates another new random public/private key pair (recipient key - `RK`) that she did not use before for her to sign commands and to decrypt the transmissions received from the server.
-
-    4. Sends `"NEW"` command to the server to create a simplex queue (see `create` in [Create queue command](#create-queue-command)). This command contains previously generated unique "public" key `RK` that will be used to sign the following commands related to the same queue, for example to subscribe to the messages received to this queue or to update the queue, e.g. by setting the key required to send the messages (initially Alice creates the queue that accepts unsigned messages, so anybody could send the message via this queue if they knew the queue sender's ID and server address).
-
-    5. The server sends `"IDS"` response with queue IDs (`queueIds`):
-
-        - Recipient ID `RID` for Alice to manage the queue and to receive the messages.
-
-        - Sender ID `SID` for Bob to send messages to the queue.
-
-2. Alice sends an out-of-band message to Bob via the alternative channel that both Alice and Bob trust (see [protocol abstract](#simplex-messaging-protocol-abstract)). The message must include:
-
-    - Unique "public" key (`EK`) that Bob must use to encrypt messages.
-
-    - SMP server hostname and information to open secure encrypted transport connection (see [Appendix A](#appendix-a)).
-
-    - Sender queue ID `SID` for Bob to use.
-
-3. Bob, having received the out-of-band message from Alice, connects to the queue:
-
-    1. Generates a new random public/private key pair (sender key - `SK`) that he did not use before for him to sign messages sent to Alice's server.
-
-    2. Prepares the confirmation message for Alice to secure the queue. This message includes:
-
-        - Previously generated "public" key `SK` that will be used by Alice's server to authenticate Bob's messages, once the queue is secured.
-
-        - Optionally, any additional information (application specific, e.g. Bob's profile name and details).
-
-    3. Encrypts the confirmation body with the "public" key `EK` (that Alice provided via the out-of-band message).
-
-    4. Sends the encrypted message to the server with queue ID `SID` (see `send` in [Send message](#send-message)). This initial message to the queue must not be signed - signed messages will be rejected until Alice secures the queue (below).
-
-4. Alice receives Bob's message from the server using recipient queue ID `RID` (possibly, via the same transport connection she already has opened - see `message` in [Deliver queue message](#deliver-queue-message)):
-
-    1. She decrypts received message with "private" key `EK`.
-
-    2. Even though anybody could have sent the message to the queue with ID `SID` before it is secured (e.g. if communication is compromised), Alice would ignore all messages until the decryption succeeds (i.e. the result contains the expected message format). Optionally, in the client application, she also may identify Bob using the information provided, but it is out of scope of SMP protocol.
-
-5. Alice secures the queue `RID` with `"KEY"` command so only Bob can send messages to it (see [Secure queue command](#secure-queue-command)):
-
-    1. She sends the `KEY` command with `RID` signed with "private" key `RK` to update the queue to only accept requests signed by "private" key `SK` provided by Bob.
-
-    2. From this moment the server will accept only signed commands to `SID`, so only Bob will be able to send messages to the queue `SID` (corresponding to `RID` that Alice has).
-
-    3. Once queue is secured, Alice deletes `SID` and `SK` - even if Alice's client is compromised in the future, the attacker would not be able to send messages pretending to be Bob.
-
-6. The simplex queue `RID` is now ready to be used.
-
-This flow is shown on the sequence diagram below.
-
-**Creating simplex queue from Bob to Alice:**
-
-![Creating queue](/diagrams/simplex-messaging/simplex-creating.svg)
-
-Bob now can securely send messages to Alice:
-
-1. Bob sends the message:
-
-    1. He encrypts the message to Alice with "public" key `EK` (provided by Alice, only known to Bob, used only for one simplex queue).
-
-    2. He signs `"SEND"` command to the server queue `SID` using the "private" key `SK` (that only he knows, used only for this queue).
-
-    3. He sends the command to the server (see `send` in [Send message](#send-message)), that the server will authenticate using the "public" key `SK` (that Alice earlier provided to the server).
-
-2. Alice receives the message(s):
-
-    1. She signs `"SUB"` command to the server to subscribe to the queue `RID` with the "private" key `RK` (see `subscribe` in [Subscribe to queue](#subscribe-to-queue)).
-
-    2. The server, having authenticated Alice's command with the "public" key `RK` that she provided, delivers Bob's message(s) (see `message` in [Deliver queue message](#deliver-queue-message)).
-
-    3. She decrypts Bob's message(s) with the "private" key `EK` (that only she has).
-
-    4. She acknowledges the message reception to the server with `"ACK"` so that the server can delete the message and deliver the next messages.
-
-This flow is show on sequence diagram below.
-
-**Sending messages from Bob to Alice via simplex queue:**
-
-![Using queue](/diagrams/simplex-messaging/simplex-using.svg)
-
-**Simplex queue operation:**
-
-![Simplex queue operations](/diagrams/simplex-messaging/simplex-op.svg)
-
-Sequence diagram does not show E2E encryption - server knows nothing about encryption between the sender and the receiver.
-
-A higher level protocol application protocol should define the semantics that allow to use two simplex queues (or two sets of queues for redundancy) for the bi-directional or any other communication scenarios.
-
-The SMP is intentionally unidirectional - it provides no answer to how Bob will know that the transmission succeeded, and whether Alice received any messages. There may be a scenario when Alice wants to securely receive the messages from Bob, but she does not want Bob to have any proof that she received any messages - this low-level protocol can be used in this scenario, as all Bob knows as a fact is that he was able to send one unsigned message to the server that Alice provided, and now he can only send messages signed with the key `SK` that he sent to the server - it does not prove that any message was received by Alice.
-
-For practical purposes of bi-directional conversation, now that Bob can securely send encrypted messages to Alice, Bob can create the second simplex queue that will allow Alice to send messages to Bob in the same way, sending the second queue details via the first queue. If both Alice and Bob have their respective unique "public" keys (Alice's and Bob's `EK`s of two separate queues), or pass additional keys to sign the messages, the conversation can be both encrypted and signed.
-
-The established queues can also be used to change the encryption keys providing [forward secrecy][5], or to negotiate using other SMP queue(s).
-
-This protocol also can be used for off-the-record messaging, as Alice and Bob can use multiple queues between them and only information they pass to each other allows proving their identity, so if they want to share anything off-the-record they can initiate a new queue without linking it to any other information they exchanged. As a result, this protocol provides better anonymity and better protection from [MITM][1] than [OTR][6] protocol.
-
-## SMP qualities and features
-
-Simplex Messaging Protocol:
-
-- Defines only message-passing protocol:
-
-  - Transport agnostic - the protocol does not define how clients connect to the servers. It can be implemented over any ordered data stream channel: TCP connection, HTTP with long polling, websockets, etc.
-
-  - Not semantic - the protocol does not assign any meaning to queues and messages. While on the application level the queues and messages can have different meaning (e.g., for messages: text or image chat message, message acknowledgement, participant profile information, status updates, changing "public" key to encrypt messages, changing servers, etc.), on SMP protocol level all the messages are binary and their meaning can only be interpreted by client applications and not by the servers - this interpretation is out of scope of this protocol.
-
-- Client-server architecture:
-
-  - Multiple servers, that can be deployed by the system users, can be used to send and retrieve messages.
-
-  - Servers do not communicate with each other and do not "know" about other servers.
-
-  - Clients only communicate with servers (excluding the initial out-of-band message), so the message passing is asynchronous.
-
-  - For each queue, the message recipient defines the server through which the sender should send messages.
-
-  - While multiple servers and multiple queues can be used to pass each message, it is in scope of application level protocol(s), and out of scope of this protocol.
-
-  - Servers store messages only until they are retrieved by the recipients, and in any case, for a limited time.
-
-  - Servers are required to NOT store any message history or delivery log, but even if the server is compromised, it does not allow to decrypt the messages or to determine the list of queues established by any participant - this information is only stored on client devices.
-
-- The only element provided by SMP servers is simplex queues:
-
-  - Each queue is created and managed by the queue recipient.
-
-  - Asymmetric encryption is used to sign and verify the requests to send and receive the messages.
-
-  - One unique "public" key is used by the servers to authenticate requests to send the messages into the queue, and another unique "public" key - to retrieve the messages from the queue. "Unique" here means that each "public" key is used only for one queue and is not used for any other context - effectively, this key is not public and does not represent any participant identity.
-
-  - Both "public" keys are provided to the server by the queue recipient when the queue is created.
-
-  - The "public" keys known to the server and used to authenticate commands from the participants are unrelated to the keys used to encrypt and decrypt the messages - the latter keys are also unique per each queue but they are only known to participants, not to the servers.
-
-  - Messaging graph can be asymmetric: Bob's ability to send messages to Alice does not automatically lead to the Alice's ability to send messages to Bob.
-
-## Cryptographic algorithms
-
-Simplex messaging clients need to cryptographically sign commands for the following operations:
-
-- With the recipient's key `RK` (server to verify):
-  - create the queue (`NEW`)
-  - subscribe to queue (`SUB`)
-  - secure the queue (`KEY`)
-  - acknowledge received messages (`ACK`)
-  - suspend the queue (`OFF`)
-  - delete the queue (`DEL`)
-- With the sender's key `SK` (server to verify):
-  - send messages (`SEND`)
-
-To sign and verify commands, clients and servers MUST use RSA-PSS algorithm defined in [RFC3447][2].
-
-To optionally sign and verify messages, clients SHOULD use RSA-PSS algorithm.
-
-To encrypt and decrypt messages, clients and servers SHOULD use RSA-OAEP algorithm defined in [RFC3447][2].
-
-The reasons to use these algorithms:
-
-- They are supported by WebCrypto API.
-- They are more widely supported than ECC algorithms.
-- They are newer versions than RSA-PKCS1-v1_5 encryption and signature schemes.
-
-Future versions of the protocol may allow different cryptographic algorithms.
-
-## Simplex queue IDs
-
-Simplex messaging servers MUST generate 2 different IDs for each new queue - for the recipient (that created the queue) and for the sender. It is REQUIRED that:
-
-- These IDs are different and unique within the server.
-- Based on random bytes generated with cryptographically strong pseudo-random number generator.
-
-## Server security requirements
-
-Simplex messaging server implementations MUST NOT create, store or send to any other servers:
-
-- Logs of the client commands and transport connections in the production environment.
-
-- History of deleted queues, retrieved or acknowledged messages (deleted queues MAY be stored temporarily as part of the queue persistence implementation).
-
-- Snapshots of the database they use to store queues and messages (instead simplex messaging clients must manage redundancy by using more than one simplex messaging server). In-memory persistence is recommended.
-
-- Any other information that may compromise privacy or [forward secrecy][4] of communication between clients using simplex messaging servers.
-
-## SMP commands
-
-Commands syntax below is provided using [ABNF][8] with [case-sensitive strings extension][8a].
-
-Each transmission between the client and the server must have this format/syntax (after the decryption):
-
-```abnf
-transmission = [signature] SP signed SP pad ; pad to the fixed block size
-signed = [corrId] SP [queueId] SP cmd
-cmd = ping / recipientCmd / send / serverMsg
-recipientCmd = create / subscribe / secure / acknowledge / suspend / delete
-serverMsg = pong / queueIds / message / unsubscribed / ok / error
-corrId = 1*(%x21-7F) ; any characters other than control/whitespace
-queueId = encoded ; empty queue ID is used with "create" command
-signature = encoded
-; empty signature can be used with "create", "send" and "ping" commands and server messages
-encoded = <base64 encoded binary>
-```
-
-`base64` encoding should be used with padding, as defined in section 4 of [RFC 4648][9]
-
-The syntax of specific commands and responses is defined below.
-
-### Correlating responses with commands
-
-The server should send `queueIds`, `error` and `ok` responses in the same order within each queue ID as the commands received in the transport connection, so that they can be correlated by the clients. To simplify correlation of commands and responses, the server should use the same `corrId` in the response as in the command sent by the client.
-
-If the transport connection is closed before some responses are sent, these responses should be discarded.
-
-### Command authentication
-
-SMP servers must authenticate all transmissions (excluding `ping` and `send` commands) by verifying the provided signatures. Command signature should be generated by applying RSA-PSS algorithm to the `signed` block of the transmission using the key associated with the queue ID (sender's or recipient's, depending on which queue ID is used).
-
-### Keep-alive command
-
-To keep the transport connection alive and to generate noise traffic the clients should use `ping` command to which the server responds with `pong` response. This command should be sent unsigned and without queue ID.
-
-```abnf
-ping = %s"PING"
-pong = %s"PONG"
-```
-
-### Recipient commands
-
-Sending any of the commands in this section (other than `create`, that is sent without queue ID) is only allowed with recipient's ID (`RID`). If sender's ID is used the server must respond with `"ERR AUTH"` response (see [Error responses](#error-responses)).
-
-#### Create queue command
-
-This command is sent by the recipient to the SMP server to create a new queue. The syntax is:
-
-```abnf
-create = %s"NEW" SP recipientKey
-recipientKey = %s"rsa:" x509encoded ;  the recipient's RSA public key for this queue
-x509encoded = <base64 X509 key encoding>
-```
-
-If the queue is created successfully, the server must send `queueIds` response with the recipient's and sender's queue IDs:
-
-```abnf
-queueIds = %s"IDS" SP recipientId SP senderId
-recipientId = encoded
-senderId = encoded
-```
-
-This response should be sent with empty queue ID (the second part of the transmission).
-
-Once the queue is created, the recipient gets automatically subscribed to receive the messages from that queue, until the transport connection is closed. The `subscribe` command is needed only to start receiving the messages from the existing queue when the new transport connection is opened.
-
-NEW `transmission` must be signed using the `recipientKey` that was passed in the transmission.
-
-#### Subscribe to queue
-
-When the simplex queue was not created in the current transport connection, the recipient must use this command to start receiving messages from it:
-
-```abnf
-subscribe = %s"SUB"
-```
-
-If subscription is successful the server must respond with the first available message or with `ok` response if no messages are available. The recipient will continue receiving the messages from this queue until the transport connection is closed or until another transport connection subscribes to the same simplex queue - in this case the first subscription should be cancelled and [subscription END notification](#subscription-end-notification) delivered.
-
-The first message will be delivered either immediately or as soon as it is available; to receive the following message the recipient must acknowledge the reception of the message (see [Acknowledge message delivery](#acknowledge-message-delivery)).
-
-#### Secure queue command
-
-This command is sent by the recipient to the server to add sender's key to the queue:
-
-```
-secure = %s"KEY" SP senderKey
-senderKey = %s"rsa:" x509encoded ; the sender's RSA public key for this queue
-```
-
-`senderKey` is received from the sender as part of the first message - see [Send Message Command](#send-message-command).
-
-Once the queue is secured only signed messages can be sent to it.
-
-#### Acknowledge message delivery
-
-The recipient should send the acknowledgement of message delivery once the message was stored in the client, to notify the server that the message should be deleted:
-
-```abnf
-acknowledge = %s"ACK"
-```
-
-Even if acknowledgement is not sent by the recipient, the server should limit the time of message storage, whether it was delivered to the recipient or not.
-
-Having received the acknowledgement, SMP server should immediately delete the message and then send the next available message or respond with `ok` if there are no more messages available in this simplex queue.
-
-#### Suspend queue
-
-The recipient can suspend a queue prior to deleting it to make sure that no messages are lost:
-
-```abnf
-suspend = %s"OFF"
-```
-
-The server must respond with `"ERR AUTH"` to any messages sent after the queue was suspended (see [Error responses](#error-responses)).
-
-The server must respond `ok` to this command if it was successful.
-
-This command can be sent multiple times (in case transport connection was interrupted and the response was not delivered), the server should still respond `ok` even if the queue is already suspended.
-
-There is no command to resume the queue. Servers must delete suspended queues that were not deleted after some period of time.
-
-#### Delete queue
-
-The recipient can delete the queue, whether it was suspended or not.
-
-All undelivered messages must be deleted as soon as this command is received, before the response is sent.
-
-```abnf
-delete = %s"DEL"
-```
-
-### Sender commands
-
-Currently SMP defines only one command that can be used by senders - `send` message. This command must be used with sender's ID, if recipient's ID is used the server must respond with `"ERR AUTH"` response (see [Error responses](#error-responses)).
-
-#### Send message
-
-This command is sent to the server by the sender both to confirm the queue after the sender received out-of-band message from the recipient and to send messages after the queue is secured:
-
-```abnf
-send = %s"SEND" SP size SP msgBody SP
-; the last SP is in addition to SP in the transmission
-size = 1*DIGIT ; size in bytes
-msgBody = *OCTET ; any binary content of specified size
-```
-
-The first message is sent to confirm the queue - it should contain sender's server key (see decrypted message syntax below) - this first message must be sent without signature.
-
-Once the queue is secured (see [Secure queue command](#secure-queue-command)), the following send commands must be sent with the signature.
-
-The server must respond with `"ERR AUTH"` response in the following cases:
-
-- the queue does not exist or is suspended
-- the queue is secured but the transmission does NOT have a signature
-- the queue is NOT secured but the transmission has a signature
-
-Until the queue is secured, the server should accept any number of unsigned messages - it both enables the legitimate sender to resend the confirmation in case of failure and also allows the simplex messaging client to ignore any confirmation messages that may be sent by the attackers (assuming they could have intercepted the queue ID in the server response, but do not have a correct encryption key passed to sender in out-of-band message).
-
-The body should be encrypted with the recipient's "public" key (`EK`); once decrypted it must have this format:
-
-```abnf
-decryptedBody = [clientHeader] CRLF clientBody CRLF
-clientHeader = senderKeyMsg
-senderKeyMsg = %s"KEY" SP senderKey
-senderKey = %s"rsa:" x509encoded ; the sender's RSA public key for this queue
-clientBody = *OCTET
-```
-
-`clientHeader` in the initial unsigned message is used to transmit sender's server key and can be used in the future revisions of SMP protocol for other purposes.
-
-### Server messages
-
-#### Queue IDs response
-
-Server must respond with this message when the new queue is created.
-
-See its syntax in [Create queue command](#create-queue-command)
-
-#### Deliver queue message
-
-The server must deliver messages to all subscribed simplex queues on the currently open transport connection. The syntax for the message delivery is:
-
-```abnf
-message = %s"MSG" SP msgId SP timestamp SP size SP msgBody SP
-msgId = encoded
-timestamp = <date-time defined in RFC3339>
-```
-
-`msgId` - unique message ID generated by the server based on cryptographically strong random bytes. It should be used by the clients to detect messages that were delivered more than once (in case the transport connection was interrupted and the server did not receive the message delivery acknowledgement).
-
-`timestamp` - the UTC time when the server received the message from the sender, must be in date-time format defined by [RFC 3339][10]
-
-`binaryMsg` - see syntax in [Send message](#send-message)
-
-#### Subscription END notification
-
-When another transport connection is subscribed to the same simplex queue, the server should unsubscribe and to send the notification to the previously subscribed transport connection:
-
-```abnf
-unsubscribed = %s"END"
-```
-
-No further messages should be delivered to unsubscribed transport connection.
-
-#### Error responses
-
-- incorrect block format, encoding or signature size (`BLOCK`).
-- command errors (`CMD`):
-  - error parsing command (`SYNTAX`)
-  - prohibited command (`PROHIBITED`) - any server response sent from client or `ACK` sent without active subscription or without message delivery.
-  - incorrect RSA key size in `NEW` or `KEY` commands - only 1024, 2048 and 4096-bit keys are allowed (`KEY_SIZE`).
-  - transmission has no required signature or queue ID (`NO_AUTH`)
-  - transmission has unexpected credentials (`HAS_AUTH`)
-  - transmission has no required queue ID (`NO_QUEUE`)
-- authentication error (`AUTH`) - incorrect signature, unknown (or suspended) queue, sender's ID is used in place of recipient's and vice versa, and some other cases (see [Send message command](#send-message-command)).
-- incorrect message body size (`SIZE`).
-- internal server error (`INTERNAL`).
-
-The syntax for error responses:
-
-```abnf
-error = %s"ERR" SP errorType
-errorType = %s"BLOCK" / %s"CMD" SP cmdError / %s"AUTH" / %s"SIZE" /%s"INTERNAL"
-cmdError = %s"SYNTAX" / %s"PROHIBITED" / %s"KEY_SIZE" / %s"NO_AUTH" / %s"HAS_AUTH" / %s"NO_QUEUE"
-```
-
-Server implementations must aim to respond within the same time for each command in all cases when `"ERR AUTH"` response is required to prevent timing attacks (e.g., the server should perform signature verification even when the queue does not exist on the server or the signature of different size is sent, using any RSA key with the same size as the signature size).
-
-### OK response
-
-When the command is successfully executed by the server, it should respond with OK response:
-
-```abnf
-ok = %s"OK"
-```
-
-## Appendices
-
-### Appendix A.
-
-**SMP transport protocol.**
-
-Both the recipient and the sender can use TCP or some other, possibly higher level, transport protocol to communicate with the server. The default TCP port for SMP server is 5223.
-
-By default, the client and server should use the protocol presented below, that does not depend on a centralized certificate authority.
-
-Transport is encrypted with [AEAD-GCM][12] protocol with two random symmetric AES 256-bit keys and two random base IVs that will be agreed during the handshake. Both client and the server should maintain two 32-bit word counters, one for the sent and one for the received messages. The IV for each message should be computed by xor-ing the sequential message counter, starting from 0, with the first 32 bits of agreed base IV (the number is encoded in network byte order).
-
-To establish the session keys and base IVs, the server should have an asymmetric key pair generated during server deployment and unknown to the clients. The users should know the key hash (256 bits) in advance in order to be able to validate the server public key during transport connection handshake.
-
-The handshake sequence is the following:
-
-1. Once the connection is established, the server sends the `server_header` followed by its public RSA key encoded in X509 binary (not base-64 encoded) format to the client.
-2. The client compares the SHA256 hash of the received key with the hash it already has (e.g. received as part of connection invitation or as SMP server configuration). If the hash does not match, the client must terminate the connection.
-3. If the hash is the same, the client should generate two random symmetric 256-bit AES keys and two base IVs that will be used as session keys/IVs by the client and the server.
-4. The client then should create the `client_handshake` block and send it to the server, encrypted using [RSA-OAEP][2] scheme with the server public key: `rsa-encrypt(client_handshake)`. `snd_aes_key` and `snd_base_iv` will be used by the client to encrypt **sent** messages and by the server to decrypt them, `rcv_aes_key` and `rcv_base_iv` will be used by the client to decrypt **received** messages and by the server to encrypt them. `client_handshake` also contains `block_size` and reserved `protocol` blocks (see syntax).
-5. The server should decrypt the received AES keys and base IVs with its private RSA key.
-6. In case of successful decryption, the server should send encrypted welcome block (`encrypted_welcome_block`) that contains SMP protocol version supported by the server.
-
-All the subsequent data, both from the client and from the server, should be sent padded to the fixed agreed block size, encrypted with symmetric AES keys and base IVs (incremented by counters on both sides), that were sent by the client during the handshake. If the application needs to transmit a larger message, it should be broken down into fragments.
-
-Handshake blocks sent by the client and the server have this syntax:
-
-```abnf
-server_header = block_size protocol key_size
-block_size = 4*4(OCTET) ; 4-byte block size sent by the server
-protocol = 2*2(%x00) ; 0, reserved
-key_size = 2*2(OCTET) ; the size of the encoded key in bytes (binary encoded in X509 standard)
-
-client_handshake = client_block_size protocol snd_aes_key snd_base_iv rcv_aes_key rcv_base_iv
-client_block_size = 4*4(OCTET) ; 4-byte block size sent by the client,
-; to confirm or override the block size sent by the server
-client_protocol = 2*2(%x00) ; 0, reserved
-snd_aes_key = 32*32(OCTET)
-snd_base_iv = 16*16(OCTET)
-rcv_aes_key = 32*32(OCTET)
-rcv_base_iv = 16*16(OCTET)
-
-transport_block = aes_body_auth_tag aes_encrypted_body
-; size is sent by server during handshake, usually 4096 bytes
-aes_body_auth_tag = 16*16(OCTET)
-aes_encrypted_body = 1*OCTET
-
-encrypted_welcome_block = transport_block
-welcome_block = smp_version SP pad ; decrypt(encrypted_welcome_block)
-smp_version = %s"v" 1*DIGIT "." 1*DIGIT "." 1*DIGIT ["-" 1*ALPHA "." 1*DIGIT] ; in semver format
-  ; for example: v123.456.789-alpha.7
-pad = 1*OCTET
-```
-
-[1]: https://en.wikipedia.org/wiki/Man-in-the-middle_attack
-[2]: https://en.wikipedia.org/wiki/End-to-end_encryption
-[3]: https://en.wikipedia.org/wiki/QR_code
-[4]: https://en.wikipedia.org/wiki/Unidirectional_network
-[5]: https://en.wikipedia.org/wiki/Forward_secrecy
-[6]: https://en.wikipedia.org/wiki/Off-the-Record_Messaging
-[8]: https://tools.ietf.org/html/rfc5234
-[8a]: https://tools.ietf.org/html/rfc7405
-[9]: https://tools.ietf.org/html/rfc4648#section-4
-[10]: https://tools.ietf.org/html/rfc3339
-[11]: https://tools.ietf.org/html/rfc5280
-[12]: https://tools.ietf.org/html/rfc7714

simplex.md

@@ -2,230 +2,83 @@
 
 ## Problems
 
-Existing chat platforms and protocols have some or all of the following
-problems:
+Existing chat platforms and protocols have some or all of the following problems:
 
-- lack of privacy of the conversation, partially caused by [E2EE][1]
-  implementations.
-- lack of privacy of the user profile and connections.
-- unsolicited messages (spam and abuse).
-- lack of data ownership and protection.
-- complexity of usage for all non-centralised protocols to all non-technical
-  users
+- Lack of privacy of the conversation, partially caused by [E2EE][1] implementations.
+- Lack of privacy of the user profile and connections.
+- Unsolicited messages (spam and abuse).
+- Lack of data ownership and protection.
+- Complexity of usage for all non-centralized protocols to non-technical users.
 
-Some of these problems are covered in more details in the proposed protocols on
-which this chat system will be based on:
+The concentration of the communication in a small number of centralized platforms makes resolving these problems quite difficult.
 
-- [simplex messaging][6] - low level client-server protocol for asynchronous
-  distributed unidirectional messaging.
-- [graph-chat][8] - high level chat protocol for client applications that
-  communicate via simplex messaging protocol.
+## Proposed solution
 
-Even though EU-wide GDPR legislation to ensure users' privacy and data
-protection was adopted, the centralisation of the communication in a small
-number of platforms makes resolving these problems quite difficult.
+Proposed stack of protocols solves these and other problems by making both messages and contacts accessible only on client devices, reducing the role of the servers to simple message brokers that only require authorization of messages sent to the queues, but do NOT require user authentication - not only the messages but also the metadata is protected.
 
-# Comparison with [P2P][9] messaging protocols
+See [SMP protocol][6] and [SMP agent protocol][8].
 
-There are several P2P chat/messaging protocols and implementations that aim to
-solve privacy and centralisation problem, but they have their own set of
-problems that makes them less reliable than the proposed chat system design,
-more complex to implement and analyse and more vulnerable to attacks.
+## Comparison with other protocols
 
-1. [P2P][9] networks either have some centralised component, which makes them
-   highly vulnerable, or, more commonly, use some variant of [DHT][10] to route
-   messages/requests through the network. DHT implementations have complex
-   designs that have to balance reliability, delivery guarantee and latency, and
-   also have some other problems. The proposed chat system design has both
-   higher delivery guarantee and low latency (the message is passed multiple
-   times in parallel, through one node each time, using servers chosen by the
-   recipient, while in P2P networks the message is passed through `O(log N)`
-   nodes sequentially, using nodes chosen by the algorithm).
-2. The proposed design, unlike most P2P networks, has no global identity of any
-   form, even temporary.
-3. P2P itself does not solve [MITM attack][2] problem, but most existing
-   solutions do not use out-of-band messages for the initial key exchange. The
-   proposed design uses out-of-band messages or, in some cases, pre-existing
-   secure and trusted connections for the initial key exchange.
-4. P2P implementations can be blocked by some Internet providers (like
-   [BitTorrent][11]). The proposed design is transport agnostic - it can work
-   over standard web protocols, and the servers can be deployed on the same
-   domains as the existing public websites.
-5. All known P2P networks are likely to be vulnerable to [Sybil attack][12],
-   because each node is discoverable, and the network operates as a whole. Known
-   measures to reduce the probability of the Sybil attack either require a
-   vulnerable centralised component or expensive [proof of work][13]. The
-   proposed design, on the opposite, has no server discoverability - servers are
-   not connected, not known to each other and to all clients. The chat network
-   is fragmented and operates as multiple isolated connections. It makes Sybil
-   attack on the whole simplex messaging network impossible - even if some
-   servers are compromised, other parts of the network can operate normally, and
-   affected clients can always switch to using other servers without losing
-   contacts or messages.
-6. P2P networks are likely to be vulnerable to [DRDoS attack][14]. In the
-   proposed design clients only relay traffic from known trusted connection and
-   cannot be used to reflect and amplify the traffic in the whole network.
+|          | SimpleX chat | Signal, big platforms | XMPP, Matrix | P2P protocols |
+|:-------- |:------------:|:---------------------:|:------------:|:-------------:|
+| Requires global identity | No = private | Yes<sup>1</sup> | Yes<sup>2</sup> | Yes<sup>3</sup> |
+| Possibility of MITM      | No = secure  | Yes<sup>4</sup> | Yes             | Yes             |
+| Dependence on DNS        | No = resilient | Yes | Yes | No |
+| Federation               | Yes | No | Yes | No<sup>5</sup> |
+| Central component or other network-wide attack | No = resilient | Yes | Yes<sup>2</sup> | Yes<sup>6</sup> |
 
-## Privacy requirements
+1. Usually based on a phone number, in some cases on usernames.
+2. DNS based.
+3. Public key or some other globally unique ID.
+4. If operator’s servers are compromised.
+5. While P2P networks are distributed, they are not federated - they operate as a single network.
+6. P2P networks either have a central authority or the whole network can be compromised - see the next section.
 
-- User profile is only visible to the profile connections, but not to the chat
-  network
-- User profile is not stored on the servers.
-- Profile connections are not stored on the server.
-- It should not be possible to construct the list of user connections by
-  analysing server database or any logs.
-- It should not be possible, other than by compromising the client, to send
-  messages from another user profile.
-- It should not be possible, other than by compromising all servers the user is
-  connected to, to prevent message delivery to a user.
-- All participants of the conversation should be able to:
-  - prove that the received messages were actually sent by another party.
-  - prove that the sent messages were received by another party.
+## Comparison with [P2P][9] messaging protocols
 
-## Chat scenarios
+There are several P2P chat/messaging protocols and implementations that aim to solve privacy and centralisation problem, but they have their own set of problems that makes them less reliable than the proposed chat system design, more complex to implement and analyse and more vulnerable to attacks.
 
-Any advanced chat system needs to support several main chat scenarios:
+1. [P2P][9] networks either have some centralized component, which makes them highly vulnerable, or, more commonly, use some variant of [DHT][10] to route messages/requests through the network. DHT implementations have complex designs that have to balance reliability, delivery guarantee and latency, and also have some other problems. The proposed chat system design has both higher delivery guarantee and low latency (the message is passed multiple times in parallel, through one node each time, using servers chosen by the recipient, while in P2P networks the message is passed through `O(log N)` nodes sequentially, using nodes chosen by the algorithm).
 
-- direct messaging.
-- group chat.
-- broadcasts.
+2. The proposed design, unlike most P2P networks, has no global identity of any form, even temporary.
 
-In addition to that, there are other important chat scenarios:
+3. P2P itself does not solve [MITM attack][2] problem, but most existing solutions do not use out-of-band messages for the initial key exchange. The proposed design uses out-of-band messages or, in some cases, pre-existing secure and trusted connections for the initial key exchange.
 
-- [OTR messaging][3]
-- multiple user devices sharing contacts and conversation histories.
-- introductions
+4. P2P implementations can be blocked by some Internet providers (like [BitTorrent][11]). The proposed design is transport agnostic - it can work over standard web protocols, and the servers can be deployed on the same domains as the websites.
 
-While it is not required to be supported in the v1 of the protocol, it is
-important to have clarity on how all these scenarios can be supported in the
-future.
+5. All known P2P networks are likely to be vulnerable to [Sybil attack][12], because each node is discoverable, and the network operates as a whole. Known measures to reduce the probability of the Sybil attack either require a vulnerable centralized component or expensive [proof of work][13]. The proposed design, on the opposite, has no server discoverability - servers are not connected, not known to each other and to all clients. The chat network is fragmented and operates as multiple isolated connections. It makes Sybil attack on the whole simplex messaging network impossible - even if some servers are compromised, other parts of the network can operate normally, and affected clients can always switch to using other servers without losing contacts or messages.
 
-## Chat system features
+6. P2P networks are likely to be vulnerable to [DRDoS attack][14]. In the proposed design clients only relay traffic from known trusted connection and cannot be used to reflect and amplify the traffic in the whole network.
 
-- No user identity known to system servers - no phone numbers, user names and no
-  DNS are needed to identify the users to the system.
-- Each user can be connected to multiple servers to ensure message delivery,
-  even if some of the servers are compromised.
-- Uses standard asymmetric cryptographic protocols, so that system users can
-  create independent server and client implementations complying with the
-  protocols.
-- Open-source server implementations that can be easily deployed by any user
-  with minimal technical expertise (e.g. on Heroku via web UI).
-- Open-source client implementations so that system users can independently
-  assess system security model.
-- Only client applications store user profiles, contacts of other user profiles,
-  messages; servers do NOT have access to any of this information and (unless
-  compromised) do NOT store encrypted messages or any logs.
-- Multiple client applications and devices can be used by each user profile to
-  communicate and to share connections and message history - the devices are not
-  known to the servers.
-- Initial key exchange and establishing connections between user profiles is
-  done by sharing QR code via any independent communication channel (or directly
-  via screen and camera), system servers are NOT used for key exchange - to
-  reduce risk of key substitution in [MITM attack][2]. QR code contains the
-  connection-specific public key and other information needed to establish the
-  connection.
-- Connections between users can be established via shared trusted connections to
-  simplify key exchange.
-- Servers do NOT communicate with each other, they only communicate with client
-  applications.
+## Network features
+
+- No user identity known to system servers - no phone numbers, user names and no DNS are needed to authorize users to the network.
+- Each user can be connected to multiple servers to ensure message delivery, even if some of the servers are compromised.
+- No single server in the system has visibility of all connections or messages of any user, as user profiles are identified by multiple rotating public keys, using separate key for each profile connection.
+- Uses standard asymmetric cryptographic protocols, so that system users can create independent server and client implementations complying with the protocols.
+- Open-source server implementations that can be easily deployed by any user with minimal technical expertise (e.g. on Heroku via web UI).
+- Open-source client implementations so that system users can independently assess system security model.
+- Only client applications store user profiles, contacts of other user profiles, messages; servers do NOT have access to any of this information and (unless compromised) do NOT store encrypted messages or any logs.
+- Multiple client applications and devices can be used by each user profile to communicate and to share connections and message history - the devices are not known to the servers.
+- Initial key exchange and establishing connections between user profiles is done by sharing the invitation (e.g. QR code via any independent communication channel (or directly via screen and camera), system servers are NOT used for key exchange - to reduce risk of key substitution in [MITM attack][2]. QR code contains the connection-specific public key and other information needed to establish the connection.
+- Connections between users can be established via shared trusted connections to simplify key exchange.
+- Servers do NOT communicate with each other, they only communicate with client applications.
 - Unique public key is used for each user profile connection in order to:
-  - reduce the risk of attacker posing as user's connection
-  - avoid exposing all user connections to the servers
-- Unique public key is used to identify each connection participant to each
-  server.
-- Public keys used between connections are regularly rotated to prevent
-  decryption of the full message history ([forward secrecy][4]) in case when
-  some servers or middlemen preserve message history and the current key is
-  compromised.
-- Users can repeat key exchange using QR code and alternative channel at any
-  point to increase communication security and trust.
-- No single server in the system has visibility of all connections or messages
-  of any user, as user profiles are identified by multiple rotating public keys,
-  using separate key for each profile connection.
-- User profile (meta-data of the user including non-unique name / handle and
-  optional additional data, e.g. avatar and status) is stored in the client apps
-  and is shared only with accepted user profile connections.
-
-## System components
-
-- simplex messaging servers
-- graph-chat client applications (using simplex messaging protocol to
-  communicate with the servers)
-
-### Chat servers
-
-Simplex messaging servers can be either available to all users or only to users
-who have a valid URI to create connections (see [simplex messaging
-protocol][6]).
-
-### Chat client application
-
-Graph-chat applications are installed by users on their devices.
-
-Client apps should provide the following features:
-
-- create and manage user profiles
-- support multiple user profiles.
-- share access to all or selected user profiles with other devices (optionally
-  including existing connections and message histories).
-- for each user profile:
-  - generate and show QR code with connection-specific public key and other
-    information that can be shown on the screen and/or sent via alternative
-    channel.
-  - read QR code (via the camera) to establish connection with another user.
-  - receive and accept connection requests.
-  - exchange user profiles once connection is accepted.
-  - send messages to connected user profiles.
-  - receive messages from connected user profile.
-  - define the servers to use.
-  - store history of all conversations encrypted using user client app key with
-    passphrase (or some other device specific encryption method).
-
-## System design
-
-The chat system design is based on 2 protocols, each with the generic part,
-describing protocol flow and logic, and implementation part, describing protocol
-transports, data structures and algorithms.
-
-1. [simplex messaging protocol][6] - a low level messaging protocol that defines
-   establishing and using a simplex connection between chat participants on a
-   single server. While this protocol is designed to support graph-chat client
-   protocol (below), it can be used for other messaging scenarios, not limited
-   to chats.
-2. [graph-chat protocol][8] - a high level generic chat protocol for client
-   applications (graph vertices) that communicate via simplex connections
-   created using simplex messaging protocol. This protocol defines connection
-   and message types and semantics for:
-   - various chat elements (user profiles, direct chats, chat groups,
-     broadcasts, etc.).
-   - other communication scenarios - e.g. introduction, off-the-record chat,
-     etc.
-   - using multiple servers to ensure message delivery.
-   - sharing user profiles, contacts and chats across multiple client devices.
-   - changing cryptographic keys and servers used to send and receive messages
-     using simplex messaging server protocol.
-   - sending and receiving out-of-band messages between client applications
-     using "visual code".
-3. graph-chat client application protocol (TODO) - a high level specific chat
-   protocol for client applications. This protocol specifies:
-   - data structures for sending and receiving messages of all types.
-   - process to send and receive out-of-band messages between client
-     applications.
-   - other requirements for graph-chat client applications.
-   - defines a specific "visual code" format to send an out-of-band message.
+  - reduce the risk of attacker posing as user's connection;
+  - avoid exposing all user connections to the servers.
+- Unique public key is used to identify each connection participant to each server.
+- Public keys used between connections are regularly rotated to prevent decryption of the full message history ([forward secrecy][4]) in case when some servers or middlemen preserve message history and the current key is compromised.
+- Users can repeat key exchange using QR code and alternative channel at any point to increase communication security and trust.
 
 [1]: https://en.wikipedia.org/wiki/End-to-end_encryption
 [2]: https://en.wikipedia.org/wiki/Man-in-the-middle_attack
-[3]: https://en.wikipedia.org/wiki/Off-the-Record_Messaging
 [4]: https://en.wikipedia.org/wiki/Forward_secrecy
-[5]: https://mermaid-js.github.io/mermaid-live-editor
-[6]: simplex-messaging.md
-[8]: graph-chat.md
+[6]: https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md
+[8]: https://github.com/simplex-chat/simplexmq/blob/master/protocol/agent-protocol.md
 [9]: https://en.wikipedia.org/wiki/Peer-to-peer
 [10]: https://en.wikipedia.org/wiki/Distributed_hash_table
 [11]: https://en.wikipedia.org/wiki/BitTorrent
 [12]: https://en.wikipedia.org/wiki/Sybil_attack
 [13]: https://en.wikipedia.org/wiki/Proof_of_work
-[14]:
-  https://www.usenix.org/conference/woot15/workshop-program/presentation/p2p-file-sharing-hell-exploiting-bittorrent
+[14]: https://www.usenix.org/conference/woot15/workshop-program/presentation/p2p-file-sharing-hell-exploiting-bittorrent

src/Simplex/Chat/Controller.hs

@@ -0,0 +1,87 @@
+{-# LANGUAGE ConstraintKinds #-}
+{-# LANGUAGE DeriveAnyClass #-}
+{-# LANGUAGE DuplicateRecordFields #-}
+{-# LANGUAGE FlexibleContexts #-}
+{-# LANGUAGE OverloadedStrings #-}
+
+module Simplex.Chat.Controller where
+
+import Control.Exception
+import Control.Monad.Except
+import Control.Monad.IO.Unlift
+import Control.Monad.Reader
+import Crypto.Random (ChaChaDRG)
+import Data.Int (Int64)
+import Data.Map.Strict (Map)
+import Numeric.Natural
+import Simplex.Chat.Notification
+import Simplex.Chat.Store (StoreError)
+import Simplex.Chat.Terminal
+import Simplex.Chat.Types
+import Simplex.Messaging.Agent (AgentClient)
+import Simplex.Messaging.Agent.Env.SQLite (AgentConfig)
+import Simplex.Messaging.Agent.Protocol (AgentErrorType)
+import Simplex.Messaging.Agent.Store.SQLite (SQLiteStore)
+import System.IO (Handle)
+import UnliftIO.STM
+
+data ChatConfig = ChatConfig
+  { agentConfig :: AgentConfig,
+    dbPoolSize :: Int,
+    tbqSize :: Natural,
+    fileChunkSize :: Integer
+  }
+
+data ChatController = ChatController
+  { currentUser :: TVar User,
+    smpAgent :: AgentClient,
+    chatTerminal :: ChatTerminal,
+    chatStore :: SQLiteStore,
+    idsDrg :: TVar ChaChaDRG,
+    inputQ :: TBQueue InputEvent,
+    notifyQ :: TBQueue Notification,
+    sendNotification :: Notification -> IO (),
+    chatLock :: TMVar (),
+    sndFiles :: TVar (Map Int64 Handle),
+    rcvFiles :: TVar (Map Int64 Handle),
+    config :: ChatConfig
+  }
+
+data InputEvent = InputCommand String | InputControl Char
+
+data ChatError
+  = ChatError ChatErrorType
+  | ChatErrorMessage String
+  | ChatErrorAgent AgentErrorType
+  | ChatErrorStore StoreError
+  deriving (Show, Exception)
+
+data ChatErrorType
+  = CEGroupUserRole
+  | CEGroupContactRole ContactName
+  | CEGroupDuplicateMember ContactName
+  | CEGroupDuplicateMemberId
+  | CEGroupNotJoined GroupName
+  | CEGroupMemberNotActive
+  | CEGroupMemberUserRemoved
+  | CEGroupMemberNotFound ContactName
+  | CEGroupInternal String
+  | CEFileNotFound String
+  | CEFileAlreadyReceiving String
+  | CEFileAlreadyExists FilePath
+  | CEFileRead FilePath SomeException
+  | CEFileWrite FilePath SomeException
+  | CEFileSend Int64 AgentErrorType
+  | CEFileRcvChunk String
+  | CEFileInternal String
+  deriving (Show, Exception)
+
+type ChatMonad m = (MonadUnliftIO m, MonadReader ChatController m, MonadError ChatError m)
+
+setActive :: (MonadUnliftIO m, MonadReader ChatController m) => ActiveTo -> m ()
+setActive to = asks (activeTo . chatTerminal) >>= atomically . (`writeTVar` to)
+
+unsetActive :: (MonadUnliftIO m, MonadReader ChatController m) => ActiveTo -> m ()
+unsetActive a = asks (activeTo . chatTerminal) >>= atomically . (`modifyTVar` unset)
+  where
+    unset a' = if a == a' then ActiveNone else a'

src/Simplex/Chat/Help.hs

@@ -0,0 +1,109 @@
+{-# LANGUAGE OverloadedStrings #-}
+
+module Simplex.Chat.Help
+  ( chatHelpInfo,
+    filesHelpInfo,
+    groupsHelpInfo,
+    markdownInfo,
+  )
+where
+
+import Data.List (intersperse)
+import Data.Text (Text)
+import Simplex.Chat.Markdown
+import Simplex.Chat.Styled
+import System.Console.ANSI.Types
+
+highlight :: Text -> Markdown
+highlight = Markdown (Colored Cyan)
+
+green :: Text -> Markdown
+green = Markdown (Colored Green)
+
+indent :: Markdown
+indent = "        "
+
+listHighlight :: [Text] -> Markdown
+listHighlight = mconcat . intersperse ", " . map highlight
+
+chatHelpInfo :: [StyledString]
+chatHelpInfo =
+  map
+    styleMarkdown
+    [ highlight "Using SimpleX chat prototype",
+      "Follow these steps to set up a connection:",
+      "",
+      green "Step 1: " <> highlight "/connect" <> " - Alice adds a contact.",
+      indent <> "Alice should send the invitation printed by the /add command",
+      indent <> "to her contact, Bob, out-of-band, via any trusted channel.",
+      "",
+      green "Step 2: " <> highlight "/connect <invitation>" <> " - Bob accepts the invitation.",
+      indent <> "Bob should use the invitation he received out-of-band.",
+      "",
+      green "Step 3: " <> "Bob and Alice are notified that the connection is set up,",
+      indent <> "both can now send messages:",
+      indent <> highlight "@bob Hello, Bob!" <> " - Alice messages Bob (assuming Bob has display name 'bob').",
+      indent <> highlight "@alice Hey, Alice!" <> " - Bob replies to Alice.",
+      "",
+      green "To send file:",
+      indent <> highlight "/file bob ./photo.jpg" <> " - Alice sends file to Bob",
+      indent <> "File commands: " <> highlight "/help files",
+      "",
+      green "To create group:",
+      indent <> highlight "/group team" <> " - create group #team",
+      indent <> "Group commands: " <> highlight "/help groups",
+      "",
+      green "Other commands:",
+      indent <> highlight "/profile         " <> " - show user profile",
+      indent <> highlight "/profile <name> [<full_name>]" <> " - update user profile",
+      indent <> highlight "/delete <contact>" <> " - delete contact and all messages with them",
+      indent <> highlight "/markdown        " <> " - show supported markdown syntax",
+      indent <> highlight "/quit            " <> " - quit chat",
+      "",
+      "The commands may be abbreviated to a single letter: " <> listHighlight ["/c", "/f", "/g", "/p", "/h"] <> ", etc."
+    ]
+
+filesHelpInfo :: [StyledString]
+filesHelpInfo =
+  map
+    styleMarkdown
+    [ green "File transfer commands:",
+      indent <> highlight "/file @<contact> <file_path>     " <> " - send file to contact",
+      indent <> highlight "/file #<group> <file_path>       " <> " - send file to group",
+      indent <> highlight "/freceive <file_id> [<file_path>]" <> " - accept to receive file",
+      indent <> highlight "/fcancel <file_id>               " <> " - cancel sending / receiving file",
+      indent <> highlight "/fstatus <file_id>               " <> " - show file transfer status",
+      "",
+      "The commands may be abbreviated: " <> listHighlight ["/f", "/fr", "/fc", "/fs"]
+    ]
+
+groupsHelpInfo :: [StyledString]
+groupsHelpInfo =
+  map
+    styleMarkdown
+    [ green "Group management commands:",
+      indent <> highlight "/group <group> [<full_name>]   " <> " - create group",
+      indent <> highlight "/add <group> <contact> [<role>]" <> " - add contact to group, roles: " <> highlight "owner" <> ", " <> highlight "admin" <> " (default), " <> highlight "member",
+      indent <> highlight "/join <group>                  " <> " - accept group invitation",
+      indent <> highlight "/remove <group> <member>       " <> " - remove member from group",
+      indent <> highlight "/leave <group>                 " <> " - leave group",
+      indent <> highlight "/delete <group>                " <> " - delete group",
+      indent <> highlight "/members <group>               " <> " - list group members",
+      indent <> highlight "#<group> <message>             " <> " - send message to group",
+      "",
+      "The commands may be abbreviated: " <> listHighlight ["/g", "/a", "/j", "/rm", "/l", "/d", "/ms"]
+    ]
+
+markdownInfo :: [StyledString]
+markdownInfo =
+  map
+    styleMarkdown
+    [ green "Markdown:",
+      indent <> highlight "*bold*         " <> " - " <> Markdown Bold "bold text",
+      indent <> highlight "_italic_       " <> " - " <> Markdown Italic "italic text" <> " (shown as underlined)",
+      indent <> highlight "+underlined+   " <> " - " <> Markdown Underline "underlined text",
+      indent <> highlight "~strikethrough~" <> " - " <> Markdown StrikeThrough "strikethrough text" <> " (shown as inverse)",
+      indent <> highlight "`code snippet` " <> " - " <> Markdown Snippet "a + b // no *markdown* here",
+      indent <> highlight "!1 text!       " <> " - " <> Markdown (Colored Red) "red text" <> " (1-6: red, green, blue, yellow, cyan, magenta)",
+      indent <> highlight "#secret#       " <> " - " <> Markdown Secret "secret text" <> " (can be copy-pasted)"
+    ]

src/Simplex/Chat/Input.hs

@@ -1,50 +1,53 @@
+{-# LANGUAGE FlexibleContexts #-}
 {-# LANGUAGE LambdaCase #-}
 {-# LANGUAGE NamedFieldPuns #-}
-{-# LANGUAGE OverloadedStrings #-}
 
-module ChatTerminal.Core where
+module Simplex.Chat.Input where
 
-import Control.Concurrent.STM
-import Data.ByteString.Char8 (ByteString)
-import qualified Data.ByteString.Char8 as B
+import Control.Monad.IO.Unlift
+import Control.Monad.Reader
 import Data.List (dropWhileEnd)
-import Data.Text (Text)
 import qualified Data.Text as T
-import Data.Text.Encoding
-import Simplex.Markdown
-import Styled
-import System.Console.ANSI.Types
+import Simplex.Chat.Controller
+import Simplex.Chat.Terminal
+import System.Exit (exitSuccess)
 import System.Terminal hiding (insertChars)
-import Types
+import UnliftIO.STM
 
-data ChatTerminal = ChatTerminal
-  { inputQ :: TBQueue String,
-    outputQ :: TBQueue [StyledString],
-    activeContact :: TVar (Maybe Contact),
-    termMode :: TermMode,
-    termState :: TVar TerminalState,
-    termSize :: Size,
-    nextMessageRow :: TVar Int,
-    termLock :: TMVar ()
-  }
+getKey :: MonadTerminal m => m (Key, Modifiers)
+getKey =
+  flush >> awaitEvent >>= \case
+    Left Interrupt -> liftIO exitSuccess
+    Right (KeyEvent key ms) -> pure (key, ms)
+    _ -> getKey
 
-data TerminalState = TerminalState
-  { inputPrompt :: String,
-    inputString :: String,
-    inputPosition :: Int,
-    previousInput :: String
-  }
+runTerminalInput :: (MonadUnliftIO m, MonadReader ChatController m) => m ()
+runTerminalInput = do
+  ChatController {inputQ, chatTerminal = ct} <- ask
+  liftIO $
+    withChatTerm ct $ do
+      updateInput ct
+      receiveFromTTY inputQ ct
 
-inputHeight :: TerminalState -> ChatTerminal -> Int
-inputHeight ts ct = length (inputPrompt ts <> inputString ts) `div` width (termSize ct) + 1
+receiveFromTTY :: MonadTerminal m => TBQueue InputEvent -> ChatTerminal -> m ()
+receiveFromTTY inputQ ct@ChatTerminal {activeTo, termSize, termState} =
+  forever $ getKey >>= processKey >> withTermLock ct (updateInput ct)
+  where
+    processKey :: MonadTerminal m => (Key, Modifiers) -> m ()
+    processKey = \case
+      (EnterKey, _) -> submitInput
+      key -> atomically $ do
+        ac <- readTVar activeTo
+        modifyTVar termState $ updateTermState ac (width termSize) key
 
-positionRowColumn :: Int -> Int -> Position
-positionRowColumn wid pos =
-  let row = pos `div` wid
-      col = pos - row * wid
-   in Position {row, col}
+    submitInput :: MonadTerminal m => m ()
+    submitInput = atomically $ do
+      ts <- readTVar termState
+      let s = inputString ts
+      writeTVar termState $ ts {inputString = "", inputPosition = 0, previousInput = s}
+      writeTBQueue inputQ $ InputCommand s
 
-updateTermState :: Maybe Contact -> Int -> (Key, Modifiers) -> TerminalState -> TerminalState
+updateTermState :: ActiveTo -> Int -> (Key, Modifiers) -> TerminalState -> TerminalState
 updateTermState ac tw (key, ms) ts@TerminalState {inputString = s, inputPosition = p} = case key of
   CharKey c
     | ms == mempty || ms == shiftKey -> insertCharsWithContact [c]
@@ -69,15 +72,16 @@ updateTermState ac tw (key, ms) ts@TerminalState {inputString = s, inputPosition
   _ -> ts
   where
     insertCharsWithContact cs
-      | null s && cs /= "@" && cs /= "/" =
+      | null s && cs /= "@" && cs /= "#" && cs /= "/" =
         insertChars $ contactPrefix <> cs
       | otherwise = insertChars cs
     insertChars = ts' . if p >= length s then append else insert
     append cs = let s' = s <> cs in (s', length s')
     insert cs = let (b, a) = splitAt p s in (b <> cs <> a, p + length cs)
     contactPrefix = case ac of
-      Just (Contact c) -> "@" <> B.unpack c <> " "
-      Nothing -> ""
+      ActiveNone -> ""
+      ActiveC c -> "@" <> T.unpack c <> " "
+      ActiveG g -> "#" <> T.unpack g <> " "
     backDeleteChar
       | p == 0 || null s = ts
       | p >= length s = ts' (init s, length s - 1)
@@ -112,23 +116,3 @@ updateTermState ac tw (key, ms) ts@TerminalState {inputString = s, inputPosition
             afterWord = dropWhile (/= ' ') $ dropWhile (== ' ') after
          in min (length s) $ p + length after - length afterWord
     ts' (s', p') = ts {inputString = s', inputPosition = p'}
-
-styleMessage :: String -> StyledString
-styleMessage = \case
-  "" -> ""
-  s@('@' : _) -> let (c, rest) = span (/= ' ') s in styled (Colored Cyan) c <> markdown rest
-  s -> markdown s
-  where
-    markdown :: String -> StyledString
-    markdown = styleMarkdownText . T.pack
-
-safeDecodeUtf8 :: ByteString -> Text
-safeDecodeUtf8 = decodeUtf8With onError
-  where
-    onError _ _ = Just '?'
-
-ttyContact :: Contact -> StyledString
-ttyContact (Contact a) = styled (Colored Green) a
-
-ttyFromContact :: Contact -> StyledString
-ttyFromContact (Contact a) = styled (Colored Yellow) $ a <> "> "

src/Simplex/Chat/Markdown.hs

@@ -0,0 +1,138 @@
+{-# LANGUAGE LambdaCase #-}
+{-# LANGUAGE OverloadedStrings #-}
+
+module Simplex.Chat.Markdown where
+
+import Control.Applicative ((<|>))
+import Data.Attoparsec.Text (Parser)
+import qualified Data.Attoparsec.Text as A
+import Data.Either (fromRight)
+import Data.Functor (($>))
+import Data.Map.Strict (Map)
+import qualified Data.Map.Strict as M
+import Data.String
+import Data.Text (Text)
+import qualified Data.Text as T
+import System.Console.ANSI.Types
+
+data Markdown = Markdown Format Text | Markdown :|: Markdown
+  deriving (Eq, Show)
+
+data Format
+  = Bold
+  | Italic
+  | Underline
+  | StrikeThrough
+  | Snippet
+  | Secret
+  | Colored Color
+  | NoFormat
+  deriving (Eq, Show)
+
+instance Semigroup Markdown where (<>) = (:|:)
+
+instance Monoid Markdown where mempty = unmarked ""
+
+instance IsString Markdown where fromString = unmarked . T.pack
+
+unmarked :: Text -> Markdown
+unmarked = Markdown NoFormat
+
+colorMD :: Char
+colorMD = '!'
+
+secretMD :: Char
+secretMD = '#'
+
+formats :: Map Char Format
+formats =
+  M.fromList
+    [ ('*', Bold),
+      ('_', Italic),
+      ('+', Underline),
+      ('~', StrikeThrough),
+      ('`', Snippet),
+      (secretMD, Secret),
+      (colorMD, Colored White)
+    ]
+
+colors :: Map Text Color
+colors =
+  M.fromList
+    [ ("red", Red),
+      ("green", Green),
+      ("blue", Blue),
+      ("yellow", Yellow),
+      ("cyan", Cyan),
+      ("magenta", Magenta),
+      ("r", Red),
+      ("g", Green),
+      ("b", Blue),
+      ("y", Yellow),
+      ("c", Cyan),
+      ("m", Magenta),
+      ("1", Red),
+      ("2", Green),
+      ("3", Blue),
+      ("4", Yellow),
+      ("5", Cyan),
+      ("6", Magenta)
+    ]
+
+parseMarkdown :: Text -> Markdown
+parseMarkdown s = fromRight (unmarked s) $ A.parseOnly (markdownP <* A.endOfInput) s
+
+markdownP :: Parser Markdown
+markdownP = merge <$> A.many' fragmentP
+  where
+    merge :: [Markdown] -> Markdown
+    merge [] = ""
+    merge fs = foldr1 (:|:) fs
+    fragmentP :: Parser Markdown
+    fragmentP =
+      A.anyChar >>= \case
+        ' ' -> unmarked . T.cons ' ' <$> A.takeWhile (== ' ')
+        c -> case M.lookup c formats of
+          Just Secret -> secretP
+          Just (Colored White) -> coloredP
+          Just f -> formattedP c "" f
+          Nothing -> unformattedP c
+    formattedP :: Char -> Text -> Format -> Parser Markdown
+    formattedP c p f = do
+      s <- A.takeTill (== c)
+      (A.char c $> markdown c p f s) <|> noFormat (c `T.cons` p <> s)
+    markdown :: Char -> Text -> Format -> Text -> Markdown
+    markdown c p f s
+      | T.null s || T.head s == ' ' || T.last s == ' ' =
+        unmarked $ c `T.cons` p <> s `T.snoc` c
+      | otherwise = Markdown f s
+    secretP :: Parser Markdown
+    secretP = secret <$> A.takeWhile (== secretMD) <*> A.takeTill (== secretMD) <*> A.takeWhile (== secretMD)
+    secret :: Text -> Text -> Text -> Markdown
+    secret b s a
+      | T.null a || T.null s || T.head s == ' ' || T.last s == ' ' =
+        unmarked $ secretMD `T.cons` ss
+      | otherwise = Markdown Secret $ T.init ss
+      where
+        ss = b <> s <> a
+    coloredP :: Parser Markdown
+    coloredP = do
+      color <- A.takeWhile (\c -> c /= ' ' && c /= colorMD)
+      case M.lookup color colors of
+        Just c ->
+          let f = Colored c
+           in (A.char ' ' *> formattedP colorMD (color `T.snoc` ' ') f)
+                <|> noFormat (colorMD `T.cons` color)
+        _ -> noFormat (colorMD `T.cons` color)
+    unformattedP :: Char -> Parser Markdown
+    unformattedP c = unmarked . T.cons c <$> wordsP
+    wordsP :: Parser Text
+    wordsP = do
+      s <- (<>) <$> A.takeTill (== ' ') <*> A.takeWhile (== ' ')
+      A.peekChar >>= \case
+        Nothing -> pure s
+        Just c -> case M.lookup c formats of
+          Just _ -> pure s
+          Nothing -> (s <>) <$> wordsP
+    noFormat :: Text -> Parser Markdown
+    noFormat = pure . unmarked

src/Simplex/Chat/Notification.hs

@@ -0,0 +1,98 @@
+{-# LANGUAGE LambdaCase #-}
+{-# LANGUAGE NamedFieldPuns #-}
+{-# LANGUAGE OverloadedStrings #-}
+{-# LANGUAGE ScopedTypeVariables #-}
+
+module Simplex.Chat.Notification (Notification (..), initializeNotifications) where
+
+import Control.Exception
+import Control.Monad (void)
+import Data.List (isInfixOf)
+import Data.Map (Map, fromList)
+import qualified Data.Map as M
+import Data.Maybe (fromMaybe)
+import Data.Text (Text)
+import qualified Data.Text as T
+import System.Directory (createDirectoryIfMissing, doesFileExist, getAppUserDataDirectory)
+import System.FilePath (combine)
+import System.Info (os)
+import System.Process (readCreateProcess, shell)
+
+data Notification = Notification {title :: Text, text :: Text}
+
+initializeNotifications :: IO (Notification -> IO ())
+initializeNotifications =
+  hideException <$> case os of
+    "darwin" -> pure $ notify macScript
+    "mingw32" -> initWinNotify
+    "linux" ->
+      doesFileExist "/proc/sys/kernel/osrelease" >>= \case
+        False -> pure $ notify linuxScript
+        True -> do
+          v <- readFile "/proc/sys/kernel/osrelease"
+          if "Microsoft" `isInfixOf` v || "WSL" `isInfixOf` v
+            then initWslNotify
+            else pure $ notify linuxScript
+    _ -> pure . const $ pure ()
+
+hideException :: (a -> IO ()) -> (a -> IO ())
+hideException f a = f a `catch` \(_ :: SomeException) -> pure ()
+
+notify :: (Notification -> Text) -> Notification -> IO ()
+notify script notification =
+  void $ readCreateProcess (shell . T.unpack $ script notification) ""
+
+linuxScript :: Notification -> Text
+linuxScript Notification {title, text} = "notify-send '" <> linuxEscape title <> "' '" <> linuxEscape text <> "'"
+
+linuxEscape :: Text -> Text
+linuxEscape = replaceAll $ fromList [('\'', "'\\''")]
+
+macScript :: Notification -> Text
+macScript Notification {title, text} = "osascript -e 'display notification \"" <> macEscape text <> "\" with title \"" <> macEscape title <> "\"'"
+
+macEscape :: Text -> Text
+macEscape = replaceAll $ fromList [('"', "\\\""), ('\'', "")]
+
+initWslNotify :: IO (Notification -> IO ())
+initWslNotify = notify . wslScript <$> savePowershellScript
+
+wslScript :: FilePath -> Notification -> Text
+wslScript path Notification {title, text} = "powershell.exe \"" <> T.pack path <> " \\\"" <> wslEscape title <> "\\\" \\\"" <> wslEscape text <> "\\\"\""
+
+wslEscape :: Text -> Text
+wslEscape = replaceAll $ fromList [('`', "\\`\\`"), ('\\', "\\\\"), ('"', "\\`\\\"")]
+
+initWinNotify :: IO (Notification -> IO ())
+initWinNotify = notify . winScript <$> savePowershellScript
+
+winScript :: FilePath -> Notification -> Text
+winScript path Notification {title, text} = "powershell.exe \"" <> T.pack path <> " '" <> winRemoveQuotes title <> "' '" <> winRemoveQuotes text <> "'\""
+
+winRemoveQuotes :: Text -> Text
+winRemoveQuotes = replaceAll $ fromList [('`', ""), ('\'', ""), ('"', "")]
+
+replaceAll :: Map Char Text -> Text -> Text
+replaceAll rules = T.concatMap $ \c -> T.singleton c `fromMaybe` M.lookup c rules
+
+savePowershellScript :: IO FilePath
+savePowershellScript = do
+  appDir <- getAppUserDataDirectory "simplex"
+  createDirectoryIfMissing False appDir
+  let psScript = combine appDir "win-toast-notify.ps1"
+  writeFile
+    psScript
+    "[Windows.UI.Notifications.ToastNotificationManager, Windows.UI.Notifications, ContentType = WindowsRuntime] > $null\n\
+    \$Template = [Windows.UI.Notifications.ToastNotificationManager]::GetTemplateContent([Windows.UI.Notifications.ToastTemplateType]::ToastText02)\n\
+    \$RawXml = [xml] $Template.GetXml()\n\
+    \($RawXml.toast.visual.binding.text|where {$_.id -eq \"1\"}).AppendChild($RawXml.CreateTextNode($args[0])) > $null\n\
+    \($RawXml.toast.visual.binding.text|where {$_.id -eq \"2\"}).AppendChild($RawXml.CreateTextNode($args[1])) > $null\n\
+    \$SerializedXml = New-Object Windows.Data.Xml.Dom.XmlDocument\n\
+    \$SerializedXml.LoadXml($RawXml.OuterXml)\n\
+    \$Toast = [Windows.UI.Notifications.ToastNotification]::new($SerializedXml)\n\
+    \$Toast.Tag = \"simplex-chat\"\n\
+    \$Toast.Group = \"simplex-chat\"\n\
+    \$Toast.ExpirationTime = [DateTimeOffset]::Now.AddMinutes(1)\n\
+    \$Notifier = [Windows.UI.Notifications.ToastNotificationManager]::CreateToastNotifier(\"PowerShell\")\n\
+    \$Notifier.Show($Toast);\n"
+  return psScript

src/Simplex/Chat/Options.hs

@@ -0,0 +1,62 @@
+{-# LANGUAGE OverloadedStrings #-}
+
+module Simplex.Chat.Options (getChatOpts, ChatOpts (..)) where
+
+import qualified Data.Attoparsec.ByteString.Char8 as A
+import qualified Data.ByteString.Char8 as B
+import Data.List.NonEmpty (NonEmpty)
+import qualified Data.List.NonEmpty as L
+import Options.Applicative
+import Simplex.Messaging.Agent.Protocol (SMPServer (..), smpServerP)
+import Simplex.Messaging.Parsers (parseAll)
+import System.FilePath (combine)
+
+data ChatOpts = ChatOpts
+  { dbFile :: String,
+    smpServers :: NonEmpty SMPServer
+  }
+
+chatOpts :: FilePath -> Parser ChatOpts
+chatOpts appDir =
+  ChatOpts
+    <$> strOption
+      ( long "database"
+          <> short 'd'
+          <> metavar "DB_FILE"
+          <> help ("sqlite database file path (" <> defaultDbFilePath <> ")")
+          <> value defaultDbFilePath
+      )
+    <*> option
+      parseSMPServer
+      ( long "server"
+          <> short 's'
+          <> metavar "SERVER"
+          <> help
+            ( "SMP server(s) to use"
+                <> "\n(smp2.simplex.im,smp3.simplex.im)"
+            )
+          <> value
+            ( L.fromList
+                [ "smp2.simplex.im#z5W2QLQ1Br3Yd6CoWg7bIq1bHdwK7Y8bEiEXBs/WfAg=", -- London, UK
+                  "smp3.simplex.im#nxc7HnrnM8dOKgkMp008ub/9o9LXJlxlMrMpR+mfMQw=" -- Fremont, CA
+                ]
+            )
+      )
+  where
+    defaultDbFilePath = combine appDir "simplex"
+
+parseSMPServer :: ReadM (NonEmpty SMPServer)
+parseSMPServer = eitherReader $ parseAll servers . B.pack
+  where
+    servers = L.fromList <$> smpServerP `A.sepBy1` A.char ','
+
+getChatOpts :: FilePath -> IO ChatOpts
+getChatOpts appDir = execParser opts
+  where
+    opts =
+      info
+        (chatOpts appDir <**> helper)
+        ( fullDesc
+            <> header "Chat prototype using Simplex Messaging Protocol (SMP)"
+            <> progDesc "Start chat with DB_FILE file and use SERVER as SMP server"
+        )

src/Simplex/Chat/Protocol.hs

@@ -0,0 +1,383 @@
+{-# LANGUAGE DataKinds #-}
+{-# LANGUAGE DuplicateRecordFields #-}
+{-# LANGUAGE GADTs #-}
+{-# LANGUAGE KindSignatures #-}
+{-# LANGUAGE LambdaCase #-}
+{-# LANGUAGE NamedFieldPuns #-}
+{-# LANGUAGE OverloadedStrings #-}
+{-# LANGUAGE StandaloneDeriving #-}
+{-# LANGUAGE TupleSections #-}
+
+module Simplex.Chat.Protocol where
+
+import Control.Applicative (optional)
+import Control.Monad ((<=<))
+import Data.Aeson (FromJSON, ToJSON)
+import qualified Data.Aeson as J
+import Data.Attoparsec.ByteString.Char8 (Parser)
+import qualified Data.Attoparsec.ByteString.Char8 as A
+import qualified Data.ByteString.Base64 as B64
+import Data.ByteString.Char8 (ByteString)
+import qualified Data.ByteString.Char8 as B
+import qualified Data.ByteString.Lazy.Char8 as LB
+import Data.Int (Int64)
+import Data.List (find)
+import Data.Text (Text)
+import qualified Data.Text as T
+import Data.Text.Encoding (encodeUtf8)
+import Simplex.Chat.Types
+import Simplex.Chat.Util (safeDecodeUtf8)
+import Simplex.Messaging.Agent.Protocol
+import Simplex.Messaging.Parsers (parseAll)
+import Simplex.Messaging.Util (bshow)
+
+data ChatDirection (p :: AParty) where
+  ReceivedDirectMessage :: Connection -> Maybe Contact -> ChatDirection 'Agent
+  SentDirectMessage :: Contact -> ChatDirection 'Client
+  ReceivedGroupMessage :: Connection -> GroupName -> GroupMember -> ChatDirection 'Agent
+  SentGroupMessage :: GroupName -> ChatDirection 'Client
+  SndFileConnection :: Connection -> SndFileTransfer -> ChatDirection 'Agent
+  RcvFileConnection :: Connection -> RcvFileTransfer -> ChatDirection 'Agent
+
+deriving instance Eq (ChatDirection p)
+
+deriving instance Show (ChatDirection p)
+
+fromConnection :: ChatDirection 'Agent -> Connection
+fromConnection = \case
+  ReceivedDirectMessage conn _ -> conn
+  ReceivedGroupMessage conn _ _ -> conn
+  SndFileConnection conn _ -> conn
+  RcvFileConnection conn _ -> conn
+
+data ChatMsgEvent
+  = XMsgNew MsgContent
+  | XFile FileInvitation
+  | XFileAcpt String
+  | XInfo Profile
+  | XGrpInv GroupInvitation
+  | XGrpAcpt MemberId
+  | XGrpMemNew MemberInfo
+  | XGrpMemIntro MemberInfo
+  | XGrpMemInv MemberId IntroInvitation
+  | XGrpMemFwd MemberInfo IntroInvitation
+  | XGrpMemInfo MemberId Profile
+  | XGrpMemCon MemberId
+  | XGrpMemConAll MemberId
+  | XGrpMemDel MemberId
+  | XGrpLeave
+  | XGrpDel
+  | XInfoProbe ByteString
+  | XInfoProbeCheck ByteString
+  | XInfoProbeOk ByteString
+  | XOk
+  deriving (Eq, Show)
+
+data MessageType = MTText | MTImage deriving (Eq, Show)
+
+data MsgContent = MsgContent
+  { messageType :: MessageType,
+    files :: [(ContentType, Int)],
+    content :: [MsgContentBody]
+  }
+  deriving (Eq, Show)
+
+toMsgType :: ByteString -> Either String MessageType
+toMsgType = \case
+  "c.text" -> Right MTText
+  "c.image" -> Right MTImage
+  t -> Left $ "invalid message type " <> B.unpack t
+
+rawMsgType :: MessageType -> ByteString
+rawMsgType = \case
+  MTText -> "c.text"
+  MTImage -> "c.image"
+
+data ChatMessage = ChatMessage
+  { chatMsgId :: Maybe Int64,
+    chatMsgEvent :: ChatMsgEvent,
+    chatDAG :: Maybe ByteString
+  }
+  deriving (Eq, Show)
+
+toChatMessage :: RawChatMessage -> Either String ChatMessage
+toChatMessage RawChatMessage {chatMsgId, chatMsgEvent, chatMsgParams, chatMsgBody} = do
+  (chatDAG, body) <- getDAG <$> mapM toMsgBodyContent chatMsgBody
+  let chatMsg msg = pure ChatMessage {chatMsgId, chatMsgEvent = msg, chatDAG}
+  case (chatMsgEvent, chatMsgParams) of
+    ("x.msg.new", mt : rawFiles) -> do
+      t <- toMsgType mt
+      files <- mapM (toContentInfo <=< parseAll contentInfoP) rawFiles
+      chatMsg . XMsgNew $ MsgContent {messageType = t, files, content = body}
+    ("x.file", [name, size, qInfo]) -> do
+      let fileName = T.unpack $ safeDecodeUtf8 name
+      fileSize <- parseAll A.decimal size
+      fileQInfo <- parseAll smpQueueInfoP qInfo
+      chatMsg . XFile $ FileInvitation {fileName, fileSize, fileQInfo}
+    ("x.file.acpt", [name]) ->
+      chatMsg . XFileAcpt . T.unpack $ safeDecodeUtf8 name
+    ("x.info", []) -> do
+      profile <- getJSON body
+      chatMsg $ XInfo profile
+    ("x.grp.inv", [fromMemId, fromRole, memId, role, qInfo]) -> do
+      fromMem <- (,) <$> B64.decode fromMemId <*> toMemberRole fromRole
+      invitedMem <- (,) <$> B64.decode memId <*> toMemberRole role
+      groupQInfo <- parseAll smpQueueInfoP qInfo
+      profile <- getJSON body
+      chatMsg . XGrpInv $ GroupInvitation fromMem invitedMem groupQInfo profile
+    ("x.grp.acpt", [memId]) ->
+      chatMsg . XGrpAcpt =<< B64.decode memId
+    ("x.grp.mem.new", [memId, role]) -> do
+      chatMsg . XGrpMemNew =<< toMemberInfo memId role body
+    ("x.grp.mem.intro", [memId, role]) ->
+      chatMsg . XGrpMemIntro =<< toMemberInfo memId role body
+    ("x.grp.mem.inv", [memId, groupQInfo, directQInfo]) ->
+      chatMsg =<< (XGrpMemInv <$> B64.decode memId <*> toIntroInv groupQInfo directQInfo)
+    ("x.grp.mem.fwd", [memId, role, groupQInfo, directQInfo]) -> do
+      chatMsg =<< (XGrpMemFwd <$> toMemberInfo memId role body <*> toIntroInv groupQInfo directQInfo)
+    ("x.grp.mem.info", [memId]) ->
+      chatMsg =<< (XGrpMemInfo <$> B64.decode memId <*> getJSON body)
+    ("x.grp.mem.con", [memId]) ->
+      chatMsg . XGrpMemCon =<< B64.decode memId
+    ("x.grp.mem.con.all", [memId]) ->
+      chatMsg . XGrpMemConAll =<< B64.decode memId
+    ("x.grp.mem.del", [memId]) ->
+      chatMsg . XGrpMemDel =<< B64.decode memId
+    ("x.grp.leave", []) ->
+      chatMsg XGrpLeave
+    ("x.grp.del", []) ->
+      chatMsg XGrpDel
+    ("x.info.probe", [probe]) -> do
+      chatMsg . XInfoProbe =<< B64.decode probe
+    ("x.info.probe.check", [probeHash]) -> do
+      chatMsg =<< (XInfoProbeCheck <$> B64.decode probeHash)
+    ("x.info.probe.ok", [probe]) -> do
+      chatMsg =<< (XInfoProbeOk <$> B64.decode probe)
+    ("x.ok", []) ->
+      chatMsg XOk
+    _ -> Left $ "bad syntax or unsupported event " <> B.unpack chatMsgEvent
+  where
+    getDAG :: [MsgContentBody] -> (Maybe ByteString, [MsgContentBody])
+    getDAG body = case break (isContentType SimplexDAG) body of
+      (b, MsgContentBody SimplexDAG dag : a) -> (Just dag, b <> a)
+      _ -> (Nothing, body)
+    toMemberInfo :: ByteString -> ByteString -> [MsgContentBody] -> Either String MemberInfo
+    toMemberInfo memId role body = MemberInfo <$> B64.decode memId <*> toMemberRole role <*> getJSON body
+    toIntroInv :: ByteString -> ByteString -> Either String IntroInvitation
+    toIntroInv groupQInfo directQInfo = IntroInvitation <$> parseAll smpQueueInfoP groupQInfo <*> parseAll smpQueueInfoP directQInfo
+    toContentInfo :: (RawContentType, Int) -> Either String (ContentType, Int)
+    toContentInfo (rawType, size) = (,size) <$> toContentType rawType
+    getJSON :: FromJSON a => [MsgContentBody] -> Either String a
+    getJSON = J.eitherDecodeStrict' <=< getSimplexContentType XCJson
+
+isContentType :: ContentType -> MsgContentBody -> Bool
+isContentType t MsgContentBody {contentType = t'} = t == t'
+
+isSimplexContentType :: XContentType -> MsgContentBody -> Bool
+isSimplexContentType = isContentType . SimplexContentType
+
+getContentType :: ContentType -> [MsgContentBody] -> Either String ByteString
+getContentType t body = case find (isContentType t) body of
+  Just MsgContentBody {contentData} -> Right contentData
+  Nothing -> Left "no required content type"
+
+getSimplexContentType :: XContentType -> [MsgContentBody] -> Either String ByteString
+getSimplexContentType = getContentType . SimplexContentType
+
+rawChatMessage :: ChatMessage -> RawChatMessage
+rawChatMessage ChatMessage {chatMsgId, chatMsgEvent, chatDAG} =
+  case chatMsgEvent of
+    XMsgNew MsgContent {messageType = t, files, content} ->
+      let rawFiles = map (serializeContentInfo . rawContentInfo) files
+       in rawMsg "x.msg.new" (rawMsgType t : rawFiles) content
+    XFile FileInvitation {fileName, fileSize, fileQInfo} ->
+      rawMsg "x.file" [encodeUtf8 $ T.pack fileName, bshow fileSize, serializeSmpQueueInfo fileQInfo] []
+    XFileAcpt fileName ->
+      rawMsg "x.file.acpt" [encodeUtf8 $ T.pack fileName] []
+    XInfo profile ->
+      rawMsg "x.info" [] [jsonBody profile]
+    XGrpInv (GroupInvitation (fromMemId, fromRole) (memId, role) qInfo groupProfile) ->
+      let params =
+            [ B64.encode fromMemId,
+              serializeMemberRole fromRole,
+              B64.encode memId,
+              serializeMemberRole role,
+              serializeSmpQueueInfo qInfo
+            ]
+       in rawMsg "x.grp.inv" params [jsonBody groupProfile]
+    XGrpAcpt memId ->
+      rawMsg "x.grp.acpt" [B64.encode memId] []
+    XGrpMemNew (MemberInfo memId role profile) ->
+      let params = [B64.encode memId, serializeMemberRole role]
+       in rawMsg "x.grp.mem.new" params [jsonBody profile]
+    XGrpMemIntro (MemberInfo memId role profile) ->
+      rawMsg "x.grp.mem.intro" [B64.encode memId, serializeMemberRole role] [jsonBody profile]
+    XGrpMemInv memId IntroInvitation {groupQInfo, directQInfo} ->
+      let params = [B64.encode memId, serializeSmpQueueInfo groupQInfo, serializeSmpQueueInfo directQInfo]
+       in rawMsg "x.grp.mem.inv" params []
+    XGrpMemFwd (MemberInfo memId role profile) IntroInvitation {groupQInfo, directQInfo} ->
+      let params =
+            [ B64.encode memId,
+              serializeMemberRole role,
+              serializeSmpQueueInfo groupQInfo,
+              serializeSmpQueueInfo directQInfo
+            ]
+       in rawMsg "x.grp.mem.fwd" params [jsonBody profile]
+    XGrpMemInfo memId profile ->
+      rawMsg "x.grp.mem.info" [B64.encode memId] [jsonBody profile]
+    XGrpMemCon memId ->
+      rawMsg "x.grp.mem.con" [B64.encode memId] []
+    XGrpMemConAll memId ->
+      rawMsg "x.grp.mem.con.all" [B64.encode memId] []
+    XGrpMemDel memId ->
+      rawMsg "x.grp.mem.del" [B64.encode memId] []
+    XGrpLeave ->
+      rawMsg "x.grp.leave" [] []
+    XGrpDel ->
+      rawMsg "x.grp.del" [] []
+    XInfoProbe probe ->
+      rawMsg "x.info.probe" [B64.encode probe] []
+    XInfoProbeCheck probeHash ->
+      rawMsg "x.info.probe.check" [B64.encode probeHash] []
+    XInfoProbeOk probe ->
+      rawMsg "x.info.probe.ok" [B64.encode probe] []
+    XOk ->
+      rawMsg "x.ok" [] []
+  where
+    rawMsg :: ByteString -> [ByteString] -> [MsgContentBody] -> RawChatMessage
+    rawMsg event chatMsgParams body =
+      RawChatMessage {chatMsgId, chatMsgEvent = event, chatMsgParams, chatMsgBody = rawWithDAG body}
+    rawContentInfo :: (ContentType, Int) -> (RawContentType, Int)
+    rawContentInfo (t, size) = (rawContentType t, size)
+    jsonBody :: ToJSON a => a -> MsgContentBody
+    jsonBody x =
+      let json = LB.toStrict $ J.encode x
+       in MsgContentBody {contentType = SimplexContentType XCJson, contentData = json}
+    rawWithDAG :: [MsgContentBody] -> [RawMsgBodyContent]
+    rawWithDAG body = map rawMsgBodyContent $ case chatDAG of
+      Nothing -> body
+      Just dag -> MsgContentBody {contentType = SimplexDAG, contentData = dag} : body
+
+toMsgBodyContent :: RawMsgBodyContent -> Either String MsgContentBody
+toMsgBodyContent RawMsgBodyContent {contentType, contentData} = do
+  cType <- toContentType contentType
+  pure MsgContentBody {contentType = cType, contentData}
+
+rawMsgBodyContent :: MsgContentBody -> RawMsgBodyContent
+rawMsgBodyContent MsgContentBody {contentType = t, contentData} =
+  RawMsgBodyContent {contentType = rawContentType t, contentData}
+
+data MsgContentBody = MsgContentBody
+  { contentType :: ContentType,
+    contentData :: ByteString
+  }
+  deriving (Eq, Show)
+
+data ContentType
+  = SimplexContentType XContentType
+  | MimeContentType MContentType
+  | SimplexDAG
+  deriving (Eq, Show)
+
+data XContentType = XCText | XCImage | XCJson deriving (Eq, Show)
+
+data MContentType = MCImageJPG | MCImagePNG deriving (Eq, Show)
+
+toContentType :: RawContentType -> Either String ContentType
+toContentType (RawContentType ns cType) = case ns of
+  "x" -> case cType of
+    "text" -> Right $ SimplexContentType XCText
+    "image" -> Right $ SimplexContentType XCImage
+    "json" -> Right $ SimplexContentType XCJson
+    "dag" -> Right SimplexDAG
+    _ -> err
+  "m" -> case cType of
+    "image/jpg" -> Right $ MimeContentType MCImageJPG
+    "image/png" -> Right $ MimeContentType MCImagePNG
+    _ -> err
+  _ -> err
+  where
+    err = Left . B.unpack $ "invalid content type " <> ns <> "." <> cType
+
+rawContentType :: ContentType -> RawContentType
+rawContentType t = case t of
+  SimplexContentType t' -> RawContentType "x" $ case t' of
+    XCText -> "text"
+    XCImage -> "image"
+    XCJson -> "json"
+  MimeContentType t' -> RawContentType "m" $ case t' of
+    MCImageJPG -> "image/jpg"
+    MCImagePNG -> "image/png"
+  SimplexDAG -> RawContentType "x" "dag"
+
+newtype ContentMsg = NewContentMsg ContentData
+
+newtype ContentData = ContentText Text
+
+data RawChatMessage = RawChatMessage
+  { chatMsgId :: Maybe Int64,
+    chatMsgEvent :: ByteString,
+    chatMsgParams :: [ByteString],
+    chatMsgBody :: [RawMsgBodyContent]
+  }
+  deriving (Eq, Show)
+
+data RawMsgBodyContent = RawMsgBodyContent
+  { contentType :: RawContentType,
+    contentData :: ByteString
+  }
+  deriving (Eq, Show)
+
+data RawContentType = RawContentType NameSpace ByteString
+  deriving (Eq, Show)
+
+type NameSpace = ByteString
+
+newtype MsgData = MsgData ByteString
+  deriving (Eq, Show)
+
+class DataLength a where
+  dataLength :: a -> Int
+
+rawChatMessageP :: Parser RawChatMessage
+rawChatMessageP = do
+  chatMsgId <- optional A.decimal <* A.space
+  chatMsgEvent <- B.intercalate "." <$> identifierP `A.sepBy1'` A.char '.' <* A.space
+  chatMsgParams <- A.takeWhile1 (not . A.inClass ", ") `A.sepBy'` A.char ',' <* A.space
+  chatMsgBody <- msgBodyContent =<< contentInfoP `A.sepBy'` A.char ',' <* A.space
+  pure RawChatMessage {chatMsgId, chatMsgEvent, chatMsgParams, chatMsgBody}
+  where
+    msgBodyContent :: [(RawContentType, Int)] -> Parser [RawMsgBodyContent]
+    msgBodyContent [] = pure []
+    msgBodyContent ((contentType, size) : ps) = do
+      contentData <- A.take size <* A.space
+      ((RawMsgBodyContent {contentType, contentData}) :) <$> msgBodyContent ps
+
+contentInfoP :: Parser (RawContentType, Int)
+contentInfoP = do
+  contentType <- RawContentType <$> identifierP <* A.char '.' <*> A.takeTill (A.inClass ":, ")
+  size <- A.char ':' *> A.decimal
+  pure (contentType, size)
+
+identifierP :: Parser ByteString
+identifierP = B.cons <$> A.letter_ascii <*> A.takeWhile (\c -> A.isAlpha_ascii c || A.isDigit c)
+
+serializeRawChatMessage :: RawChatMessage -> ByteString
+serializeRawChatMessage RawChatMessage {chatMsgId, chatMsgEvent, chatMsgParams, chatMsgBody} =
+  B.unwords
+    [ maybe "" bshow chatMsgId,
+      chatMsgEvent,
+      B.intercalate "," chatMsgParams,
+      B.unwords $ map serializeBodyContentInfo chatMsgBody,
+      B.unwords $ map msgContentData chatMsgBody
+    ]
+
+serializeBodyContentInfo :: RawMsgBodyContent -> ByteString
+serializeBodyContentInfo RawMsgBodyContent {contentType = t, contentData} =
+  serializeContentInfo (t, B.length contentData)
+
+serializeContentInfo :: (RawContentType, Int) -> ByteString
+serializeContentInfo (RawContentType ns cType, size) = ns <> "." <> cType <> ":" <> bshow size
+
+msgContentData :: RawMsgBodyContent -> ByteString
+msgContentData RawMsgBodyContent {contentData} = contentData <> " "

src/Simplex/Chat/Styled.hs

@@ -1,14 +1,13 @@
 {-# LANGUAGE FlexibleInstances #-}
 {-# LANGUAGE LambdaCase #-}
 
-module Styled
+module Simplex.Chat.Styled
   ( StyledString (..),
-    bPlain,
-    plain,
+    StyledFormat (..),
     styleMarkdown,
     styleMarkdownText,
-    styled,
     sLength,
+    sShow,
   )
 where
 
@@ -17,7 +16,7 @@ import qualified Data.ByteString.Char8 as B
 import Data.String
 import Data.Text (Text)
 import qualified Data.Text as T
-import Simplex.Markdown
+import Simplex.Chat.Markdown
 import System.Console.ANSI.Types
 
 data StyledString = Styled [SGR] String | StyledString :<>: StyledString
@@ -28,12 +27,6 @@ instance Monoid StyledString where mempty = plain ""
 
 instance IsString StyledString where fromString = plain
 
-plain :: String -> StyledString
-plain = Styled []
-
-bPlain :: ByteString -> StyledString
-bPlain = Styled [] . B.unpack
-
 styleMarkdownText :: Text -> StyledString
 styleMarkdownText = styleMarkdown . parseMarkdown
 
@@ -48,12 +41,22 @@ wrap c s = plain [c] <> s <> plain [c]
 
 class StyledFormat a where
   styled :: Format -> a -> StyledString
+  plain :: a -> StyledString
 
-instance StyledFormat String where styled = Styled . sgr
+instance StyledFormat String where
+  styled = Styled . sgr
+  plain = Styled []
 
-instance StyledFormat ByteString where styled f = styled f . B.unpack
+instance StyledFormat ByteString where
+  styled f = styled f . B.unpack
+  plain = Styled [] . B.unpack
 
-instance StyledFormat Text where styled f = styled f . T.unpack
+instance StyledFormat Text where
+  styled f = styled f . T.unpack
+  plain = Styled [] . T.unpack
+
+sShow :: Show a => a -> StyledString
+sShow = plain . show
 
 sgr :: Format -> [SGR]
 sgr = \case

src/Simplex/Chat/Terminal.hs

@@ -0,0 +1,176 @@
+{-# LANGUAGE GADTs #-}
+{-# LANGUAGE LambdaCase #-}
+{-# LANGUAGE NamedFieldPuns #-}
+{-# LANGUAGE RankNTypes #-}
+{-# LANGUAGE ScopedTypeVariables #-}
+
+module Simplex.Chat.Terminal where
+
+import Control.Monad.Catch (MonadMask)
+import Control.Monad.IO.Class (MonadIO)
+import Simplex.Chat.Styled
+import Simplex.Chat.Types
+import System.Console.ANSI.Types
+import System.Terminal
+import System.Terminal.Internal (LocalTerminal, Terminal, VirtualTerminal)
+import UnliftIO.STM
+
+data ActiveTo = ActiveNone | ActiveC ContactName | ActiveG GroupName
+  deriving (Eq)
+
+data ChatTerminal = ChatTerminal
+  { activeTo :: TVar ActiveTo,
+    termDevice :: TerminalDevice,
+    termState :: TVar TerminalState,
+    termSize :: Size,
+    nextMessageRow :: TVar Int,
+    termLock :: TMVar ()
+  }
+
+data TerminalState = TerminalState
+  { inputPrompt :: String,
+    inputString :: String,
+    inputPosition :: Int,
+    previousInput :: String
+  }
+
+class Terminal t => WithTerminal t where
+  withTerm :: (MonadIO m, MonadMask m) => t -> (t -> m a) -> m a
+
+data TerminalDevice = forall t. WithTerminal t => TerminalDevice t
+
+instance WithTerminal LocalTerminal where
+  withTerm _ = withTerminal
+
+instance WithTerminal VirtualTerminal where
+  withTerm t = ($ t)
+
+withChatTerm :: (MonadIO m, MonadMask m) => ChatTerminal -> (forall t. WithTerminal t => TerminalT t m a) -> m a
+withChatTerm ChatTerminal {termDevice = TerminalDevice t} action = withTerm t $ runTerminalT action
+
+newChatTerminal :: WithTerminal t => t -> IO ChatTerminal
+newChatTerminal t = do
+  activeTo <- newTVarIO ActiveNone
+  termSize <- withTerm t . runTerminalT $ getWindowSize
+  let lastRow = height termSize - 1
+  termState <- newTVarIO newTermState
+  termLock <- newTMVarIO ()
+  nextMessageRow <- newTVarIO lastRow
+  -- threadDelay 500000 -- this delay is the same as timeout in getTerminalSize
+  return ChatTerminal {activeTo, termDevice = TerminalDevice t, termState, termSize, nextMessageRow, termLock}
+
+newTermState :: TerminalState
+newTermState =
+  TerminalState
+    { inputString = "",
+      inputPosition = 0,
+      inputPrompt = "> ",
+      previousInput = ""
+    }
+
+withTermLock :: MonadTerminal m => ChatTerminal -> m () -> m ()
+withTermLock ChatTerminal {termLock} action = do
+  _ <- atomically $ takeTMVar termLock
+  action
+  atomically $ putTMVar termLock ()
+
+printToTerminal :: ChatTerminal -> [StyledString] -> IO ()
+printToTerminal ct s =
+  withChatTerm ct $
+    withTermLock ct $ do
+      printMessage ct s
+      updateInput ct
+
+updateInput :: forall m. MonadTerminal m => ChatTerminal -> m ()
+updateInput ChatTerminal {termSize = Size {height, width}, termState, nextMessageRow} = do
+  hideCursor
+  ts <- readTVarIO termState
+  nmr <- readTVarIO nextMessageRow
+  let ih = inputHeight ts
+      iStart = height - ih
+      prompt = inputPrompt ts
+      Position {row, col} = positionRowColumn width $ length prompt + inputPosition ts
+  if nmr >= iStart
+    then atomically $ writeTVar nextMessageRow iStart
+    else clearLines nmr iStart
+  setCursorPosition $ Position {row = max nmr iStart, col = 0}
+  putString $ prompt <> inputString ts <> " "
+  eraseInLine EraseForward
+  setCursorPosition $ Position {row = iStart + row, col}
+  showCursor
+  flush
+  where
+    clearLines :: Int -> Int -> m ()
+    clearLines from till
+      | from >= till = return ()
+      | otherwise = do
+        setCursorPosition $ Position {row = from, col = 0}
+        eraseInLine EraseForward
+        clearLines (from + 1) till
+    inputHeight :: TerminalState -> Int
+    inputHeight ts = length (inputPrompt ts <> inputString ts) `div` width + 1
+    positionRowColumn :: Int -> Int -> Position
+    positionRowColumn wid pos =
+      let row = pos `div` wid
+          col = pos - row * wid
+       in Position {row, col}
+
+printMessage :: forall m. MonadTerminal m => ChatTerminal -> [StyledString] -> m ()
+printMessage ChatTerminal {termSize = Size {height, width}, nextMessageRow} msg = do
+  nmr <- readTVarIO nextMessageRow
+  setCursorPosition $ Position {row = nmr, col = 0}
+  mapM_ printStyled msg
+  flush
+  let lc = sum $ map lineCount msg
+  atomically . writeTVar nextMessageRow $ min (height - 1) (nmr + lc)
+  where
+    lineCount :: StyledString -> Int
+    lineCount s = sLength s `div` width + 1
+    printStyled :: StyledString -> m ()
+    printStyled s = do
+      putStyled s
+      eraseInLine EraseForward
+      putLn
+
+-- Currently it is assumed that the message does not have internal line breaks.
+-- Previous implementation "kind of" supported them,
+-- but it was not determining the number of printed lines correctly
+-- because of accounting for control sequences in length
+putStyled :: MonadTerminal m => StyledString -> m ()
+putStyled (s1 :<>: s2) = putStyled s1 >> putStyled s2
+putStyled (Styled [] s) = putString s
+putStyled (Styled sgr s) = setSGR sgr >> putString s >> resetAttributes
+
+setSGR :: MonadTerminal m => [SGR] -> m ()
+setSGR = mapM_ $ \case
+  Reset -> resetAttributes
+  SetConsoleIntensity BoldIntensity -> setAttribute bold
+  SetConsoleIntensity _ -> resetAttribute bold
+  SetItalicized True -> setAttribute italic
+  SetItalicized _ -> resetAttribute italic
+  SetUnderlining NoUnderline -> resetAttribute underlined
+  SetUnderlining _ -> setAttribute underlined
+  SetSwapForegroundBackground True -> setAttribute inverted
+  SetSwapForegroundBackground _ -> resetAttribute inverted
+  SetColor l i c -> setAttribute . layer l . intensity i $ color c
+  SetBlinkSpeed _ -> pure ()
+  SetVisible _ -> pure ()
+  SetRGBColor _ _ -> pure ()
+  SetPaletteColor _ _ -> pure ()
+  SetDefaultColor _ -> pure ()
+  where
+    layer = \case
+      Foreground -> foreground
+      Background -> background
+    intensity = \case
+      Dull -> id
+      Vivid -> bright
+    color = \case
+      Black -> black
+      Red -> red
+      Green -> green
+      Yellow -> yellow
+      Blue -> blue
+      Magenta -> magenta
+      Cyan -> cyan
+      White -> white

src/Simplex/Chat/Types.hs

@@ -0,0 +1,497 @@
+{-# LANGUAGE DeriveGeneric #-}
+{-# LANGUAGE DuplicateRecordFields #-}
+{-# LANGUAGE LambdaCase #-}
+{-# LANGUAGE NamedFieldPuns #-}
+{-# LANGUAGE OverloadedStrings #-}
+
+module Simplex.Chat.Types where
+
+import Data.Aeson (FromJSON, ToJSON)
+import qualified Data.Aeson as J
+import Data.ByteString.Char8 (ByteString)
+import qualified Data.ByteString.Char8 as B
+import Data.Int (Int64)
+import Data.Text (Text)
+import Data.Time.Clock (UTCTime)
+import Data.Typeable (Typeable)
+import Database.SQLite.Simple (ResultError (..), SQLData (..))
+import Database.SQLite.Simple.FromField (FieldParser, FromField (..), returnError)
+import Database.SQLite.Simple.Internal (Field (..))
+import Database.SQLite.Simple.Ok (Ok (Ok))
+import Database.SQLite.Simple.ToField (ToField (..))
+import GHC.Generics
+import Simplex.Messaging.Agent.Protocol (ConnId, SMPQueueInfo)
+import Simplex.Messaging.Agent.Store.SQLite (fromTextField_)
+
+class IsContact a where
+  contactId' :: a -> Int64
+  profile' :: a -> Profile
+  localDisplayName' :: a -> ContactName
+
+instance IsContact User where
+  contactId' = userContactId
+  profile' = profile
+  localDisplayName' = localDisplayName
+
+instance IsContact Contact where
+  contactId' = contactId
+  profile' = profile
+  localDisplayName' = localDisplayName
+
+data User = User
+  { userId :: UserId,
+    userContactId :: Int64,
+    localDisplayName :: ContactName,
+    profile :: Profile,
+    activeUser :: Bool
+  }
+
+type UserId = Int64
+
+data Contact = Contact
+  { contactId :: Int64,
+    localDisplayName :: ContactName,
+    profile :: Profile,
+    activeConn :: Connection,
+    viaGroup :: Maybe Int64
+  }
+  deriving (Eq, Show)
+
+contactConnId :: Contact -> ConnId
+contactConnId Contact {activeConn = Connection {agentConnId}} = agentConnId
+
+type ContactName = Text
+
+type GroupName = Text
+
+data Group = Group
+  { groupId :: Int64,
+    localDisplayName :: GroupName,
+    groupProfile :: GroupProfile,
+    members :: [GroupMember],
+    membership :: GroupMember
+  }
+  deriving (Eq, Show)
+
+data Profile = Profile
+  { displayName :: ContactName,
+    fullName :: Text
+  }
+  deriving (Generic, Eq, Show)
+
+instance ToJSON Profile where toEncoding = J.genericToEncoding J.defaultOptions
+
+instance FromJSON Profile
+
+data GroupProfile = GroupProfile
+  { displayName :: GroupName,
+    fullName :: Text
+  }
+  deriving (Generic, Eq, Show)
+
+instance ToJSON GroupProfile where toEncoding = J.genericToEncoding J.defaultOptions
+
+instance FromJSON GroupProfile
+
+data GroupInvitation = GroupInvitation
+  { fromMember :: (MemberId, GroupMemberRole),
+    invitedMember :: (MemberId, GroupMemberRole),
+    queueInfo :: SMPQueueInfo,
+    groupProfile :: GroupProfile
+  }
+  deriving (Eq, Show)
+
+data IntroInvitation = IntroInvitation
+  { groupQInfo :: SMPQueueInfo,
+    directQInfo :: SMPQueueInfo
+  }
+  deriving (Eq, Show)
+
+data MemberInfo = MemberInfo MemberId GroupMemberRole Profile
+  deriving (Eq, Show)
+
+memberInfo :: GroupMember -> MemberInfo
+memberInfo m = MemberInfo (memberId m) (memberRole m) (memberProfile m)
+
+data ReceivedGroupInvitation = ReceivedGroupInvitation
+  { fromMember :: GroupMember,
+    userMember :: GroupMember,
+    queueInfo :: SMPQueueInfo,
+    groupProfile :: GroupProfile
+  }
+  deriving (Eq, Show)
+
+data GroupMember = GroupMember
+  { groupMemberId :: Int64,
+    groupId :: Int64,
+    memberId :: MemberId,
+    memberRole :: GroupMemberRole,
+    memberCategory :: GroupMemberCategory,
+    memberStatus :: GroupMemberStatus,
+    invitedBy :: InvitedBy,
+    localDisplayName :: ContactName,
+    memberProfile :: Profile,
+    memberContactId :: Maybe Int64,
+    activeConn :: Maybe Connection
+  }
+  deriving (Eq, Show)
+
+memberConnId :: GroupMember -> Maybe ConnId
+memberConnId GroupMember {activeConn} = case activeConn of
+  Just Connection {agentConnId} -> Just agentConnId
+  Nothing -> Nothing
+
+data NewGroupMember = NewGroupMember
+  { memInfo :: MemberInfo,
+    memCategory :: GroupMemberCategory,
+    memStatus :: GroupMemberStatus,
+    memInvitedBy :: InvitedBy,
+    localDisplayName :: ContactName,
+    memProfileId :: Int64,
+    memContactId :: Maybe Int64
+  }
+
+type MemberId = ByteString
+
+data InvitedBy = IBContact Int64 | IBUser | IBUnknown
+  deriving (Eq, Show)
+
+toInvitedBy :: Int64 -> Maybe Int64 -> InvitedBy
+toInvitedBy userCtId (Just ctId)
+  | userCtId == ctId = IBUser
+  | otherwise = IBContact ctId
+toInvitedBy _ Nothing = IBUnknown
+
+fromInvitedBy :: Int64 -> InvitedBy -> Maybe Int64
+fromInvitedBy userCtId = \case
+  IBUnknown -> Nothing
+  IBContact ctId -> Just ctId
+  IBUser -> Just userCtId
+
+data GroupMemberRole = GRMember | GRAdmin | GROwner
+  deriving (Eq, Show, Ord)
+
+instance FromField GroupMemberRole where fromField = fromBlobField_ toMemberRole
+
+instance ToField GroupMemberRole where toField = toField . serializeMemberRole
+
+toMemberRole :: ByteString -> Either String GroupMemberRole
+toMemberRole = \case
+  "owner" -> Right GROwner
+  "admin" -> Right GRAdmin
+  "member" -> Right GRMember
+  r -> Left $ "invalid group member role " <> B.unpack r
+
+serializeMemberRole :: GroupMemberRole -> ByteString
+serializeMemberRole = \case
+  GROwner -> "owner"
+  GRAdmin -> "admin"
+  GRMember -> "member"
+
+fromBlobField_ :: Typeable k => (ByteString -> Either String k) -> FieldParser k
+fromBlobField_ p = \case
+  f@(Field (SQLBlob b) _) ->
+    case p b of
+      Right k -> Ok k
+      Left e -> returnError ConversionFailed f ("could not parse field: " ++ e)
+  f -> returnError ConversionFailed f "expecting SQLBlob column type"
+
+data GroupMemberCategory
+  = GCUserMember
+  | GCInviteeMember -- member invited by the user
+  | GCHostMember -- member who invited the user
+  | GCPreMember -- member who joined before the user and was introduced to the user (user receives x.grp.mem.intro about such members)
+  | GCPostMember -- member who joined after the user to whom the user was introduced (user receives x.grp.mem.new announcing these members and then x.grp.mem.fwd with invitation from these members)
+  deriving (Eq, Show)
+
+instance FromField GroupMemberCategory where fromField = fromTextField_ memberCategoryT
+
+instance ToField GroupMemberCategory where toField = toField . serializeMemberCategory
+
+memberCategoryT :: Text -> Maybe GroupMemberCategory
+memberCategoryT = \case
+  "user" -> Just GCUserMember
+  "invitee" -> Just GCInviteeMember
+  "host" -> Just GCHostMember
+  "pre" -> Just GCPreMember
+  "post" -> Just GCPostMember
+  _ -> Nothing
+
+serializeMemberCategory :: GroupMemberCategory -> Text
+serializeMemberCategory = \case
+  GCUserMember -> "user"
+  GCInviteeMember -> "invitee"
+  GCHostMember -> "host"
+  GCPreMember -> "pre"
+  GCPostMember -> "post"
+
+data GroupMemberStatus
+  = GSMemRemoved -- member who was removed from the group
+  | GSMemLeft -- member who left the group
+  | GSMemGroupDeleted -- user member of the deleted group
+  | GSMemInvited -- member is sent to or received invitation to join the group
+  | GSMemIntroduced -- user received x.grp.mem.intro for this member (only with GCPreMember)
+  | GSMemIntroInvited -- member is sent to or received from intro invitation
+  | GSMemAccepted -- member accepted invitation (only User and Invitee)
+  | GSMemAnnounced -- host announced (x.grp.mem.new) a member (Invitee and PostMember) to the group - at this point this member can send messages and invite other members (if they have sufficient permissions)
+  | GSMemConnected -- member created the group connection with the inviting member
+  | GSMemComplete -- host confirmed (x.grp.mem.all) that a member (User, Invitee and PostMember) created group connections with all previous members
+  | GSMemCreator -- user member that created the group (only GCUserMember)
+  deriving (Eq, Show, Ord)
+
+instance FromField GroupMemberStatus where fromField = fromTextField_ memberStatusT
+
+instance ToField GroupMemberStatus where toField = toField . serializeMemberStatus
+
+memberActive :: GroupMember -> Bool
+memberActive m = case memberStatus m of
+  GSMemRemoved -> False
+  GSMemLeft -> False
+  GSMemGroupDeleted -> False
+  GSMemInvited -> False
+  GSMemIntroduced -> False
+  GSMemIntroInvited -> False
+  GSMemAccepted -> False
+  GSMemAnnounced -> False
+  GSMemConnected -> True
+  GSMemComplete -> True
+  GSMemCreator -> True
+
+memberCurrent :: GroupMember -> Bool
+memberCurrent m = case memberStatus m of
+  GSMemRemoved -> False
+  GSMemLeft -> False
+  GSMemGroupDeleted -> False
+  GSMemInvited -> False
+  GSMemIntroduced -> True
+  GSMemIntroInvited -> True
+  GSMemAccepted -> True
+  GSMemAnnounced -> True
+  GSMemConnected -> True
+  GSMemComplete -> True
+  GSMemCreator -> True
+
+memberStatusT :: Text -> Maybe GroupMemberStatus
+memberStatusT = \case
+  "removed" -> Just GSMemRemoved
+  "left" -> Just GSMemLeft
+  "deleted" -> Just GSMemGroupDeleted
+  "invited" -> Just GSMemInvited
+  "introduced" -> Just GSMemIntroduced
+  "intro-inv" -> Just GSMemIntroInvited
+  "accepted" -> Just GSMemAccepted
+  "announced" -> Just GSMemAnnounced
+  "connected" -> Just GSMemConnected
+  "complete" -> Just GSMemComplete
+  "creator" -> Just GSMemCreator
+  _ -> Nothing
+
+serializeMemberStatus :: GroupMemberStatus -> Text
+serializeMemberStatus = \case
+  GSMemRemoved -> "removed"
+  GSMemLeft -> "left"
+  GSMemGroupDeleted -> "deleted"
+  GSMemInvited -> "invited"
+  GSMemIntroduced -> "introduced"
+  GSMemIntroInvited -> "intro-inv"
+  GSMemAccepted -> "accepted"
+  GSMemAnnounced -> "announced"
+  GSMemConnected -> "connected"
+  GSMemComplete -> "complete"
+  GSMemCreator -> "creator"
+
+data SndFileTransfer = SndFileTransfer
+  { fileId :: Int64,
+    fileName :: String,
+    filePath :: String,
+    fileSize :: Integer,
+    chunkSize :: Integer,
+    recipientDisplayName :: ContactName,
+    connId :: Int64,
+    agentConnId :: ConnId,
+    fileStatus :: FileStatus
+  }
+  deriving (Eq, Show)
+
+data FileInvitation = FileInvitation
+  { fileName :: String,
+    fileSize :: Integer,
+    fileQInfo :: SMPQueueInfo
+  }
+  deriving (Eq, Show)
+
+data RcvFileTransfer = RcvFileTransfer
+  { fileId :: Int64,
+    fileInvitation :: FileInvitation,
+    fileStatus :: RcvFileStatus,
+    senderDisplayName :: ContactName,
+    chunkSize :: Integer
+  }
+  deriving (Eq, Show)
+
+data RcvFileStatus
+  = RFSNew
+  | RFSAccepted RcvFileInfo
+  | RFSConnected RcvFileInfo
+  | RFSComplete RcvFileInfo
+  | RFSCancelled RcvFileInfo
+  deriving (Eq, Show)
+
+data RcvFileInfo = RcvFileInfo
+  { filePath :: FilePath,
+    connId :: Int64,
+    agentConnId :: ConnId
+  }
+  deriving (Eq, Show)
+
+data FileTransfer = FTSnd [SndFileTransfer] | FTRcv RcvFileTransfer
+
+data FileStatus = FSNew | FSAccepted | FSConnected | FSComplete | FSCancelled deriving (Eq, Ord, Show)
+
+instance FromField FileStatus where fromField = fromTextField_ fileStatusT
+
+instance ToField FileStatus where toField = toField . serializeFileStatus
+
+fileStatusT :: Text -> Maybe FileStatus
+fileStatusT = \case
+  "new" -> Just FSNew
+  "accepted" -> Just FSAccepted
+  "connected" -> Just FSConnected
+  "complete" -> Just FSComplete
+  "cancelled" -> Just FSCancelled
+  _ -> Nothing
+
+serializeFileStatus :: FileStatus -> Text
+serializeFileStatus = \case
+  FSNew -> "new"
+  FSAccepted -> "accepted"
+  FSConnected -> "connected"
+  FSComplete -> "complete"
+  FSCancelled -> "cancelled"
+
+data RcvChunkStatus = RcvChunkOk | RcvChunkFinal | RcvChunkDuplicate | RcvChunkError
+  deriving (Eq, Show)
+
+data Connection = Connection
+  { connId :: Int64,
+    agentConnId :: ConnId,
+    connLevel :: Int,
+    viaContact :: Maybe Int64,
+    connType :: ConnType,
+    connStatus :: ConnStatus,
+    entityId :: Maybe Int64, -- contact, group member or file ID
+    createdAt :: UTCTime
+  }
+  deriving (Eq, Show)
+
+data ConnStatus
+  = -- | connection is created by initiating party with agent NEW command (createConnection)
+    ConnNew
+  | -- | connection is joined by joining party with agent JOIN command (joinConnection)
+    ConnJoined
+  | -- | initiating party received CONF notification (to be renamed to REQ)
+    ConnRequested
+  | -- | initiating party accepted connection with agent LET command (to be renamed to ACPT) (allowConnection)
+    ConnAccepted
+  | -- | connection can be sent messages to (after joining party received INFO notification)
+    ConnSndReady
+  | -- | connection is ready for both parties to send and receive messages
+    ConnReady
+  | -- | connection deleted
+    ConnDeleted
+  deriving (Eq, Show)
+
+instance FromField ConnStatus where fromField = fromTextField_ connStatusT
+
+instance ToField ConnStatus where toField = toField . serializeConnStatus
+
+connStatusT :: Text -> Maybe ConnStatus
+connStatusT = \case
+  "new" -> Just ConnNew
+  "joined" -> Just ConnJoined
+  "requested" -> Just ConnRequested
+  "accepted" -> Just ConnAccepted
+  "snd-ready" -> Just ConnSndReady
+  "ready" -> Just ConnReady
+  "deleted" -> Just ConnDeleted
+  _ -> Nothing
+
+serializeConnStatus :: ConnStatus -> Text
+serializeConnStatus = \case
+  ConnNew -> "new"
+  ConnJoined -> "joined"
+  ConnRequested -> "requested"
+  ConnAccepted -> "accepted"
+  ConnSndReady -> "snd-ready"
+  ConnReady -> "ready"
+  ConnDeleted -> "deleted"
+
+data ConnType = ConnContact | ConnMember | ConnSndFile | ConnRcvFile
+  deriving (Eq, Show)
+
+instance FromField ConnType where fromField = fromTextField_ connTypeT
+
+instance ToField ConnType where toField = toField . serializeConnType
+
+connTypeT :: Text -> Maybe ConnType
+connTypeT = \case
+  "contact" -> Just ConnContact
+  "member" -> Just ConnMember
+  "snd_file" -> Just ConnSndFile
+  "rcv_file" -> Just ConnRcvFile
+  _ -> Nothing
+
+serializeConnType :: ConnType -> Text
+serializeConnType = \case
+  ConnContact -> "contact"
+  ConnMember -> "member"
+  ConnSndFile -> "snd_file"
+  ConnRcvFile -> "rcv_file"
+
+data NewConnection = NewConnection
+  { agentConnId :: ByteString,
+    connLevel :: Int,
+    viaConn :: Maybe Int64
+  }
+
+data GroupMemberIntro = GroupMemberIntro
+  { introId :: Int64,
+    reMember :: GroupMember,
+    toMember :: GroupMember,
+    introStatus :: GroupMemberIntroStatus,
+    introInvitation :: Maybe IntroInvitation
+  }
+
+data GroupMemberIntroStatus
+  = GMIntroPending
+  | GMIntroSent
+  | GMIntroInvReceived
+  | GMIntroInvForwarded
+  | GMIntroReConnected
+  | GMIntroToConnected
+  | GMIntroConnected
+
+instance FromField GroupMemberIntroStatus where fromField = fromTextField_ introStatusT
+
+instance ToField GroupMemberIntroStatus where toField = toField . serializeIntroStatus
+
+introStatusT :: Text -> Maybe GroupMemberIntroStatus
+introStatusT = \case
+  "new" -> Just GMIntroPending
+  "sent" -> Just GMIntroSent
+  "rcv" -> Just GMIntroInvReceived
+  "fwd" -> Just GMIntroInvForwarded
+  "re-con" -> Just GMIntroReConnected
+  "to-con" -> Just GMIntroToConnected
+  "con" -> Just GMIntroConnected
+  _ -> Nothing
+
+serializeIntroStatus :: GroupMemberIntroStatus -> Text
+serializeIntroStatus = \case
+  GMIntroPending -> "new"
+  GMIntroSent -> "sent"
+  GMIntroInvReceived -> "rcv"
+  GMIntroInvForwarded -> "fwd"
+  GMIntroReConnected -> "re-con"
+  GMIntroToConnected -> "to-con"
+  GMIntroConnected -> "con"

src/Simplex/Chat/Util.hs

@@ -0,0 +1,16 @@
+module Simplex.Chat.Util where
+
+import Data.ByteString.Char8 (ByteString)
+import Data.Text (Text)
+import Data.Text.Encoding (decodeUtf8With)
+
+safeDecodeUtf8 :: ByteString -> Text
+safeDecodeUtf8 = decodeUtf8With onError
+  where
+    onError _ _ = Just '?'
+
+ifM :: Monad m => m Bool -> m a -> m a -> m a
+ifM ba t f = ba >>= \b -> if b then t else f
+
+unlessM :: Monad m => m Bool -> m () -> m ()
+unlessM b = ifM b $ pure ()

src/Simplex/Chat/View.hs

@@ -0,0 +1,687 @@
+{-# LANGUAGE ConstraintKinds #-}
+{-# LANGUAGE DuplicateRecordFields #-}
+{-# LANGUAGE FlexibleContexts #-}
+{-# LANGUAGE LambdaCase #-}
+{-# LANGUAGE NamedFieldPuns #-}
+{-# LANGUAGE OverloadedStrings #-}
+
+module Simplex.Chat.View
+  ( printToView,
+    showInvitation,
+    showChatError,
+    showContactDeleted,
+    showContactGroups,
+    showContactConnected,
+    showContactDisconnected,
+    showContactAnotherClient,
+    showContactSubscribed,
+    showContactSubError,
+    showGroupSubscribed,
+    showGroupEmpty,
+    showGroupRemoved,
+    showMemberSubError,
+    showReceivedMessage,
+    showReceivedGroupMessage,
+    showSentMessage,
+    showSentGroupMessage,
+    showSentFileInvitation,
+    showSentGroupFileInvitation,
+    showSentFileInfo,
+    showSndFileStart,
+    showSndFileComplete,
+    showSndFileCancelled,
+    showSndGroupFileCancelled,
+    showSndFileRcvCancelled,
+    receivedFileInvitation,
+    showRcvFileAccepted,
+    showRcvFileStart,
+    showRcvFileComplete,
+    showRcvFileCancelled,
+    showRcvFileSndCancelled,
+    showFileTransferStatus,
+    showSndFileSubError,
+    showRcvFileSubError,
+    showGroupCreated,
+    showGroupDeletedUser,
+    showGroupDeleted,
+    showSentGroupInvitation,
+    showReceivedGroupInvitation,
+    showJoinedGroupMember,
+    showUserJoinedGroup,
+    showJoinedGroupMemberConnecting,
+    showConnectedToGroupMember,
+    showDeletedMember,
+    showDeletedMemberUser,
+    showLeftMemberUser,
+    showLeftMember,
+    showGroupMembers,
+    showContactsMerged,
+    showUserProfile,
+    showUserProfileUpdated,
+    showContactUpdated,
+    showMessageError,
+    safeDecodeUtf8,
+    msgPlain,
+  )
+where
+
+import Control.Monad.IO.Unlift
+import Control.Monad.Reader
+import Data.ByteString.Char8 (ByteString)
+import Data.Composition ((.:), (.:.))
+import Data.Function (on)
+import Data.Int (Int64)
+import Data.List (groupBy, intersperse, sortOn)
+import Data.Text (Text)
+import qualified Data.Text as T
+import Data.Time.Clock (DiffTime, UTCTime)
+import Data.Time.Format (defaultTimeLocale, formatTime)
+import Data.Time.LocalTime (TimeZone, ZonedTime, getCurrentTimeZone, getZonedTime, localDay, localTimeOfDay, timeOfDayToTime, utcToLocalTime, zonedTimeToLocalTime)
+import Numeric (showFFloat)
+import Simplex.Chat.Controller
+import Simplex.Chat.Markdown
+import Simplex.Chat.Store (StoreError (..))
+import Simplex.Chat.Styled
+import Simplex.Chat.Terminal (printToTerminal)
+import Simplex.Chat.Types
+import Simplex.Chat.Util (safeDecodeUtf8)
+import Simplex.Messaging.Agent.Protocol
+import System.Console.ANSI.Types
+
+type ChatReader m = (MonadUnliftIO m, MonadReader ChatController m)
+
+showInvitation :: ChatReader m => SMPQueueInfo -> m ()
+showInvitation = printToView . invitation
+
+showChatError :: ChatReader m => ChatError -> m ()
+showChatError = printToView . chatError
+
+showContactDeleted :: ChatReader m => ContactName -> m ()
+showContactDeleted = printToView . contactDeleted
+
+showContactGroups :: ChatReader m => ContactName -> [GroupName] -> m ()
+showContactGroups = printToView .: contactGroups
+
+showContactConnected :: ChatReader m => Contact -> m ()
+showContactConnected = printToView . contactConnected
+
+showContactDisconnected :: ChatReader m => ContactName -> m ()
+showContactDisconnected = printToView . contactDisconnected
+
+showContactAnotherClient :: ChatReader m => ContactName -> m ()
+showContactAnotherClient = printToView . contactAnotherClient
+
+showContactSubscribed :: ChatReader m => ContactName -> m ()
+showContactSubscribed = printToView . contactSubscribed
+
+showContactSubError :: ChatReader m => ContactName -> ChatError -> m ()
+showContactSubError = printToView .: contactSubError
+
+showGroupSubscribed :: ChatReader m => GroupName -> m ()
+showGroupSubscribed = printToView . groupSubscribed
+
+showGroupEmpty :: ChatReader m => GroupName -> m ()
+showGroupEmpty = printToView . groupEmpty
+
+showGroupRemoved :: ChatReader m => GroupName -> m ()
+showGroupRemoved = printToView . groupRemoved
+
+showMemberSubError :: ChatReader m => GroupName -> ContactName -> ChatError -> m ()
+showMemberSubError = printToView .:. memberSubError
+
+showReceivedMessage :: ChatReader m => ContactName -> UTCTime -> [StyledString] -> MsgIntegrity -> m ()
+showReceivedMessage = showReceivedMessage_ . ttyFromContact
+
+showReceivedGroupMessage :: ChatReader m => GroupName -> ContactName -> UTCTime -> [StyledString] -> MsgIntegrity -> m ()
+showReceivedGroupMessage = showReceivedMessage_ .: ttyFromGroup
+
+showReceivedMessage_ :: ChatReader m => StyledString -> UTCTime -> [StyledString] -> MsgIntegrity -> m ()
+showReceivedMessage_ from utcTime msg mOk = printToView =<< liftIO (receivedMessage from utcTime msg mOk)
+
+showSentMessage :: ChatReader m => ContactName -> ByteString -> m ()
+showSentMessage = showSentMessage_ . ttyToContact
+
+showSentGroupMessage :: ChatReader m => GroupName -> ByteString -> m ()
+showSentGroupMessage = showSentMessage_ . ttyToGroup
+
+showSentMessage_ :: ChatReader m => StyledString -> ByteString -> m ()
+showSentMessage_ to msg = printToView =<< liftIO (sentMessage to msg)
+
+showSentFileInvitation :: ChatReader m => ContactName -> FilePath -> m ()
+showSentFileInvitation = showSentFileInvitation_ . ttyToContact
+
+showSentGroupFileInvitation :: ChatReader m => GroupName -> FilePath -> m ()
+showSentGroupFileInvitation = showSentFileInvitation_ . ttyToGroup
+
+showSentFileInvitation_ :: ChatReader m => StyledString -> FilePath -> m ()
+showSentFileInvitation_ to filePath = printToView =<< liftIO (sentFileInvitation to filePath)
+
+showSentFileInfo :: ChatReader m => Int64 -> m ()
+showSentFileInfo = printToView . sentFileInfo
+
+showSndFileStart :: ChatReader m => SndFileTransfer -> m ()
+showSndFileStart = printToView . sndFileStart
+
+showSndFileComplete :: ChatReader m => SndFileTransfer -> m ()
+showSndFileComplete = printToView . sndFileComplete
+
+showSndFileCancelled :: ChatReader m => SndFileTransfer -> m ()
+showSndFileCancelled = printToView . sndFileCancelled
+
+showSndGroupFileCancelled :: ChatReader m => [SndFileTransfer] -> m ()
+showSndGroupFileCancelled = printToView . sndGroupFileCancelled
+
+showSndFileRcvCancelled :: ChatReader m => SndFileTransfer -> m ()
+showSndFileRcvCancelled = printToView . sndFileRcvCancelled
+
+showRcvFileAccepted :: ChatReader m => RcvFileTransfer -> FilePath -> m ()
+showRcvFileAccepted = printToView .: rcvFileAccepted
+
+showRcvFileStart :: ChatReader m => RcvFileTransfer -> m ()
+showRcvFileStart = printToView . rcvFileStart
+
+showRcvFileComplete :: ChatReader m => RcvFileTransfer -> m ()
+showRcvFileComplete = printToView . rcvFileComplete
+
+showRcvFileCancelled :: ChatReader m => RcvFileTransfer -> m ()
+showRcvFileCancelled = printToView . rcvFileCancelled
+
+showRcvFileSndCancelled :: ChatReader m => RcvFileTransfer -> m ()
+showRcvFileSndCancelled = printToView . rcvFileSndCancelled
+
+showFileTransferStatus :: ChatReader m => (FileTransfer, [Integer]) -> m ()
+showFileTransferStatus = printToView . fileTransferStatus
+
+showSndFileSubError :: ChatReader m => SndFileTransfer -> ChatError -> m ()
+showSndFileSubError = printToView .: sndFileSubError
+
+showRcvFileSubError :: ChatReader m => RcvFileTransfer -> ChatError -> m ()
+showRcvFileSubError = printToView .: rcvFileSubError
+
+showGroupCreated :: ChatReader m => Group -> m ()
+showGroupCreated = printToView . groupCreated
+
+showGroupDeletedUser :: ChatReader m => GroupName -> m ()
+showGroupDeletedUser = printToView . groupDeletedUser
+
+showGroupDeleted :: ChatReader m => GroupName -> GroupMember -> m ()
+showGroupDeleted = printToView .: groupDeleted
+
+showSentGroupInvitation :: ChatReader m => GroupName -> ContactName -> m ()
+showSentGroupInvitation = printToView .: sentGroupInvitation
+
+showReceivedGroupInvitation :: ChatReader m => Group -> ContactName -> GroupMemberRole -> m ()
+showReceivedGroupInvitation = printToView .:. receivedGroupInvitation
+
+showJoinedGroupMember :: ChatReader m => GroupName -> GroupMember -> m ()
+showJoinedGroupMember = printToView .: joinedGroupMember
+
+showUserJoinedGroup :: ChatReader m => GroupName -> m ()
+showUserJoinedGroup = printToView . userJoinedGroup
+
+showJoinedGroupMemberConnecting :: ChatReader m => GroupName -> GroupMember -> GroupMember -> m ()
+showJoinedGroupMemberConnecting = printToView .:. joinedGroupMemberConnecting
+
+showConnectedToGroupMember :: ChatReader m => GroupName -> GroupMember -> m ()
+showConnectedToGroupMember = printToView .: connectedToGroupMember
+
+showDeletedMember :: ChatReader m => GroupName -> Maybe GroupMember -> Maybe GroupMember -> m ()
+showDeletedMember = printToView .:. deletedMember
+
+showDeletedMemberUser :: ChatReader m => GroupName -> GroupMember -> m ()
+showDeletedMemberUser = printToView .: deletedMemberUser
+
+showLeftMemberUser :: ChatReader m => GroupName -> m ()
+showLeftMemberUser = printToView . leftMemberUser
+
+showLeftMember :: ChatReader m => GroupName -> GroupMember -> m ()
+showLeftMember = printToView .: leftMember
+
+showGroupMembers :: ChatReader m => Group -> m ()
+showGroupMembers = printToView . groupMembers
+
+showContactsMerged :: ChatReader m => Contact -> Contact -> m ()
+showContactsMerged = printToView .: contactsMerged
+
+showUserProfile :: ChatReader m => Profile -> m ()
+showUserProfile = printToView . userProfile
+
+showUserProfileUpdated :: ChatReader m => User -> User -> m ()
+showUserProfileUpdated = printToView .: userProfileUpdated
+
+showContactUpdated :: ChatReader m => Contact -> Contact -> m ()
+showContactUpdated = printToView .: contactUpdated
+
+showMessageError :: ChatReader m => Text -> Text -> m ()
+showMessageError = printToView .: messageError
+
+invitation :: SMPQueueInfo -> [StyledString]
+invitation qInfo =
+  [ "pass this invitation to your contact (via another channel): ",
+    "",
+    (plain . serializeSmpQueueInfo) qInfo,
+    "",
+    "and ask them to connect: " <> highlight' "/c <invitation_above>"
+  ]
+
+contactDeleted :: ContactName -> [StyledString]
+contactDeleted c = [ttyContact c <> ": contact is deleted"]
+
+contactGroups :: ContactName -> [GroupName] -> [StyledString]
+contactGroups c gNames = [ttyContact c <> ": contact cannot be deleted, it is a member of the group(s) " <> ttyGroups gNames]
+  where
+    ttyGroups :: [GroupName] -> StyledString
+    ttyGroups [] = ""
+    ttyGroups [g] = ttyGroup g
+    ttyGroups (g : gs) = ttyGroup g <> ", " <> ttyGroups gs
+
+contactConnected :: Contact -> [StyledString]
+contactConnected ct = [ttyFullContact ct <> ": contact is connected"]
+
+contactDisconnected :: ContactName -> [StyledString]
+contactDisconnected c = [ttyContact c <> ": disconnected from server (messages will be queued)"]
+
+contactAnotherClient :: ContactName -> [StyledString]
+contactAnotherClient c = [ttyContact c <> ": contact is connected to another client"]
+
+contactSubscribed :: ContactName -> [StyledString]
+contactSubscribed c = [ttyContact c <> ": connected to server"]
+
+contactSubError :: ContactName -> ChatError -> [StyledString]
+contactSubError c e = [ttyContact c <> ": contact error " <> sShow e]
+
+groupSubscribed :: GroupName -> [StyledString]
+groupSubscribed g = [ttyGroup g <> ": connected to server(s)"]
+
+groupEmpty :: GroupName -> [StyledString]
+groupEmpty g = [ttyGroup g <> ": group is empty"]
+
+groupRemoved :: GroupName -> [StyledString]
+groupRemoved g = [ttyGroup g <> ": you are no longer a member or group deleted"]
+
+memberSubError :: GroupName -> ContactName -> ChatError -> [StyledString]
+memberSubError g c e = [ttyGroup g <> " member " <> ttyContact c <> " error: " <> sShow e]
+
+groupCreated :: Group -> [StyledString]
+groupCreated g@Group {localDisplayName} =
+  [ "group " <> ttyFullGroup g <> " is created",
+    "use " <> highlight ("/a " <> localDisplayName <> " <name>") <> " to add members"
+  ]
+
+groupDeletedUser :: GroupName -> [StyledString]
+groupDeletedUser g = groupDeleted_ g Nothing
+
+groupDeleted :: GroupName -> GroupMember -> [StyledString]
+groupDeleted g m = groupDeleted_ g (Just m) <> ["use " <> highlight ("/d #" <> g) <> " to delete the local copy of the group"]
+
+groupDeleted_ :: GroupName -> Maybe GroupMember -> [StyledString]
+groupDeleted_ g m = [ttyGroup g <> ": " <> memberOrUser m <> " deleted the group"]
+
+sentGroupInvitation :: GroupName -> ContactName -> [StyledString]
+sentGroupInvitation g c = ["invitation to join the group " <> ttyGroup g <> " sent to " <> ttyContact c]
+
+receivedGroupInvitation :: Group -> ContactName -> GroupMemberRole -> [StyledString]
+receivedGroupInvitation g@Group {localDisplayName} c role =
+  [ ttyFullGroup g <> ": " <> ttyContact c <> " invites you to join the group as " <> plain (serializeMemberRole role),
+    "use " <> highlight ("/j " <> localDisplayName) <> " to accept"
+  ]
+
+joinedGroupMember :: GroupName -> GroupMember -> [StyledString]
+joinedGroupMember g m = [ttyGroup g <> ": " <> ttyMember m <> " joined the group "]
+
+userJoinedGroup :: GroupName -> [StyledString]
+userJoinedGroup g = [ttyGroup g <> ": you joined the group"]
+
+joinedGroupMemberConnecting :: GroupName -> GroupMember -> GroupMember -> [StyledString]
+joinedGroupMemberConnecting g host m = [ttyGroup g <> ": " <> ttyMember host <> " added " <> ttyFullMember m <> " to the group (connecting...)"]
+
+connectedToGroupMember :: GroupName -> GroupMember -> [StyledString]
+connectedToGroupMember g m = [ttyGroup g <> ": " <> connectedMember m <> " is connected"]
+
+deletedMember :: GroupName -> Maybe GroupMember -> Maybe GroupMember -> [StyledString]
+deletedMember g by m = [ttyGroup g <> ": " <> memberOrUser by <> " removed " <> memberOrUser m <> " from the group"]
+
+deletedMemberUser :: GroupName -> GroupMember -> [StyledString]
+deletedMemberUser g by = deletedMember g (Just by) Nothing <> groupPreserved g
+
+leftMemberUser :: GroupName -> [StyledString]
+leftMemberUser g = leftMember_ g Nothing <> groupPreserved g
+
+leftMember :: GroupName -> GroupMember -> [StyledString]
+leftMember g m = leftMember_ g (Just m)
+
+leftMember_ :: GroupName -> Maybe GroupMember -> [StyledString]
+leftMember_ g m = [ttyGroup g <> ": " <> memberOrUser m <> " left the group"]
+
+groupPreserved :: GroupName -> [StyledString]
+groupPreserved g = ["use " <> highlight ("/d #" <> g) <> " to delete the group"]
+
+memberOrUser :: Maybe GroupMember -> StyledString
+memberOrUser = maybe "you" ttyMember
+
+connectedMember :: GroupMember -> StyledString
+connectedMember m = case memberCategory m of
+  GCPreMember -> "member " <> ttyFullMember m
+  GCPostMember -> "new member " <> ttyMember m -- without fullName as as it was shown in joinedGroupMemberConnecting
+  _ -> "member " <> ttyMember m -- these case is not used
+
+groupMembers :: Group -> [StyledString]
+groupMembers Group {membership, members} = map groupMember . filter (not . removedOrLeft) $ membership : members
+  where
+    removedOrLeft m = let s = memberStatus m in s == GSMemRemoved || s == GSMemLeft
+    groupMember m = ttyFullMember m <> ": " <> role m <> ", " <> category m <> status m
+    role = plain . serializeMemberRole . memberRole
+    category m = case memberCategory m of
+      GCUserMember -> "you, "
+      GCInviteeMember -> "invited, "
+      GCHostMember -> "host, "
+      _ -> ""
+    status m = case memberStatus m of
+      GSMemRemoved -> "removed"
+      GSMemLeft -> "left"
+      GSMemInvited -> "not yet joined"
+      GSMemConnected -> "connected"
+      GSMemComplete -> "connected"
+      GSMemCreator -> "created group"
+      _ -> ""
+
+contactsMerged :: Contact -> Contact -> [StyledString]
+contactsMerged _to@Contact {localDisplayName = c1} _from@Contact {localDisplayName = c2} =
+  [ "contact " <> ttyContact c2 <> " is merged into " <> ttyContact c1,
+    "use " <> ttyToContact c1 <> highlight' "<message>" <> " to send messages"
+  ]
+
+userProfile :: Profile -> [StyledString]
+userProfile Profile {displayName, fullName} =
+  [ "user profile: " <> ttyFullName displayName fullName,
+    "use " <> highlight' "/p <display name> [<full name>]" <> " to change it",
+    "(the updated profile will be sent to all your contacts)"
+  ]
+
+userProfileUpdated :: User -> User -> [StyledString]
+userProfileUpdated
+  User {localDisplayName = n, profile = Profile {fullName}}
+  User {localDisplayName = n', profile = Profile {fullName = fullName'}}
+    | n == n' && fullName == fullName' = []
+    | n == n' = ["user full name " <> (if T.null fullName' || fullName' == n' then "removed" else "changed to " <> plain fullName') <> notified]
+    | otherwise = ["user profile is changed to " <> ttyFullName n' fullName' <> notified]
+    where
+      notified = " (your contacts are notified)"
+
+contactUpdated :: Contact -> Contact -> [StyledString]
+contactUpdated
+  Contact {localDisplayName = n, profile = Profile {fullName}}
+  Contact {localDisplayName = n', profile = Profile {fullName = fullName'}}
+    | n == n' && fullName == fullName' = []
+    | n == n' = ["contact " <> ttyContact n <> fullNameUpdate]
+    | otherwise =
+      [ "contact " <> ttyContact n <> " changed to " <> ttyFullName n' fullName',
+        "use " <> ttyToContact n' <> highlight' "<message>" <> " to send messages"
+      ]
+    where
+      fullNameUpdate = if T.null fullName' || fullName' == n' then " removed full name" else " updated full name: " <> plain fullName'
+
+messageError :: Text -> Text -> [StyledString]
+messageError prefix err = [plain prefix <> ": " <> plain err]
+
+receivedMessage :: StyledString -> UTCTime -> [StyledString] -> MsgIntegrity -> IO [StyledString]
+receivedMessage from utcTime msg mOk = do
+  t <- formatUTCTime <$> getCurrentTimeZone <*> getZonedTime
+  pure $ prependFirst (t <> " " <> from) msg ++ showIntegrity mOk
+  where
+    formatUTCTime :: TimeZone -> ZonedTime -> StyledString
+    formatUTCTime localTz currentTime =
+      let localTime = utcToLocalTime localTz utcTime
+          format =
+            if (localDay localTime < localDay (zonedTimeToLocalTime currentTime))
+              && (timeOfDayToTime (localTimeOfDay localTime) > (6 * 60 * 60 :: DiffTime))
+              then "%m-%d" -- if message is from yesterday or before and 6 hours has passed since midnight
+              else "%H:%M"
+       in styleTime $ formatTime defaultTimeLocale format localTime
+    showIntegrity :: MsgIntegrity -> [StyledString]
+    showIntegrity MsgOk = []
+    showIntegrity (MsgError err) = msgError $ case err of
+      MsgSkipped fromId toId ->
+        "skipped message ID " <> show fromId
+          <> if fromId == toId then "" else ".." <> show toId
+      MsgBadId msgId -> "unexpected message ID " <> show msgId
+      MsgBadHash -> "incorrect message hash"
+      MsgDuplicate -> "duplicate message ID"
+    msgError :: String -> [StyledString]
+    msgError s = [styled (Colored Red) s]
+
+sentMessage :: StyledString -> ByteString -> IO [StyledString]
+sentMessage to msg = sendWithTime_ to . msgPlain $ safeDecodeUtf8 msg
+
+sentFileInvitation :: StyledString -> FilePath -> IO [StyledString]
+sentFileInvitation to f = sendWithTime_ ("/f " <> to) [ttyFilePath f]
+
+sendWithTime_ :: StyledString -> [StyledString] -> IO [StyledString]
+sendWithTime_ to styledMsg = do
+  time <- formatTime defaultTimeLocale "%H:%M" <$> getZonedTime
+  pure $ prependFirst (styleTime time <> " " <> to) styledMsg
+
+prependFirst :: StyledString -> [StyledString] -> [StyledString]
+prependFirst s [] = [s]
+prependFirst s (s' : ss) = (s <> s') : ss
+
+msgPlain :: Text -> [StyledString]
+msgPlain = map styleMarkdownText . T.lines
+
+sentFileInfo :: Int64 -> [StyledString]
+sentFileInfo fileId =
+  ["use " <> highlight ("/fc " <> show fileId) <> " to cancel sending"]
+
+sndFileStart :: SndFileTransfer -> [StyledString]
+sndFileStart = sendingFile_ "started"
+
+sndFileComplete :: SndFileTransfer -> [StyledString]
+sndFileComplete = sendingFile_ "completed"
+
+sndFileCancelled :: SndFileTransfer -> [StyledString]
+sndFileCancelled = sendingFile_ "cancelled"
+
+sndGroupFileCancelled :: [SndFileTransfer] -> [StyledString]
+sndGroupFileCancelled fts =
+  case filter (\SndFileTransfer {fileStatus = s} -> s /= FSCancelled && s /= FSComplete) fts of
+    [] -> ["sending file can't be cancelled"]
+    ts@(ft : _) -> ["cancelled sending " <> sndFile ft <> " to " <> listMembers ts]
+
+sendingFile_ :: StyledString -> SndFileTransfer -> [StyledString]
+sendingFile_ status ft@SndFileTransfer {recipientDisplayName = c} =
+  [status <> " sending " <> sndFile ft <> " to " <> ttyContact c]
+
+sndFileRcvCancelled :: SndFileTransfer -> [StyledString]
+sndFileRcvCancelled ft@SndFileTransfer {recipientDisplayName = c} =
+  [ttyContact c <> " cancelled receiving " <> sndFile ft]
+
+sndFile :: SndFileTransfer -> StyledString
+sndFile SndFileTransfer {fileId, fileName} = fileTransfer fileId fileName
+
+receivedFileInvitation :: RcvFileTransfer -> [StyledString]
+receivedFileInvitation RcvFileTransfer {fileId, fileInvitation = FileInvitation {fileName, fileSize}} =
+  [ "sends file " <> ttyFilePath fileName <> " (" <> humanReadableSize fileSize <> " / " <> sShow fileSize <> " bytes)",
+    "use " <> highlight ("/fr " <> show fileId <> " [<dir>/ | <path>]") <> " to receive it"
+  ]
+
+humanReadableSize :: Integer -> StyledString
+humanReadableSize size
+  | size < kB = sShow size <> " bytes"
+  | size < mB = hrSize kB "KiB"
+  | size < gB = hrSize mB "MiB"
+  | otherwise = hrSize gB "GiB"
+  where
+    hrSize sB name = plain $ unwords [showFFloat (Just 1) (fromIntegral size / (fromIntegral sB :: Double)) "", name]
+    kB = 1024
+    mB = kB * 1024
+    gB = mB * 1024
+
+rcvFileAccepted :: RcvFileTransfer -> FilePath -> [StyledString]
+rcvFileAccepted RcvFileTransfer {fileId, senderDisplayName = c} filePath =
+  ["saving file " <> sShow fileId <> " from " <> ttyContact c <> " to " <> plain filePath]
+
+rcvFileStart :: RcvFileTransfer -> [StyledString]
+rcvFileStart = receivingFile_ "started"
+
+rcvFileComplete :: RcvFileTransfer -> [StyledString]
+rcvFileComplete = receivingFile_ "completed"
+
+rcvFileCancelled :: RcvFileTransfer -> [StyledString]
+rcvFileCancelled = receivingFile_ "cancelled"
+
+receivingFile_ :: StyledString -> RcvFileTransfer -> [StyledString]
+receivingFile_ status ft@RcvFileTransfer {senderDisplayName = c} =
+  [status <> " receiving " <> rcvFile ft <> " from " <> ttyContact c]
+
+rcvFileSndCancelled :: RcvFileTransfer -> [StyledString]
+rcvFileSndCancelled ft@RcvFileTransfer {senderDisplayName = c} =
+  [ttyContact c <> " cancelled sending " <> rcvFile ft]
+
+rcvFile :: RcvFileTransfer -> StyledString
+rcvFile RcvFileTransfer {fileId, fileInvitation = FileInvitation {fileName}} = fileTransfer fileId fileName
+
+fileTransfer :: Int64 -> String -> StyledString
+fileTransfer fileId fileName = "file " <> sShow fileId <> " (" <> ttyFilePath fileName <> ")"
+
+fileTransferStatus :: (FileTransfer, [Integer]) -> [StyledString]
+fileTransferStatus (FTSnd [ft@SndFileTransfer {fileStatus, fileSize, chunkSize}], chunksNum) =
+  ["sending " <> sndFile ft <> " " <> sndStatus]
+  where
+    sndStatus = case fileStatus of
+      FSNew -> "not accepted yet"
+      FSAccepted -> "just started"
+      FSConnected -> "progress " <> fileProgress chunksNum chunkSize fileSize
+      FSComplete -> "complete"
+      FSCancelled -> "cancelled"
+fileTransferStatus (FTSnd [], _) = ["no file transfers (empty group)"]
+fileTransferStatus (FTSnd fts@(ft : _), chunksNum) =
+  case concatMap membersTransferStatus $ groupBy ((==) `on` fs) $ sortOn fs fts of
+    [membersStatus] -> ["sending " <> sndFile ft <> " " <> membersStatus]
+    membersStatuses -> ("sending " <> sndFile ft <> ": ") : map ("  " <>) membersStatuses
+  where
+    fs = fileStatus :: SndFileTransfer -> FileStatus
+    membersTransferStatus [] = []
+    membersTransferStatus ts@(SndFileTransfer {fileStatus, fileSize, chunkSize} : _) = [sndStatus <> ": " <> listMembers ts]
+      where
+        sndStatus = case fileStatus of
+          FSNew -> "not accepted"
+          FSAccepted -> "just started"
+          FSConnected -> "in progress (" <> sShow (sum chunksNum * chunkSize * 100 `div` (toInteger (length chunksNum) * fileSize)) <> "%)"
+          FSComplete -> "complete"
+          FSCancelled -> "cancelled"
+fileTransferStatus (FTRcv ft@RcvFileTransfer {fileId, fileInvitation = FileInvitation {fileSize}, fileStatus, chunkSize}, chunksNum) =
+  ["receiving " <> rcvFile ft <> " " <> rcvStatus]
+  where
+    rcvStatus = case fileStatus of
+      RFSNew -> "not accepted yet, use " <> highlight ("/fr " <> show fileId) <> " to receive file"
+      RFSAccepted _ -> "just started"
+      RFSConnected _ -> "progress " <> fileProgress chunksNum chunkSize fileSize
+      RFSComplete RcvFileInfo {filePath} -> "complete, path: " <> plain filePath
+      RFSCancelled RcvFileInfo {filePath} -> "cancelled, received part path: " <> plain filePath
+
+listMembers :: [SndFileTransfer] -> StyledString
+listMembers = mconcat . intersperse ", " . map (ttyContact . recipientDisplayName)
+
+fileProgress :: [Integer] -> Integer -> Integer -> StyledString
+fileProgress chunksNum chunkSize fileSize =
+  sShow (sum chunksNum * chunkSize * 100 `div` fileSize) <> "% of " <> humanReadableSize fileSize
+
+sndFileSubError :: SndFileTransfer -> ChatError -> [StyledString]
+sndFileSubError SndFileTransfer {fileId, fileName} e =
+  ["sent file " <> sShow fileId <> " (" <> plain fileName <> ") error: " <> sShow e]
+
+rcvFileSubError :: RcvFileTransfer -> ChatError -> [StyledString]
+rcvFileSubError RcvFileTransfer {fileId, fileInvitation = FileInvitation {fileName}} e =
+  ["received file " <> sShow fileId <> " (" <> plain fileName <> ") error: " <> sShow e]
+
+chatError :: ChatError -> [StyledString]
+chatError = \case
+  ChatError err -> case err of
+    CEGroupDuplicateMember c -> ["contact " <> ttyContact c <> " is already in the group"]
+    CEGroupDuplicateMemberId -> ["cannot add member - duplicate member ID"]
+    CEGroupUserRole -> ["you have insufficient permissions for this group command"]
+    CEGroupContactRole c -> ["contact " <> ttyContact c <> " has insufficient permissions for this group action"]
+    CEGroupNotJoined g -> ["you did not join this group, use " <> highlight ("/join #" <> g)]
+    CEGroupMemberNotActive -> ["you cannot invite other members yet, try later"]
+    CEGroupMemberUserRemoved -> ["you are no longer the member of the group"]
+    CEGroupMemberNotFound c -> ["contact " <> ttyContact c <> " is not a group member"]
+    CEGroupInternal s -> ["chat group bug: " <> plain s]
+    CEFileNotFound f -> ["file not found: " <> plain f]
+    CEFileAlreadyReceiving f -> ["file is already accepted: " <> plain f]
+    CEFileAlreadyExists f -> ["file already exists: " <> plain f]
+    CEFileRead f e -> ["cannot read file " <> plain f, sShow e]
+    CEFileWrite f e -> ["cannot write file " <> plain f, sShow e]
+    CEFileSend fileId e -> ["error sending file " <> sShow fileId <> ": " <> sShow e]
+    CEFileRcvChunk e -> ["error receiving file: " <> plain e]
+    CEFileInternal e -> ["file error: " <> plain e]
+  -- e -> ["chat error: " <> sShow e]
+  ChatErrorStore err -> case err of
+    SEDuplicateName -> ["this display name is already used by user, contact or group"]
+    SEContactNotFound c -> ["no contact " <> ttyContact c]
+    SEContactNotReady c -> ["contact " <> ttyContact c <> " is not active yet"]
+    SEGroupNotFound g -> ["no group " <> ttyGroup g]
+    SEGroupAlreadyJoined -> ["you already joined this group"]
+    SEFileNotFound fileId -> fileNotFound fileId
+    SESndFileNotFound fileId -> fileNotFound fileId
+    SERcvFileNotFound fileId -> fileNotFound fileId
+    e -> ["chat db error: " <> sShow e]
+  ChatErrorAgent e -> ["smp agent error: " <> sShow e]
+  ChatErrorMessage e -> ["chat message error: " <> sShow e]
+  where
+    fileNotFound fileId = ["file " <> sShow fileId <> " not found"]
+
+printToView :: (MonadUnliftIO m, MonadReader ChatController m) => [StyledString] -> m ()
+printToView s = asks chatTerminal >>= liftIO . (`printToTerminal` s)
+
+ttyContact :: ContactName -> StyledString
+ttyContact = styled (Colored Green)
+
+ttyFullContact :: Contact -> StyledString
+ttyFullContact Contact {localDisplayName, profile = Profile {fullName}} =
+  ttyFullName localDisplayName fullName
+
+ttyMember :: GroupMember -> StyledString
+ttyMember GroupMember {localDisplayName} = ttyContact localDisplayName
+
+ttyFullMember :: GroupMember -> StyledString
+ttyFullMember GroupMember {localDisplayName, memberProfile = Profile {fullName}} =
+  ttyFullName localDisplayName fullName
+
+ttyFullName :: ContactName -> Text -> StyledString
+ttyFullName c fullName = ttyContact c <> optFullName c fullName
+
+ttyToContact :: ContactName -> StyledString
+ttyToContact c = styled (Colored Cyan) $ "@" <> c <> " "
+
+ttyFromContact :: ContactName -> StyledString
+ttyFromContact c = styled (Colored Yellow) $ c <> "> "
+
+ttyGroup :: GroupName -> StyledString
+ttyGroup g = styled (Colored Blue) $ "#" <> g
+
+ttyFullGroup :: Group -> StyledString
+ttyFullGroup Group {localDisplayName, groupProfile = GroupProfile {fullName}} =
+  ttyGroup localDisplayName <> optFullName localDisplayName fullName
+
+ttyFromGroup :: GroupName -> ContactName -> StyledString
+ttyFromGroup g c = styled (Colored Yellow) $ "#" <> g <> " " <> c <> "> "
+
+ttyToGroup :: GroupName -> StyledString
+ttyToGroup g = styled (Colored Cyan) $ "#" <> g <> " "
+
+ttyFilePath :: FilePath -> StyledString
+ttyFilePath = plain
+
+optFullName :: ContactName -> Text -> StyledString
+optFullName localDisplayName fullName
+  | T.null fullName || localDisplayName == fullName = ""
+  | otherwise = plain (" (" <> fullName <> ")")
+
+highlight :: StyledFormat a => a -> StyledString
+highlight = styled (Colored Cyan)
+
+highlight' :: String -> StyledString
+highlight' = highlight
+
+styleTime :: String -> StyledString
+styleTime = Styled [SetColor Foreground Vivid Black]

src/Simplex/Chat/protocol.md

@@ -0,0 +1,168 @@
+# Chat protocol
+
+## Design constraints
+
+- the transport message has a fixed size (8 or 16kb), but the SMP agent will be updated to support sending messages up to maximum configured size (TBC - 64-256kb) in 8-16Kb blocks.
+- the chat message can have multiple content parts, but it should fit the agent message of the variable size.
+- one of the chat message types should support transmitting large binaries in chunks that could potentially be interleaved with other messages. For example, image preview would fit the message, but the full size image will be transmitted in chunks later - same for large files.
+- using object storage can be effective for large groups, but we will postpone it until content channels are implemented.
+
+## Questions
+
+- should content types be:
+  - limited to MIME-types
+  - separate content types vocabulary
+  - both MIME types and extensions (currently we support MIME (m.) and Simplex (x.) namespaces)
+  - allow additional content types namespaces
+
+## Message syntax
+
+The syntax of the message inside agent MSG:
+
+```abnf
+agentMessageBody = [chatMsgId] SP msgEvent SP [parameters] SP [contentParts [SP msgBodyParts]]
+chatMsgId = 1*DIGIT ; used to refer to previous message;
+                    ; in the group should only be used in messages sent to all members,
+                    ; which is the main reason not to use external agent ID -
+                    ; some messages are sent only to one member
+msgEvent = protocolNamespace 1*("." msgTypeName)
+protocolNamespace = 1*ALPHA ; "x" for all events defined in the protocol
+msgTypeName = 1*ALPHA
+parameters = parameter *("," parameter)
+parameter = 1*(%x21-2B / %x2D-7E) ; exclude control characters, space, comma (%x2C)
+contentParts = contentPart *("," contentPart)
+contentPart = contentTypeNamespace "." contentType ":" contentSize [":" contentHash]
+contentType = "i." <mime-type> / contentTypeNamespace "." 1*("." contentTypeName)
+contentTypeNamespace = 1*ALPHA
+contentTypeName = 1*ALPHA
+contentHash = <base64>
+msgBodyParts = msgBodyPart *(SP msgBodyPart)
+msgEventParents = msgEventParent *msgEventParent ; binary body part for content type "x.dag"
+msgEventParent = memberId refMsgId refMsgHash
+memberId = 8*8(OCTET) ; shared member ID
+refMsgId = 8*8(OCTET) ; sequential message number - external agent message ID
+refMsgHash = 16*16(OCTET) ; SHA256 of agent message body
+```
+
+### Example: messages, updates, groups
+
+```
+"3 x.msg.new c.text x.text:5 hello "
+"4 x.msg.new c.image i.image/jpg:256,i.image/png:4096 abcd abcd "
+"4 x.msg.new c.image x.dag:32,i.image/jpg:8000,i.image/png:16000 binary1"
+"5 x.msg.new c.image,i.image/jpg:150000 i.image/jpg:256 abcd "
+"6 x.msg.file 5,1.1 x.file:60000 abcd "
+"7 x.msg.file 5,1.2 x.file:60000 abcd "
+"8 x.msg.file 5,1.3 x.file:30000 abcd "
+'8 x.msg.update 3 x.text:11,x.dag:16 hello there abcd '
+'9 x.msg.delete 3'
+'10 x.msg.new app/v1 i.text/html:NNN,i.text/css:NNN,c.js:NNN,c.json:NNN ... ... ... {...} '
+'11 x.msg.eval 8 c.json:NNN {...} '
+'12 x.msg.new c.text x.text:16,x.dag:32 hello there @123 abcd '
+' x.grp.mem.inv 23456,123 x.json:NNN {...} '
+' x.grp.mem.acpt 23456 x.text:NNN <invitation> '
+' x.grp.mem.intro 23456,234 x.json:NNN {...} '
+' x.grp.mem.inv 23456,234 x.text:NNN <invitation> '
+' x.grp.mem.req 23456,123 x.json:NNN {...} '
+' x.grp.mem.direct.inv 23456,234 x.text:NNN <invitation>  '
+' x.file name,size x.text:NNN <invitation> '
+```
+
+### Group protocol
+
+#### Add group member
+
+A -> B: invite to group - `MSG: x.grp.inv G_MEM_ID_A,G_MEM_ROLE_A,G_MEM_ID_B,G_MEM_ROLE_B,<invitation> x.json:NNN <group_profile>`
+user B confirms
+B -> A: establish group connection (B: JOIN, A: LET)
+B -> Ag: join group - `in SMP confirmation: x.grp.acpt G_MEM_ID_B`
+A -> group (including B)): announce group member: `MSG: N x.grp.mem.new G_MEM_ID_B,G_MEM_ROLE_B,G_MEM_ID_M,... x.json:NNN <B_profile>`
+
+In the message `x.grp.mem.new` A sends the sorted list of all members to whom A is connected followed by the new member ID, role and profile. The following introductions will be sent about/to all members A "knows about" (includes members introduced to A and members who accepted group invitation but not connected yet), once they are connected, so it can be a bigger list than sent in `x.grp.mem.new`.
+
+All members who received `x.grp.mem.new` from A should check the list of connected members and if any connected members that recipients invited to the group are not in this list, they should introduce them to this new member (the last ID, role and profile in `x.grp.mem.new`). That might lead to double introductions that would provide a stronger consistency of group membership at a cost of extra connection between some members that will be unused.
+
+subsequent messages between A and B are via group connection
+A -> Bg: intro member - `MSG: x.grp.mem.intro G_MEM_ID_M,G_MEM_ROLE_M x.json:NNN <M_profile>`
+B -> Ag: inv for mem - `MSG: x.grp.mem.inv G_MEM_ID_M,<gr_invitation>,<dm_invitation>,<probe>`
+M is an existing member, messages are via group connection
+A -> Mg: fwd inv - `MSG: x.grp.mem.fwd G_MEM_ID_B,<gr_invitation>,<dm_invitation>,<probe>`
+M -> Bg: establish group connection (M: JOIN, B: LET)
+M -> B: establish direct connection (M: JOIN, B: LET)
+M -> Bg: confirm profile and role - `CONF: x.grp.mem.info G_MEM_ID_M,G_MEM_ROLE x.json:NNN <M_profile>`
+B -> Mg: send profile probe - `MSG: x.info.probe <probe>` - it should always be send, even when there is no profile match.
+if M is a known contact (profile match) send probe to M:
+  B -> M (via old DM conn): profile match probe: `MSG: x.info.probe.check <probe_hash>`
+  M -> B (via old DM conn): probe confirm: `MSG: x.info.probe.ok <probe>`
+  link to the same contact
+B -> Ag: connected to M: `MSG: x.grp.mem.con G_MEM_ID_M`
+M -> Ag: connected to M: `MSG: x.grp.mem.con G_MEM_ID_B`
+
+once all members connected
+A -> group: `MSG: N x.grp.mem.con.all G_MEM_ID_B`
+
+#### Send group message
+
+Example:
+
+`MSG: N x.msg.new c.text x.text:5 hello `
+
+#### Group member statuses
+
+1. Me
+  - invited
+  - accepted
+  - connected to member who invited me
+  - announced to group
+    - x.grp.mem.new to group
+  - confirmed as connected to group
+    - this happens once member who invited me sends x.grp.mem.ok to group
+1. Member that I invited:
+  - invited
+  - accepted
+  - connected to me
+  - announced to group
+    - this happens after x.grp.mem.new but before introductions are sent.
+    This message is used to determine which members should be additionally introduced if they were announced before (or in "parallel").
+  - confirmed as connected to group
+2. Member who invited me
+  - invited_me
+  - connected to me
+    - I won't know whether this member was announced or confirmed to group - with the correctly functioning clients it must have happened.
+3. Prior member introduced to me after I joined (x.grp.mem.intro)
+  - introduced
+  - sent invitation
+  - connected
+  - connected directly (or confirmed existing contact)
+4. Member I was introduced to after that member joined (via x.grp.mem.fwd)
+  - announced via x.grp.mem.new
+  - received invitation
+  - connected
+  - connected directly (or confirmed existing contact)
+
+#### Introductions
+
+1. Introductions I sent to members I invited
+  - the time of joining is determined by the time of creating the connection and sending the x.grp.mem.new message to the group.
+  - introductions of the members who were connected before the new member should be sent - how to determine which members were connected before?
+    - use time stamp of creating connection, possibly in the member record - not very reliable, as time can change.
+    - use record ID - requires changing the schema, as currently members are added as invited, not as connected. So possibly invited members should be tracked in a separate table, and all members should still be tracked together to ensure that memberId is unique.
+    - record ID is also not 100% sufficient, as there can be forks in message history and I may need to intro the member I invited to the member that was announced after my member in my chronology, but in another graph branch.
+    - some other mechanism that allows to establish who should be connected to whom and whether I should introduce or another member (in case of forks - although maybe we both can introduce and eventually two group connections will be created between these members and they would just ignore the first one - although in cases of multiple branches in the graph it can be N connections).
+    - introductions/member connection statuses:
+      - created introduction
+      - sent to the member I invited
+      - received the invitation from the member I invited
+      - forwarded this invitation to previously connected member
+      - received confirmation from member I invited
+      - received confirmation from member I forwarded to
+      - completed introduction and recorded that these members are now fully connected to each other
+2. Introductions I received from the member who invited me
+  - if somebody else sends such introduction - this is an error (can be logged or ignored)
+  - duplicate memberId is an error (e.g. it is a member that was announced in the group broadcast - I should be introduced to this member, and not the other way around? Although it can happen in case of fork and maybe I should establish the connection anyway).
+  - member connection status in this case is just a member status from part 3, so maybe no need to track invitations separately and just put SMPQueueInfo on member record.
+3. Invitation forwarded to me by any prior member
+  - any admin/owner can add members, so they can forward their queue invitations - I should just check forwarding member permission
+  - duplicate memberId is an error
+  - unannounced memberId is an error - I should have seen member announcement prior to receiving this forwarded invitation. Fork would not happen here as it is the same member that announces and forwards the invitation, so they should be in order.
+  - member connection status in this case is just a member status from part 4, so maybe no need to track invitations separately and just put SMPQueueInfo on member record.

stack.yaml

@@ -17,7 +17,7 @@
 #
 # resolver: ./custom-snapshot.yaml
 # resolver: https://example.com/snapshots/2018-01-01.yaml
-resolver: lts-16.17
+resolver: lts-17.12
 
 # User packages to be built.
 # Various formats can be used as shown in the example below.
@@ -40,11 +40,10 @@ extra-deps:
   - simple-logger-0.1.0@sha256:be8ede4bd251a9cac776533bae7fb643369ebd826eb948a9a18df1a8dd252ff8,1079
   - sqlite-simple-0.4.18.0@sha256:3ceea56375c0a3590c814e411a4eb86943f8d31b93b110ca159c90689b6b39e5,3002
   - terminal-0.2.0.0@sha256:de6770ecaae3197c66ac1f0db5a80cf5a5b1d3b64a66a05b50f442de5ad39570,2977
-  - github: simplex-chat/simplexmq
-    commit: 2b0950e78e390b41cf4064818534d7791aa293ae
-# - network-run-0.2.4@sha256:7dbb06def522dab413bce4a46af476820bffdff2071974736b06f52f4ab57c96,885
-# - git: https://github.com/commercialhaskell/stack.git
-#   commit: e7b331f14bcffb8367cd58fbfc8b40ec7642100a
+  - simplexmq-0.4.1@sha256:3a1bc40d85e4e398458e5b9b79757e0af4fe27b8ef44eb3157f7f1e07412a8e8,7640
+  # - ../simplexmq
+  # - github: simplex-chat/simplexmq
+  #   commit: 35e6593581e68f7b444e0f8f4fb6a2e2cc59a5ea
 #
 # extra-deps: []
 

tests/ChatClient.hs

@@ -0,0 +1,199 @@
+{-# LANGUAGE DuplicateRecordFields #-}
+{-# LANGUAGE LambdaCase #-}
+{-# LANGUAGE NamedFieldPuns #-}
+{-# LANGUAGE OverloadedLists #-}
+{-# LANGUAGE OverloadedStrings #-}
+{-# LANGUAGE TypeApplications #-}
+
+module ChatClient where
+
+import Control.Concurrent (ThreadId, forkIOWithUnmask, killThread)
+import Control.Concurrent.Async
+import Control.Concurrent.STM
+import Control.Exception (bracket, bracket_)
+import Control.Monad.Except
+import Data.List (dropWhileEnd)
+import Network.Socket
+import Simplex.Chat
+import Simplex.Chat.Controller (ChatConfig (..), ChatController (..))
+import Simplex.Chat.Options
+import Simplex.Chat.Store
+import Simplex.Chat.Types (Profile)
+import Simplex.Messaging.Agent.Env.SQLite
+import Simplex.Messaging.Agent.RetryInterval
+import Simplex.Messaging.Server (runSMPServerBlocking)
+import Simplex.Messaging.Server.Env.STM
+import Simplex.Messaging.Transport
+import System.Directory (createDirectoryIfMissing, removeDirectoryRecursive)
+import qualified System.Terminal as C
+import System.Terminal.Internal (VirtualTerminal (..), VirtualTerminalSettings (..), withVirtualTerminal)
+import System.Timeout (timeout)
+
+testDBPrefix :: FilePath
+testDBPrefix = "tests/tmp/test"
+
+serverPort :: ServiceName
+serverPort = "5000"
+
+opts :: ChatOpts
+opts =
+  ChatOpts
+    { dbFile = undefined,
+      smpServers = ["localhost:5000"]
+    }
+
+termSettings :: VirtualTerminalSettings
+termSettings =
+  VirtualTerminalSettings
+    { virtualType = "xterm",
+      virtualWindowSize = pure C.Size {height = 24, width = 1000},
+      virtualEvent = retry,
+      virtualInterrupt = retry
+    }
+
+data TestCC = TestCC
+  { chatController :: ChatController,
+    virtualTerminal :: VirtualTerminal,
+    chatAsync :: Async (),
+    termAsync :: Async (),
+    termQ :: TQueue String
+  }
+
+aCfg :: AgentConfig
+aCfg = agentConfig defaultChatConfig
+
+cfg :: ChatConfig
+cfg =
+  defaultChatConfig
+    { agentConfig =
+        aCfg {retryInterval = (retryInterval aCfg) {initialInterval = 50000}}
+    }
+
+virtualSimplexChat :: FilePath -> Profile -> IO TestCC
+virtualSimplexChat dbFile profile = do
+  st <- createStore (dbFile <> ".chat.db") 1
+  void . runExceptT $ createUser st profile True
+  t <- withVirtualTerminal termSettings pure
+  cc <- newChatController cfg opts {dbFile} t . const $ pure () -- no notifications
+  chatAsync <- async $ runSimplexChat cc
+  termQ <- newTQueueIO
+  termAsync <- async $ readTerminalOutput t termQ
+  pure TestCC {chatController = cc, virtualTerminal = t, chatAsync, termAsync, termQ}
+
+readTerminalOutput :: VirtualTerminal -> TQueue String -> IO ()
+readTerminalOutput t termQ = do
+  let w = virtualWindow t
+  winVar <- atomically $ newTVar . init =<< readTVar w
+  forever . atomically $ do
+    win <- readTVar winVar
+    win' <- init <$> readTVar w
+    if win' == win
+      then retry
+      else do
+        let diff = getDiff win' win
+        forM_ diff $ writeTQueue termQ
+        writeTVar winVar win'
+  where
+    getDiff :: [String] -> [String] -> [String]
+    getDiff win win' = getDiff_ 1 (length win) win win'
+    getDiff_ :: Int -> Int -> [String] -> [String] -> [String]
+    getDiff_ n len win' win =
+      let diff = drop (len - n) win'
+       in if drop n win <> diff == win'
+            then map (dropWhileEnd (== ' ')) diff
+            else getDiff_ (n + 1) len win' win
+
+testChatN :: [Profile] -> ([TestCC] -> IO ()) -> IO ()
+testChatN ps test =
+  bracket_
+    (createDirectoryIfMissing False "tests/tmp")
+    (removeDirectoryRecursive "tests/tmp")
+    $ do
+      let envs = zip ps $ map ((testDBPrefix <>) . show) [(1 :: Int) ..]
+      tcs <- getTestCCs envs []
+      test tcs
+  where
+    getTestCCs [] tcs = pure tcs
+    getTestCCs ((p, db) : envs') tcs = (:) <$> virtualSimplexChat db p <*> getTestCCs envs' tcs
+
+testChat2 :: Profile -> Profile -> (TestCC -> TestCC -> IO ()) -> IO ()
+testChat2 p1 p2 test = testChatN [p1, p2] test_
+  where
+    test_ :: [TestCC] -> IO ()
+    test_ [tc1, tc2] = test tc1 tc2
+    test_ _ = error "expected 2 chat clients"
+
+testChat3 :: Profile -> Profile -> Profile -> (TestCC -> TestCC -> TestCC -> IO ()) -> IO ()
+testChat3 p1 p2 p3 test = testChatN [p1, p2, p3] test_
+  where
+    test_ :: [TestCC] -> IO ()
+    test_ [tc1, tc2, tc3] = test tc1 tc2 tc3
+    test_ _ = error "expected 3 chat clients"
+
+testChat4 :: Profile -> Profile -> Profile -> Profile -> (TestCC -> TestCC -> TestCC -> TestCC -> IO ()) -> IO ()
+testChat4 p1 p2 p3 p4 test = testChatN [p1, p2, p3, p4] test_
+  where
+    test_ :: [TestCC] -> IO ()
+    test_ [tc1, tc2, tc3, tc4] = test tc1 tc2 tc3 tc4
+    test_ _ = error "expected 4 chat clients"
+
+concurrentlyN_ :: [IO a] -> IO ()
+concurrentlyN_ = mapConcurrently_ id
+
+serverCfg :: ServerConfig
+serverCfg =
+  ServerConfig
+    { transports = [(serverPort, transport @TCP)],
+      tbqSize = 1,
+      msgQueueQuota = 4,
+      queueIdBytes = 12,
+      msgIdBytes = 6,
+      storeLog = Nothing,
+      blockSize = 4096,
+      serverPrivateKey =
+        -- full RSA private key (only for tests)
+        "MIIFIwIBAAKCAQEArZyrri/NAwt5buvYjwu+B/MQeJUszDBpRgVqNddlI9kNwDXu\
+        \kaJ8chEhrtaUgXeSWGooWwqjXEUQE6RVbCC6QVo9VEBSP4xFwVVd9Fj7OsgfcXXh\
+        \AqWxfctDcBZQ5jTUiJpdBc+Vz2ZkumVNl0W+j9kWm9nfkMLQj8c0cVSDxz4OKpZb\
+        \qFuj0uzHkis7e7wsrKSKWLPg3M5ZXPZM1m9qn7SfJzDRDfJifamxWI7uz9XK2+Dp\
+        \NkUQlGQgFJEv1cKN88JAwIqZ1s+TAQMQiB+4QZ2aNfSqGEzRJN7FMCKRK7pM0A9A\
+        \PCnijyuImvKFxTdk8Bx1q+XNJzsY6fBrLWJZ+QKBgQCySG4tzlcEm+tOVWRcwrWh\
+        \6zsczGZp9mbf9c8itRx6dlldSYuDG1qnddL70wuAZF2AgS1JZgvcRZECoZRoWP5q\
+        \Kq2wvpTIYjFPpC39lxgUoA/DXKVKZZdan+gwaVPAPT54my1CS32VrOiAY4gVJ3LJ\
+        \Mn1/FqZXUFQA326pau3loQKCAQEAoljmJMp88EZoy3HlHUbOjl5UEhzzVsU1TnQi\
+        \QmPm+aWRe2qelhjW4aTvSVE5mAUJsN6UWTeMf4uvM69Z9I5pfw2pEm8x4+GxRibY\
+        \iiwF2QNaLxxmzEHm1zQQPTgb39o8mgklhzFPill0JsnL3f6IkVwjFJofWSmpqEGs\
+        \dFSMRSXUTVXh1p/o7QZrhpwO/475iWKVS7o48N/0Xp513re3aXw+DRNuVnFEaBIe\
+        \TLvWM9Czn16ndAu1HYiTBuMvtRbAWnGZxU8ewzF4wlWK5tdIL5PTJDd1VhZJAKtB\
+        \npDvJpwxzKmjAhcTmjx0ckMIWtdVaOVm/2gWCXDty2FEdg7koQKBgQDOUUguJ/i7\
+        \q0jldWYRnVkotKnpInPdcEaodrehfOqYEHnvro9xlS6OeAS4Vz5AdH45zQ/4J3bV\
+        \2cH66tNr18ebM9nL//t5G69i89R9W7szyUxCI3LmAIdi3oSEbmz5GQBaw4l6h9Wi\
+        \n4FmFQaAXZrjQfO2qJcAHvWRsMp2pmqAGwKBgQDXaza0DRsKWywWznsHcmHa0cx8\
+        \I4jxqGaQmLO7wBJRP1NSFrywy1QfYrVX9CTLBK4V3F0PCgZ01Qv94751CzN43TgF\
+        \ebd/O9r5NjNTnOXzdWqETbCffLGd6kLgCMwPQWpM9ySVjXHWCGZsRAnF2F6M1O32\
+        \43StIifvwJQFqSM3ewKBgCaW6y7sRY90Ua7283RErezd9EyT22BWlDlACrPu3FNC\
+        \LtBf1j43uxBWBQrMLsHe2GtTV0xt9m0MfwZsm2gSsXcm4Xi4DJgfN+Z7rIlyy9UY\
+        \PCDSdZiU1qSr+NrffDrXlfiAM1cUmCdUX7eKjp/ltkUHNaOGfSn5Pdr3MkAiD/Hf\
+        \AoGBAKIdKCuOwuYlwjS9J+IRGuSSM4o+OxQdwGmcJDTCpyWb5dEk68e7xKIna3zf\
+        \jc+H+QdMXv1nkRK9bZgYheXczsXaNZUSTwpxaEldzVD3hNvsXSgJRy9fqHwA4PBq\
+        \vqiBHoO3RNbqg+2rmTMfDuXreME3S955ZiPZm4Z+T8Hj52mPAoGAQm5QH/gLFtY5\
+        \+znqU/0G8V6BKISCQMxbbmTQVcTgGySrP2gVd+e4MWvUttaZykhWqs8rpr7mgpIY\
+        \hul7Swx0SHFN3WpXu8uj+B6MLpRcCbDHO65qU4kQLs+IaXXsuuTjMvJ5LwjkZVrQ\
+        \TmKzSAw7iVWwEUZR/PeiEKazqrpp9VU="
+    }
+
+withSmpServer :: IO a -> IO a
+withSmpServer = serverBracket (`runSMPServerBlocking` serverCfg) (pure ()) . const
+
+serverBracket :: (TMVar Bool -> IO ()) -> IO () -> (ThreadId -> IO a) -> IO a
+serverBracket process afterProcess f = do
+  started <- newEmptyTMVarIO
+  bracket
+    (forkIOWithUnmask ($ process started))
+    (\t -> killThread t >> afterProcess >> waitFor started "stop")
+    (\t -> waitFor started "start" >> f t)
+  where
+    waitFor started s =
+      5000000 `timeout` atomically (takeTMVar started) >>= \case
+        Nothing -> error $ "server did not " <> s
+        _ -> pure ()

tests/ChatTests.hs

@@ -0,0 +1,667 @@
+{-# LANGUAGE NamedFieldPuns #-}
+{-# LANGUAGE OverloadedStrings #-}
+{-# LANGUAGE PostfixOperators #-}
+
+module ChatTests where
+
+import ChatClient
+import Control.Concurrent.Async (concurrently_)
+import Control.Concurrent.STM
+import qualified Data.ByteString as B
+import Data.Char (isDigit)
+import qualified Data.Text as T
+import Simplex.Chat.Controller
+import Simplex.Chat.Types (Profile (..), User (..))
+import Simplex.Chat.Util (unlessM)
+import System.Directory (doesFileExist)
+import System.Timeout (timeout)
+import Test.Hspec
+
+aliceProfile :: Profile
+aliceProfile = Profile {displayName = "alice", fullName = "Alice"}
+
+bobProfile :: Profile
+bobProfile = Profile {displayName = "bob", fullName = "Bob"}
+
+cathProfile :: Profile
+cathProfile = Profile {displayName = "cath", fullName = "Catherine"}
+
+danProfile :: Profile
+danProfile = Profile {displayName = "dan", fullName = "Daniel"}
+
+chatTests :: Spec
+chatTests = do
+  describe "direct messages" $
+    it "add contact and send/receive message" testAddContact
+  describe "chat groups" $ do
+    it "add contacts, create group and send/receive messages" testGroup
+    it "create and join group with 4 members" testGroup2
+    it "create and delete group" testGroupDelete
+    it "remove contact from group and add again" testGroupRemoveAdd
+  describe "user profiles" $
+    it "update user profiles and notify contacts" testUpdateProfile
+  describe "sending and receiving files" $ do
+    it "send and receive file" testFileTransfer
+    it "send and receive a small file" testSmallFileTransfer
+    it "sender cancelled file transfer" testFileSndCancel
+    it "recipient cancelled file transfer" testFileRcvCancel
+    it "send and receive file to group" testGroupFileTransfer
+
+testAddContact :: IO ()
+testAddContact =
+  testChat2 aliceProfile bobProfile $
+    \alice bob -> do
+      alice ##> "/c"
+      inv <- getInvitation alice
+      bob ##> ("/c " <> inv)
+      concurrently_
+        (bob <## "alice (Alice): contact is connected")
+        (alice <## "bob (Bob): contact is connected")
+      alice #> "@bob hello"
+      bob <# "alice> hello"
+      bob #> "@alice hi"
+      alice <# "bob> hi"
+      -- test adding the same contact one more time - local name will be different
+      alice ##> "/c"
+      inv' <- getInvitation alice
+      bob ##> ("/c " <> inv')
+      concurrently_
+        (bob <## "alice_1 (Alice): contact is connected")
+        (alice <## "bob_1 (Bob): contact is connected")
+      alice #> "@bob_1 hello"
+      bob <# "alice_1> hello"
+      bob #> "@alice_1 hi"
+      alice <# "bob_1> hi"
+      -- test deleting contact
+      alice ##> "/d bob_1"
+      alice <## "bob_1: contact is deleted"
+      alice #> "@bob_1 hey"
+      alice <## "no contact bob_1"
+
+testGroup :: IO ()
+testGroup =
+  testChat3 aliceProfile bobProfile cathProfile $
+    \alice bob cath -> do
+      connectUsers alice bob
+      connectUsers alice cath
+      alice ##> "/g team"
+      alice <## "group #team is created"
+      alice <## "use /a team <name> to add members"
+      alice ##> "/a team bob"
+      concurrentlyN_
+        [ alice <## "invitation to join the group #team sent to bob",
+          do
+            bob <## "#team: alice invites you to join the group as admin"
+            bob <## "use /j team to accept"
+        ]
+      bob ##> "/j team"
+      concurrently_
+        (alice <## "#team: bob joined the group")
+        (bob <## "#team: you joined the group")
+      alice ##> "/a team cath"
+      concurrentlyN_
+        [ alice <## "invitation to join the group #team sent to cath",
+          do
+            cath <## "#team: alice invites you to join the group as admin"
+            cath <## "use /j team to accept"
+        ]
+      cath ##> "/j team"
+      concurrentlyN_
+        [ alice <## "#team: cath joined the group",
+          do
+            cath <## "#team: you joined the group"
+            cath <## "#team: member bob (Bob) is connected",
+          do
+            bob <## "#team: alice added cath (Catherine) to the group (connecting...)"
+            bob <## "#team: new member cath is connected"
+        ]
+      alice #> "#team hello"
+      concurrently_
+        (bob <# "#team alice> hello")
+        (cath <# "#team alice> hello")
+      bob #> "#team hi there"
+      concurrently_
+        (alice <# "#team bob> hi there")
+        (cath <# "#team bob> hi there")
+      cath #> "#team hey"
+      concurrently_
+        (alice <# "#team cath> hey")
+        (bob <# "#team cath> hey")
+      bob <##> cath
+      -- remove member
+      bob ##> "/rm team cath"
+      concurrentlyN_
+        [ bob <## "#team: you removed cath from the group",
+          alice <## "#team: bob removed cath from the group",
+          do
+            cath <## "#team: bob removed you from the group"
+            cath <## "use /d #team to delete the group"
+        ]
+      bob #> "#team hi"
+      concurrently_
+        (alice <# "#team bob> hi")
+        (cath </)
+      alice #> "#team hello"
+      concurrently_
+        (bob <# "#team alice> hello")
+        (cath </)
+      cath #> "#team hello"
+      cath <## "you are no longer the member of the group"
+      bob <##> cath
+
+testGroup2 :: IO ()
+testGroup2 =
+  testChat4 aliceProfile bobProfile cathProfile danProfile $
+    \alice bob cath dan -> do
+      connectUsers alice bob
+      connectUsers alice cath
+      connectUsers bob dan
+      connectUsers alice dan
+      alice ##> "/g club"
+      alice <## "group #club is created"
+      alice <## "use /a club <name> to add members"
+      alice ##> "/a club bob"
+      concurrentlyN_
+        [ alice <## "invitation to join the group #club sent to bob",
+          do
+            bob <## "#club: alice invites you to join the group as admin"
+            bob <## "use /j club to accept"
+        ]
+      alice ##> "/a club cath"
+      concurrentlyN_
+        [ alice <## "invitation to join the group #club sent to cath",
+          do
+            cath <## "#club: alice invites you to join the group as admin"
+            cath <## "use /j club to accept"
+        ]
+      bob ##> "/j club"
+      concurrently_
+        (alice <## "#club: bob joined the group")
+        (bob <## "#club: you joined the group")
+      cath ##> "/j club"
+      concurrentlyN_
+        [ alice <## "#club: cath joined the group",
+          do
+            cath <## "#club: you joined the group"
+            cath <## "#club: member bob (Bob) is connected",
+          do
+            bob <## "#club: alice added cath (Catherine) to the group (connecting...)"
+            bob <## "#club: new member cath is connected"
+        ]
+      bob ##> "/a club dan"
+      concurrentlyN_
+        [ bob <## "invitation to join the group #club sent to dan",
+          do
+            dan <## "#club: bob invites you to join the group as admin"
+            dan <## "use /j club to accept"
+        ]
+      dan ##> "/j club"
+      concurrentlyN_
+        [ bob <## "#club: dan joined the group",
+          do
+            dan <## "#club: you joined the group"
+            dan
+              <### [ "#club: member alice_1 (Alice) is connected",
+                     "contact alice_1 is merged into alice",
+                     "use @alice <message> to send messages",
+                     "#club: member cath (Catherine) is connected"
+                   ],
+          do
+            alice <## "#club: bob added dan_1 (Daniel) to the group (connecting...)"
+            alice <## "#club: new member dan_1 is connected"
+            alice <## "contact dan_1 is merged into dan"
+            alice <## "use @dan <message> to send messages",
+          do
+            cath <## "#club: bob added dan (Daniel) to the group (connecting...)"
+            cath <## "#club: new member dan is connected"
+        ]
+      alice #> "#club hello"
+      concurrentlyN_
+        [ bob <# "#club alice> hello",
+          cath <# "#club alice> hello",
+          dan <# "#club alice> hello"
+        ]
+      bob #> "#club hi there"
+      concurrentlyN_
+        [ alice <# "#club bob> hi there",
+          cath <# "#club bob> hi there",
+          dan <# "#club bob> hi there"
+        ]
+      cath #> "#club hey"
+      concurrentlyN_
+        [ alice <# "#club cath> hey",
+          bob <# "#club cath> hey",
+          dan <# "#club cath> hey"
+        ]
+      dan #> "#club how is it going?"
+      concurrentlyN_
+        [ alice <# "#club dan> how is it going?",
+          bob <# "#club dan> how is it going?",
+          cath <# "#club dan> how is it going?"
+        ]
+      bob <##> cath
+      dan <##> cath
+      dan <##> alice
+      -- remove member
+      cath ##> "/rm club dan"
+      concurrentlyN_
+        [ cath <## "#club: you removed dan from the group",
+          alice <## "#club: cath removed dan from the group",
+          bob <## "#club: cath removed dan from the group",
+          do
+            dan <## "#club: cath removed you from the group"
+            dan <## "use /d #club to delete the group"
+        ]
+      alice #> "#club hello"
+      concurrentlyN_
+        [ bob <# "#club alice> hello",
+          cath <# "#club alice> hello",
+          (dan </)
+        ]
+      bob #> "#club hi there"
+      concurrentlyN_
+        [ alice <# "#club bob> hi there",
+          cath <# "#club bob> hi there",
+          (dan </)
+        ]
+      cath #> "#club hey"
+      concurrentlyN_
+        [ alice <# "#club cath> hey",
+          bob <# "#club cath> hey",
+          (dan </)
+        ]
+      dan #> "#club how is it going?"
+      dan <## "you are no longer the member of the group"
+      dan <##> cath
+      dan <##> alice
+      -- member leaves
+      bob ##> "/l club"
+      concurrentlyN_
+        [ do
+            bob <## "#club: you left the group"
+            bob <## "use /d #club to delete the group",
+          alice <## "#club: bob left the group",
+          cath <## "#club: bob left the group"
+        ]
+      alice #> "#club hello"
+      concurrently_
+        (cath <# "#club alice> hello")
+        (bob </)
+      cath #> "#club hey"
+      concurrently_
+        (alice <# "#club cath> hey")
+        (bob </)
+      bob #> "#club how is it going?"
+      bob <## "you are no longer the member of the group"
+      bob <##> cath
+      bob <##> alice
+
+testGroupDelete :: IO ()
+testGroupDelete =
+  testChat3 aliceProfile bobProfile cathProfile $
+    \alice bob cath -> do
+      createGroup3 "team" alice bob cath
+      alice ##> "/d #team"
+      concurrentlyN_
+        [ alice <## "#team: you deleted the group",
+          do
+            bob <## "#team: alice deleted the group"
+            bob <## "use /d #team to delete the local copy of the group",
+          do
+            cath <## "#team: alice deleted the group"
+            cath <## "use /d #team to delete the local copy of the group"
+        ]
+      bob ##> "/d #team"
+      bob <## "#team: you deleted the group"
+      cath #> "#team hi"
+      cath <## "you are no longer the member of the group"
+
+testGroupRemoveAdd :: IO ()
+testGroupRemoveAdd =
+  testChat3 aliceProfile bobProfile cathProfile $
+    \alice bob cath -> do
+      createGroup3 "team" alice bob cath
+      -- remove member
+      alice ##> "/rm team bob"
+      concurrentlyN_
+        [ alice <## "#team: you removed bob from the group",
+          do
+            bob <## "#team: alice removed you from the group"
+            bob <## "use /d #team to delete the group",
+          cath <## "#team: alice removed bob from the group"
+        ]
+      alice ##> "/a team bob"
+      alice <## "invitation to join the group #team sent to bob"
+      bob <## "#team_1 (team): alice invites you to join the group as admin"
+      bob <## "use /j team_1 to accept"
+      bob ##> "/j team_1"
+      concurrentlyN_
+        [ alice <## "#team: bob joined the group",
+          do
+            bob <## "#team_1: you joined the group"
+            bob <## "#team_1: member cath_1 (Catherine) is connected"
+            bob <## "contact cath_1 is merged into cath"
+            bob <## "use @cath <message> to send messages",
+          do
+            cath <## "#team: alice added bob_1 (Bob) to the group (connecting...)"
+            cath <## "#team: new member bob_1 is connected"
+            cath <## "contact bob_1 is merged into bob"
+            cath <## "use @bob <message> to send messages"
+        ]
+      alice #> "#team hi"
+      concurrently_
+        (bob <# "#team_1 alice> hi")
+        (cath <# "#team alice> hi")
+      bob #> "#team_1 hey"
+      concurrently_
+        (alice <# "#team bob> hey")
+        (cath <# "#team bob> hey")
+      cath #> "#team hello"
+      concurrently_
+        (alice <# "#team cath> hello")
+        (bob <# "#team_1 cath> hello")
+
+testUpdateProfile :: IO ()
+testUpdateProfile =
+  testChat3 aliceProfile bobProfile cathProfile $
+    \alice bob cath -> do
+      createGroup3 "team" alice bob cath
+      alice ##> "/p"
+      alice <## "user profile: alice (Alice)"
+      alice <## "use /p <display name> [<full name>] to change it"
+      alice <## "(the updated profile will be sent to all your contacts)"
+      alice ##> "/p alice"
+      concurrentlyN_
+        [ alice <## "user full name removed (your contacts are notified)",
+          bob <## "contact alice removed full name",
+          cath <## "contact alice removed full name"
+        ]
+      alice ##> "/p alice Alice Jones"
+      concurrentlyN_
+        [ alice <## "user full name changed to Alice Jones (your contacts are notified)",
+          bob <## "contact alice updated full name: Alice Jones",
+          cath <## "contact alice updated full name: Alice Jones"
+        ]
+      cath ##> "/p cate"
+      concurrentlyN_
+        [ cath <## "user profile is changed to cate (your contacts are notified)",
+          do
+            alice <## "contact cath changed to cate"
+            alice <## "use @cate <message> to send messages",
+          do
+            bob <## "contact cath changed to cate"
+            bob <## "use @cate <message> to send messages"
+        ]
+      cath ##> "/p cat Cate"
+      concurrentlyN_
+        [ cath <## "user profile is changed to cat (Cate) (your contacts are notified)",
+          do
+            alice <## "contact cate changed to cat (Cate)"
+            alice <## "use @cat <message> to send messages",
+          do
+            bob <## "contact cate changed to cat (Cate)"
+            bob <## "use @cat <message> to send messages"
+        ]
+
+testFileTransfer :: IO ()
+testFileTransfer =
+  testChat2 aliceProfile bobProfile $
+    \alice bob -> do
+      connectUsers alice bob
+      startFileTransfer alice bob
+      concurrentlyN_
+        [ do
+            bob #> "@alice receiving here..."
+            bob <## "completed receiving file 1 (test.jpg) from alice",
+          do
+            alice <# "bob> receiving here..."
+            alice <## "completed sending file 1 (test.jpg) to bob"
+        ]
+      src <- B.readFile "./tests/fixtures/test.jpg"
+      dest <- B.readFile "./tests/tmp/test.jpg"
+      dest `shouldBe` src
+
+testSmallFileTransfer :: IO ()
+testSmallFileTransfer =
+  testChat2 aliceProfile bobProfile $
+    \alice bob -> do
+      connectUsers alice bob
+      alice #> "/f @bob ./tests/fixtures/test.txt"
+      alice <## "use /fc 1 to cancel sending"
+      bob <# "alice> sends file test.txt (11 bytes / 11 bytes)"
+      bob <## "use /fr 1 [<dir>/ | <path>] to receive it"
+      bob ##> "/fr 1 ./tests/tmp"
+      bob <## "saving file 1 from alice to ./tests/tmp/test.txt"
+      concurrentlyN_
+        [ do
+            bob <## "started receiving file 1 (test.txt) from alice"
+            bob <## "completed receiving file 1 (test.txt) from alice",
+          do
+            alice <## "started sending file 1 (test.txt) to bob"
+            alice <## "completed sending file 1 (test.txt) to bob"
+        ]
+      src <- B.readFile "./tests/fixtures/test.txt"
+      dest <- B.readFile "./tests/tmp/test.txt"
+      dest `shouldBe` src
+
+testFileSndCancel :: IO ()
+testFileSndCancel =
+  testChat2 aliceProfile bobProfile $
+    \alice bob -> do
+      connectUsers alice bob
+      startFileTransfer alice bob
+      alice ##> "/fc 1"
+      concurrentlyN_
+        [ do
+            alice <## "cancelled sending file 1 (test.jpg) to bob"
+            alice ##> "/fs 1"
+            alice <## "sending file 1 (test.jpg) cancelled",
+          do
+            bob <## "alice cancelled sending file 1 (test.jpg)"
+            bob ##> "/fs 1"
+            bob <## "receiving file 1 (test.jpg) cancelled, received part path: ./tests/tmp/test.jpg"
+        ]
+      checkPartialTransfer
+
+testFileRcvCancel :: IO ()
+testFileRcvCancel =
+  testChat2 aliceProfile bobProfile $
+    \alice bob -> do
+      connectUsers alice bob
+      startFileTransfer alice bob
+      bob ##> "/fs 1"
+      getTermLine bob >>= (`shouldStartWith` "receiving file 1 (test.jpg) progress")
+      waitFileExists "./tests/tmp/test.jpg"
+      bob ##> "/fc 1"
+      concurrentlyN_
+        [ do
+            bob <## "cancelled receiving file 1 (test.jpg) from alice"
+            bob ##> "/fs 1"
+            bob <## "receiving file 1 (test.jpg) cancelled, received part path: ./tests/tmp/test.jpg",
+          do
+            alice <## "bob cancelled receiving file 1 (test.jpg)"
+            alice ##> "/fs 1"
+            alice <## "sending file 1 (test.jpg) cancelled"
+        ]
+      checkPartialTransfer
+  where
+    waitFileExists f = unlessM (doesFileExist f) $ waitFileExists f
+
+testGroupFileTransfer :: IO ()
+testGroupFileTransfer =
+  testChat3 aliceProfile bobProfile cathProfile $
+    \alice bob cath -> do
+      createGroup3 "team" alice bob cath
+      alice #> "/f #team ./tests/fixtures/test.jpg"
+      alice <## "use /fc 1 to cancel sending"
+      concurrentlyN_
+        [ do
+            bob <# "#team alice> sends file test.jpg (136.5 KiB / 139737 bytes)"
+            bob <## "use /fr 1 [<dir>/ | <path>] to receive it",
+          do
+            cath <# "#team alice> sends file test.jpg (136.5 KiB / 139737 bytes)"
+            cath <## "use /fr 1 [<dir>/ | <path>] to receive it"
+        ]
+      alice ##> "/fs 1"
+      getTermLine alice >>= (`shouldStartWith` "sending file 1 (test.jpg) not accepted")
+      bob ##> "/fr 1 ./tests/tmp/"
+      bob <## "saving file 1 from alice to ./tests/tmp/test.jpg"
+      concurrentlyN_
+        [ do
+            alice <## "started sending file 1 (test.jpg) to bob"
+            alice <## "completed sending file 1 (test.jpg) to bob"
+            alice ##> "/fs 1"
+            alice <## "sending file 1 (test.jpg):"
+            alice <### ["  complete: bob", "  not accepted: cath"],
+          do
+            bob <## "started receiving file 1 (test.jpg) from alice"
+            bob <## "completed receiving file 1 (test.jpg) from alice"
+        ]
+      cath ##> "/fr 1 ./tests/tmp/"
+      cath <## "saving file 1 from alice to ./tests/tmp/test_1.jpg"
+      concurrentlyN_
+        [ do
+            alice <## "started sending file 1 (test.jpg) to cath"
+            alice <## "completed sending file 1 (test.jpg) to cath"
+            alice ##> "/fs 1"
+            getTermLine alice >>= (`shouldStartWith` "sending file 1 (test.jpg) complete"),
+          do
+            cath <## "started receiving file 1 (test.jpg) from alice"
+            cath <## "completed receiving file 1 (test.jpg) from alice"
+        ]
+
+startFileTransfer :: TestCC -> TestCC -> IO ()
+startFileTransfer alice bob = do
+  alice #> "/f @bob ./tests/fixtures/test.jpg"
+  alice <## "use /fc 1 to cancel sending"
+  bob <# "alice> sends file test.jpg (136.5 KiB / 139737 bytes)"
+  bob <## "use /fr 1 [<dir>/ | <path>] to receive it"
+  bob ##> "/fr 1 ./tests/tmp"
+  bob <## "saving file 1 from alice to ./tests/tmp/test.jpg"
+  concurrently_
+    (bob <## "started receiving file 1 (test.jpg) from alice")
+    (alice <## "started sending file 1 (test.jpg) to bob")
+
+checkPartialTransfer :: IO ()
+checkPartialTransfer = do
+  src <- B.readFile "./tests/fixtures/test.jpg"
+  dest <- B.readFile "./tests/tmp/test.jpg"
+  B.unpack src `shouldStartWith` B.unpack dest
+  B.length src > B.length dest `shouldBe` True
+
+connectUsers :: TestCC -> TestCC -> IO ()
+connectUsers cc1 cc2 = do
+  name1 <- showName cc1
+  name2 <- showName cc2
+  cc1 ##> "/c"
+  inv <- getInvitation cc1
+  cc2 ##> ("/c " <> inv)
+  concurrently_
+    (cc2 <## (name1 <> ": contact is connected"))
+    (cc1 <## (name2 <> ": contact is connected"))
+
+showName :: TestCC -> IO String
+showName (TestCC ChatController {currentUser} _ _ _ _) = do
+  User {localDisplayName, profile = Profile {fullName}} <- readTVarIO currentUser
+  pure . T.unpack $ localDisplayName <> " (" <> fullName <> ")"
+
+createGroup3 :: String -> TestCC -> TestCC -> TestCC -> IO ()
+createGroup3 gName cc1 cc2 cc3 = do
+  connectUsers cc1 cc2
+  connectUsers cc1 cc3
+  name2 <- userName cc2
+  name3 <- userName cc3
+  sName2 <- showName cc2
+  sName3 <- showName cc3
+  cc1 ##> ("/g " <> gName)
+  cc1 <## ("group #" <> gName <> " is created")
+  cc1 <## ("use /a " <> gName <> " <name> to add members")
+  addMember cc2
+  cc2 ##> ("/j " <> gName)
+  concurrently_
+    (cc1 <## ("#" <> gName <> ": " <> name2 <> " joined the group"))
+    (cc2 <## ("#" <> gName <> ": you joined the group"))
+  addMember cc3
+  cc3 ##> ("/j " <> gName)
+  concurrentlyN_
+    [ cc1 <## ("#" <> gName <> ": " <> name3 <> " joined the group"),
+      do
+        cc3 <## ("#" <> gName <> ": you joined the group")
+        cc3 <## ("#" <> gName <> ": member " <> sName2 <> " is connected"),
+      do
+        cc2 <## ("#" <> gName <> ": alice added " <> sName3 <> " to the group (connecting...)")
+        cc2 <## ("#" <> gName <> ": new member " <> name3 <> " is connected")
+    ]
+  where
+    addMember :: TestCC -> IO ()
+    addMember mem = do
+      name1 <- userName cc1
+      memName <- userName mem
+      cc1 ##> ("/a " <> gName <> " " <> memName)
+      concurrentlyN_
+        [ cc1 <## ("invitation to join the group #" <> gName <> " sent to " <> memName),
+          do
+            mem <## ("#" <> gName <> ": " <> name1 <> " invites you to join the group as admin")
+            mem <## ("use /j " <> gName <> " to accept")
+        ]
+
+-- | test sending direct messages
+(<##>) :: TestCC -> TestCC -> IO ()
+cc1 <##> cc2 = do
+  name1 <- userName cc1
+  name2 <- userName cc2
+  cc1 #> ("@" <> name2 <> " hi")
+  cc2 <# (name1 <> "> hi")
+  cc2 #> ("@" <> name1 <> " hey")
+  cc1 <# (name2 <> "> hey")
+
+userName :: TestCC -> IO [Char]
+userName (TestCC ChatController {currentUser} _ _ _ _) = T.unpack . localDisplayName <$> readTVarIO currentUser
+
+(##>) :: TestCC -> String -> IO ()
+cc ##> cmd = do
+  cc `send` cmd
+  cc <## cmd
+
+(#>) :: TestCC -> String -> IO ()
+cc #> cmd = do
+  cc `send` cmd
+  cc <# cmd
+
+send :: TestCC -> String -> IO ()
+send TestCC {chatController = cc} cmd = atomically $ writeTBQueue (inputQ cc) $ InputCommand cmd
+
+(<##) :: TestCC -> String -> Expectation
+cc <## line = getTermLine cc `shouldReturn` line
+
+(<###) :: TestCC -> [String] -> Expectation
+_ <### [] = pure ()
+cc <### ls = do
+  line <- getTermLine cc
+  if line `elem` ls
+    then cc <### filter (/= line) ls
+    else error $ "unexpected output: " <> line
+
+(<#) :: TestCC -> String -> Expectation
+cc <# line = (dropTime <$> getTermLine cc) `shouldReturn` line
+
+(</) :: TestCC -> Expectation
+(</) cc = timeout 500000 (getTermLine cc) `shouldReturn` Nothing
+
+dropTime :: String -> String
+dropTime msg = case splitAt 6 msg of
+  ([m, m', ':', s, s', ' '], text) ->
+    if all isDigit [m, m', s, s'] then text else error "invalid time"
+  _ -> error "invalid time"
+
+getTermLine :: TestCC -> IO String
+getTermLine = atomically . readTQueue . termQ
+
+getInvitation :: TestCC -> IO String
+getInvitation cc = do
+  cc <## "pass this invitation to your contact (via another channel):"
+  cc <## ""
+  inv <- getTermLine cc
+  cc <## ""
+  cc <## "and ask them to connect: /c <invitation_above>"
+  pure inv

tests/MarkdownTests.hs

@@ -0,0 +1,128 @@
+{-# LANGUAGE BlockArguments #-}
+{-# LANGUAGE OverloadedStrings #-}
+
+module MarkdownTests where
+
+import Data.Text (Text)
+import Simplex.Chat.Markdown
+import System.Console.ANSI.Types
+import Test.Hspec
+
+markdownTests :: Spec
+markdownTests = do
+  textFormat
+  secretText
+  textColor
+
+textFormat :: Spec
+textFormat = describe "text format (bold)" do
+  it "correct markdown" do
+    parseMarkdown "this is *bold formatted* text"
+      `shouldBe` "this is " <> Markdown Bold "bold formatted" <> " " <> "text"
+    parseMarkdown "*bold formatted* text"
+      `shouldBe` Markdown Bold "bold formatted" <> " " <> "text"
+    parseMarkdown "this is *bold*"
+      `shouldBe` "this is " <> Markdown Bold "bold"
+    parseMarkdown " *bold* text"
+      `shouldBe` " " <> Markdown Bold "bold" <> " " <> "text"
+    parseMarkdown "   *bold* text"
+      `shouldBe` "   " <> Markdown Bold "bold" <> " " <> "text"
+    parseMarkdown "this is *bold* "
+      `shouldBe` "this is " <> Markdown Bold "bold" <> " "
+    parseMarkdown "this is *bold*   "
+      `shouldBe` "this is " <> Markdown Bold "bold" <> "   "
+  it "ignored as markdown" do
+    parseMarkdown "this is * unformatted * text"
+      `shouldBe` "this is " <> "* unformatted *" <> " " <> "text"
+    parseMarkdown "this is *unformatted * text"
+      `shouldBe` "this is " <> "*unformatted *" <> " " <> "text"
+    parseMarkdown "this is * unformatted* text"
+      `shouldBe` "this is " <> "* unformatted*" <> " " <> "text"
+    parseMarkdown "this is **unformatted** text"
+      `shouldBe` "this is " <> "**" <> "unformatted** text"
+    parseMarkdown "this is*unformatted* text"
+      `shouldBe` "this is*unformatted* text"
+    parseMarkdown "this is *unformatted text"
+      `shouldBe` "this is " <> "*unformatted text"
+  it "ignored internal markdown" do
+    parseMarkdown "this is *long _bold_ (not italic)* text"
+      `shouldBe` "this is " <> Markdown Bold "long _bold_ (not italic)" <> " " <> "text"
+    parseMarkdown "snippet: `this is *bold text*`"
+      `shouldBe` "snippet: " <> Markdown Snippet "this is *bold text*"
+
+secretText :: Spec
+secretText = describe "secret text" do
+  it "correct markdown" do
+    parseMarkdown "this is #black_secret# text"
+      `shouldBe` "this is " <> Markdown Secret "black_secret" <> " " <> "text"
+    parseMarkdown "##black_secret### text"
+      `shouldBe` Markdown Secret "#black_secret##" <> " " <> "text"
+    parseMarkdown "this is #black secret# text"
+      `shouldBe` "this is " <> Markdown Secret "black secret" <> " " <> "text"
+    parseMarkdown "##black secret### text"
+      `shouldBe` Markdown Secret "#black secret##" <> " " <> "text"
+    parseMarkdown "this is #secret#"
+      `shouldBe` "this is " <> Markdown Secret "secret"
+    parseMarkdown " #secret# text"
+      `shouldBe` " " <> Markdown Secret "secret" <> " " <> "text"
+    parseMarkdown "   #secret# text"
+      `shouldBe` "   " <> Markdown Secret "secret" <> " " <> "text"
+    parseMarkdown "this is #secret# "
+      `shouldBe` "this is " <> Markdown Secret "secret" <> " "
+    parseMarkdown "this is #secret#   "
+      `shouldBe` "this is " <> Markdown Secret "secret" <> "   "
+  it "ignored as markdown" do
+    parseMarkdown "this is # unformatted # text"
+      `shouldBe` "this is " <> "# unformatted #" <> " " <> "text"
+    parseMarkdown "this is #unformatted # text"
+      `shouldBe` "this is " <> "#unformatted #" <> " " <> "text"
+    parseMarkdown "this is # unformatted# text"
+      `shouldBe` "this is " <> "# unformatted#" <> " " <> "text"
+    parseMarkdown "this is ## unformatted ## text"
+      `shouldBe` "this is " <> "## unformatted ##" <> " " <> "text"
+    parseMarkdown "this is#unformatted# text"
+      `shouldBe` "this is#unformatted# text"
+    parseMarkdown "this is #unformatted text"
+      `shouldBe` "this is " <> "#unformatted text"
+  it "ignored internal markdown" do
+    parseMarkdown "snippet: `this is #secret_text#`"
+      `shouldBe` "snippet: " <> Markdown Snippet "this is #secret_text#"
+
+red :: Text -> Markdown
+red = Markdown (Colored Red)
+
+textColor :: Spec
+textColor = describe "text color (red)" do
+  it "correct markdown" do
+    parseMarkdown "this is !1 red color! text"
+      `shouldBe` "this is " <> red "red color" <> " " <> "text"
+    parseMarkdown "!1 red! text"
+      `shouldBe` red "red" <> " " <> "text"
+    parseMarkdown "this is !1 red!"
+      `shouldBe` "this is " <> red "red"
+    parseMarkdown " !1 red! text"
+      `shouldBe` " " <> red "red" <> " " <> "text"
+    parseMarkdown "   !1 red! text"
+      `shouldBe` "   " <> red "red" <> " " <> "text"
+    parseMarkdown "this is !1 red! "
+      `shouldBe` "this is " <> red "red" <> " "
+    parseMarkdown "this is !1 red!   "
+      `shouldBe` "this is " <> red "red" <> "   "
+  it "ignored as markdown" do
+    parseMarkdown "this is !1 unformatted ! text"
+      `shouldBe` "this is " <> "!1 unformatted !" <> " " <> "text"
+    parseMarkdown "this is !1 unformatted ! text"
+      `shouldBe` "this is " <> "!1 unformatted !" <> " " <> "text"
+    parseMarkdown "this is !1  unformatted! text"
+      `shouldBe` "this is " <> "!1  unformatted!" <> " " <> "text"
+    -- parseMarkdown "this is !!1 unformatted!! text"
+    --   `shouldBe` "this is " <> "!!1" <> "unformatted!! text"
+    parseMarkdown "this is!1 unformatted! text"
+      `shouldBe` "this is!1 unformatted! text"
+    parseMarkdown "this is !1 unformatted text"
+      `shouldBe` "this is " <> "!1 unformatted text"
+  it "ignored internal markdown" do
+    parseMarkdown "this is !1 long *red* (not bold)! text"
+      `shouldBe` "this is " <> red "long *red* (not bold)" <> " " <> "text"
+    parseMarkdown "snippet: `this is !1 red text!`"
+      `shouldBe` "snippet: " <> Markdown Snippet "this is !1 red text!"

tests/ProtocolTests.hs

@@ -0,0 +1,47 @@
+{-# LANGUAGE OverloadedLists #-}
+{-# LANGUAGE OverloadedStrings #-}
+
+module ProtocolTests where
+
+import Data.ByteString.Char8 (ByteString)
+import Simplex.Chat.Protocol
+import Simplex.Messaging.Parsers (parseAll)
+import Test.Hspec
+
+protocolTests :: Spec
+protocolTests = do
+  parseChatMessageTest
+
+(#==) :: ByteString -> RawChatMessage -> Expectation
+s #== msg = parseAll rawChatMessageP s `shouldBe` Right msg
+
+parseChatMessageTest :: Spec
+parseChatMessageTest = describe "Raw chat message format" $ do
+  it "no parameters and content" $
+    "5 x.grp.mem.leave   " #== RawChatMessage (Just 5) "x.grp.mem.leave" [] []
+  it "one parameter, no content" $
+    "6 x.msg.del 3  " #== RawChatMessage (Just 6) "x.msg.del" ["3"] []
+  it "with content that fits the message" $
+    "7 x.msg.new c.text x.text:11 hello there "
+      #== RawChatMessage
+        (Just 7)
+        "x.msg.new"
+        ["c.text"]
+        [RawMsgBodyContent (RawContentType "x" "text") "hello there"]
+  it "with DAG reference and partial content" $
+    "8 x.msg.new c.image x.dag:16,x.text:7,m.image/jpg:6 0123456789012345 picture abcdef "
+      #== RawChatMessage
+        (Just 8)
+        "x.msg.new"
+        ["c.image"]
+        [ RawMsgBodyContent (RawContentType "x" "dag") "0123456789012345",
+          RawMsgBodyContent (RawContentType "x" "text") "picture",
+          RawMsgBodyContent (RawContentType "m" "image/jpg") "abcdef"
+        ]
+  it "without message id" $
+    " x.grp.mem.inv 23456,123 x.json:46 {\"contactRef\":\"john\",\"displayName\":\"John Doe\"} "
+      #== RawChatMessage
+        Nothing
+        "x.grp.mem.inv"
+        ["23456", "123"]
+        [RawMsgBodyContent (RawContentType "x" "json") "{\"contactRef\":\"john\",\"displayName\":\"John Doe\"}"]

tests/Test.hs

@@ -0,0 +1,11 @@
+import ChatClient
+import ChatTests
+import MarkdownTests
+import ProtocolTests
+import Test.Hspec
+
+main :: IO ()
+main = withSmpServer . hspec $ do
+  describe "SimpleX chat markdown" markdownTests
+  describe "SimpleX chat protocol" protocolTests
+  describe "SimpleX chat client" chatTests

tests/fixtures/test.txt

@@ -0,0 +1 @@
+hello there