Merge branch 'master' into f/ui-images

This commit is contained in:
JRoberts
2022-04-15 19:36:06 +04:00
38 changed files with 842 additions and 414 deletions
+95 -255
View File
@@ -1,6 +1,6 @@
<img src="images/simplex-chat-logo.svg" alt="SimpleX logo" width="100%">
# SimpleX - the first chat platform that is 100% private by design - it has no access to your connection graph!
# SimpleX - the first messaging platform that has no user identifiers of any kind - 100% private by design!
[![GitHub build](https://github.com/simplex-chat/simplex-chat/workflows/build/badge.svg)](https://github.com/simplex-chat/simplex-chat/actions?query=workflow%3Abuild)
[![GitHub downloads](https://img.shields.io/github/downloads/simplex-chat/simplex-chat/total)](https://github.com/simplex-chat/simplex-chat/releases)
@@ -19,280 +19,120 @@
[<img src="https://github.com/simplex-chat/.github/blob/master/profile/images/apk_icon.png" alt="APK" height="41">](https://github.com/simplex-chat/website/raw/master/simplex.apk)
- 🖲 Protects your messages and metadata - who you talk to and when.
- 🔐 Double ratchet encryption.
- 📱 Mobile apps for Android ([Google Play](https://play.google.com/store/apps/details?id=chat.simplex.app), [APK](https://github.com/simplex-chat/website/raw/master/simplex.apk)) and [iOS](https://apps.apple.com/us/app/simplex-chat/id1605771084). [See the announcement here](https://github.com/simplex-chat/simplex-chat/blob/master/blog/20220308-simplex-chat-mobile-apps.md).
- 🔐 Double ratchet end-to-end encryption, with additional encryption layer.
- 📱 Mobile apps for Android ([Google Play](https://play.google.com/store/apps/details?id=chat.simplex.app), [APK](https://github.com/simplex-chat/website/raw/master/simplex.apk)) and [iOS](https://apps.apple.com/us/app/simplex-chat/id1605771084).
- 🚀 [TestFlight preview for iOS](https://testflight.apple.com/join/DWuT2LQu) with the new features 1-2 weeks earlier - **limited to 10,000 users**!
- 🖥 Available as a [terminal (console) app / CLI](#zap-quick-installation-of-a-terminal-app) on Linux, MacOS, Windows.
- 🖥 Available as a terminal (console) app / CLI on Linux, MacOS, Windows.
See [SimpleX overview](https://github.com/simplex-chat/simplexmq/blob/master/protocol/overview-tjr.md) for more information on platform objectives and technical design.
## Why privacy of communications matter
### :zap: Quick installation of a terminal app
Everyone should care about privacy and security of their communications - innocuous conversations can put you in danger even if there is nothing to hide.
One of the most shocking stories is the experience of [Mohamedou Ould Salahi](https://en.wikipedia.org/wiki/Mohamedou_Ould_Slahi) that he wrote about in his memoir and that is shown in The Mauritanian movie. He was put into Guantanamo camp, without trial, and was tortured there for 15 years after a phone call to his relative in Afghanistan, under suspicion of being involved in 9/11 attacks, even though he lived in Germany for the 10 years prior to the attacks.
It is not enough to use an end-to-end encrypted messenger, we all should use the messengers that protect the privacy of our personal networks - who we are connected with.
## SimpleX unique approach to privacy and security
### Full privacy of your identity, profile, contacts and metadata
**Unlike any other existing messaging platform, SimpleX has no identifiers assigned to the users** - not even random numbers. This protects the privacy of who are you communicating with, hiding it from SimpleX platform servers and from any observers. [Read more](./docs/SIMPLEX.md#full-privacy-of-your-identity-profile-contacts-and-metadata).
### The best protection against spam and abuse
As you have no identifier on SimpleX platform, you cannot be contacted unless you share a one-time invitation link or an optional temporary user address. [Read more](./docs/SIMPLEX.md#the-best-protection-against-spam-and-abuse).
### Complete ownership, control and security of your data
SimpleX stores all user data on client devices, the messages are only held temporarily on SimpleX relay servers until they are received. [Read more](./docs/SIMPLEX.md#complete-ownership-control-and-security-of-your-data).
### Users own SimpleX network
You can use SimpleX with your own servers and still communicate with people using the servers that are pre-configured in the apps or any other SimpleX servers. [Read more](./docs/SIMPLEX.md#users-own-simplex-network).
## For developers
We plan that the SimpleX platform will grow into the platform supporting any distributed Internet application. This will allow you to build any service that people can access via chat, with custom web-based UI widgets that anybody with basic HTML/CSS/JavaScript knowledge can create in a few hours.
You already can:
- use SimpleX Chat library to integrate chat functionality into your apps.
- use SimpleX Chat bot templates in Haskell to build your own chat bot services (TypeScript SDK is coming soon).
If you are considering developing with SimpleX platform please get in touch for any advice and support.
## News and updates
[Apr 04, 2022. Instant notifications for SimpleX Chat mobile apps](./blog/20220404-simplex-chat-instant-notifications.md). We would really appreciate any feedback on the design we are implementing.
[Mar 08, 2022 Mobile apps for iOS and Android released](./blog/20220308-simplex-chat-mobile-apps.md)
[Feb 14, 2022. SimpleX Chat: join our public beta for iOS](./blog/20220214-simplex-chat-ios-public-beta.md)
[All updates](./blog)
## Make a private connection
You need to share a link or scan a QR code (in person or during a video call) to make a connection and start messaging.
The channel through which you share the link does not have to be secure - it is enough that you can confirm who sent you the message and that your SimpleX connection is established.
<img src="https://github.com/simplex-chat/.github/blob/master/profile/images/conversation.png" alt="Make a private connection" width="594" height="360">
## :zap: Quick installation of a terminal app
```sh
curl -o- https://raw.githubusercontent.com/simplex-chat/simplex-chat/master/install.sh | bash
curl -o- https://raw.githubusercontent.com/simplex-chat/simplex-chat/stable/install.sh | bash
```
Once the chat client is installed, simply run `simplex-chat` from your terminal.
![simplex-chat](./images/connection.gif)
## Table of contents
Read more about [installing and using the terminal app](./docs/CLI.md).
- [Disclaimer](#disclaimer)
- [Network topology](#network-topology)
- [Terminal chat features](#terminal-chat-features)
- [Installation](#🚀-installation)
- [Download chat client](#download-chat-client)
- [Linux and MacOS](#linux-and-macos)
- [Windows](#windows)
- [Build from source](#build-from-source)
- [Using Docker](#using-docker)
- [Using Haskell stack](#using-haskell-stack)
- [Usage](#usage)
- [Running the chat client](#running-the-chat-client)
- [How to use SimpleX chat](#how-to-use-simplex-chat)
- [Groups](#groups)
- [Sending files](#sending-files)
- [User contact addresses](#user-contact-addresses)
- [Access chat history](#access-chat-history)
- [Roadmap](#Roadmap)
- [License](#license)
## SimpleX Platform design
## Disclaimer
SimpleX is a client-server network with a unique network topology that uses redundant, disposable message relay nodes to asynchronously pass messages via unidirectional (simplex) message queues, providing recipient and sender anonymity.
SimpleX Chat implements a new network topology for asynchronous communication combining the advantages and avoiding the disadvantages of federated and P2P networks.
Unlike P2P networks, all messages are passed through one or several server nodes, that do not even need to have persistence. In fact, the current [SMP server implementation](https://github.com/simplex-chat/simplexmq#smp-server) uses in-memory message storage, persisting only the queue records. SimpleX provides better metadata protection than P2P designs, as no global participant identifiers are used to deliver messages, and avoids [the problems of P2P networks](./docs/SIMPLEX.md#comparison-with-p2p-messaging-protocols).
[SimpleXMQ security model](https://github.com/simplex-chat/simplexmq/blob/master/protocol/overview-tjr.md) had many improvements in v1.0.0; the implementation has not been audited yet.
Unlike federated networks, the server nodes **do not have records of the users**, **do not communicate with each other** and **do not store messages** after they are delivered to the recipients. There is no way to discover the full list of servers participating in SimpleX network. This design avoids the problem of metadata visibility that all federated networks have and better protects from the network-wide attacks.
We use SimpleX Chat all the time, but you may find some bugs. We would really appreciate if you use it and let us know anything that needs to be fixed or improved.
Only the client devices have information about users, their contacts and groups.
## Network topology
SimpleX is a client-server network that uses redundant, disposable nodes to asynchronously pass messages via message queues, providing receiver and sender anonymity.
Unlike P2P networks, all messages are passed through one or several (for redundancy) servers, that do not even need to have persistence (in fact, the current [SMP server implementation](https://github.com/simplex-chat/simplexmq#smp-server) uses in-memory message storage, persisting only the queue records) - it provides better metadata protection than P2P designs, as no global participant ID is required, and avoids many [problems of P2P networks](https://github.com/simplex-chat/simplex-chat/blob/master/simplex.md#comparison-with-p2p-messaging-protocols).
Unlike federated networks, the participating server nodes **do not have records of the users**, **do not communicate with each other**, **do not store messages** after they are delivered to the recipients, and there is no way to discover the full list of participating servers. SimpleX network avoids the problem of metadata visibility that federated networks have and better protects the network, as servers do not communicate with each other. Each server node provides unidirectional "dumb pipes" to the users, that do authorization without authentication, having no knowledge of the the users or their contacts. Each queue is assigned two Ed448 keys - one for receiver and one for sender - and each queue access is authorized with a signature created using a respective key's private counterpart.
The routing of messages relies on the knowledge of client devices how user contacts and groups map at any given moment of time to these disposable queues on server nodes.
## Terminal chat features
- 1-to-1 chat with multiple people in the same terminal window.
- Group messaging.
- Sending files to contacts and groups.
- User contact addresses - establish connections via multiple-use contact links.
- Messages persisted in a local SQLite database.
- Auto-populated recipient name - just type your messages to reply to the sender once the connection is established.
- Demo SMP servers available and pre-configured in the app - or you can [deploy your own server](https://github.com/simplex-chat/simplexmq#using-smp-server-and-smp-agent).
- No global identity or any names visible to the server(s), ensuring full privacy of your contacts and conversations.
- Two layers of E2E encryption (double-ratchet for duplex connections, using X3DH key agreement with ephemeral Curve448 keys, and NaCl crypto_box for SMP queues, using Curve25519 keys) and out-of-band passing of recipient keys (see [How to use SimpleX chat](#how-to-use-simplex-chat)).
- Message integrity validation (via including the digests of the previous messages).
- Authentication of each command/message by SMP servers with automatically generated Ed448 keys.
- TLS 1.3 transport encryption.
- Additional encryption of messages from SMP server to recipient to reduce traffic correlation.
Public keys involved in key exchange are not used as identity, they are randomly generated for each contact.
See [Encryption Primitives Used](https://github.com/simplex-chat/simplexmq/blob/master/protocol/overview-tjr.md#encryption-primitives-used) for technical details.
<a name="🚀-installation"></a>
## 🚀 Installation
### Download chat client
#### Linux and MacOS
To **install** or **update** `simplex-chat`, you should run the install script. To do that, use the following cURL or Wget command:
```sh
curl -o- https://raw.githubusercontent.com/simplex-chat/simplex-chat/master/install.sh | bash
```
```sh
wget -qO- https://raw.githubusercontent.com/simplex-chat/simplex-chat/master/install.sh | bash
```
Once the chat client downloads, you can run it with `simplex-chat` command in your terminal.
Alternatively, you can manually download the chat binary for your system from the [latest stable release](https://github.com/simplex-chat/simplex-chat/releases) and make it executable as shown below.
```sh
chmod +x <binary>
mv <binary> ~/.local/bin/simplex-chat
```
(or any other preferred location on `PATH`).
On MacOS you also need to [allow Gatekeeper to run it](https://support.apple.com/en-us/HT202491).
#### Windows
```sh
move <binary> %APPDATA%/local/bin/simplex-chat.exe
```
### Build from source
> **Please note:** to build the app use source code from [stable branch](https://github.com/simplex-chat/simplex-chat/tree/stable).
#### Using Docker
On Linux, you can build the chat executable using [docker build with custom output](https://docs.docker.com/engine/reference/commandline/build/#custom-build-outputs):
```shell
$ git clone git@github.com:simplex-chat/simplex-chat.git
$ cd simplex-chat
$ git checkout stable
$ DOCKER_BUILDKIT=1 docker build --output ~/.local/bin .
```
> **Please note:** If you encounter `` version `GLIBC_2.28' not found `` error, rebuild it with `haskell:8.10.4-stretch` base image (change it in your local [Dockerfile](Dockerfile)).
#### Using Haskell stack
Install [Haskell stack](https://docs.haskellstack.org/en/stable/README/):
```shell
curl -sSL https://get.haskellstack.org/ | sh
```
and build the project:
```shell
$ git clone git@github.com:simplex-chat/simplex-chat.git
$ cd simplex-chat
$ git checkout stable
$ stack install
```
## Usage
### Running the chat client
To start the chat client, run `simplex-chat` from the terminal.
By default, app data directory is created in the home directory (`~/.simplex`, or `%APPDATA%/simplex` on Windows), and two SQLite database files `simplex_v1_chat.db` and `simplex_v1_agent.db` are initialized in it.
To specify a different file path prefix for the database files use `-d` command line option:
```shell
$ simplex-chat -d alice
```
Running above, for example, would create `alice_v1_chat.db` and `alice_v1_agent.db` database files in current directory.
Three default SMP servers are hosted on Linode - they are [pre-configured in the app](https://github.com/simplex-chat/simplex-chat/blob/master/src/Simplex/Chat/Options.hs#L42).
If you deployed your own SMP server(s) you can configure client via `-s` option:
```shell
$ simplex-chat -s smp://LcJUMfVhwD8yxjAiSaDzzGF3-kLG4Uh0Fl_ZIjrRwjI=@smp.example.com
```
Base64url encoded string preceding the server address is the server's offline certificate fingerprint which is validated by client during TLS handshake.
You can still talk to people using default or any other server - it only affects the location of the message queue when you initiate the connection (and the reply queue can be on another server, as set by the other party's client).
Run `simplex-chat -h` to see all available options.
### How to use SimpleX chat
Once you have started the chat, you will be prompted to specify your "display name" and an optional "full name" to create a local chat profile. Your display name is an alias for your contacts to refer to you by - it is not unique and does not serve as a global identity. If some of your contacts chose the same display name, the chat client adds a numeric suffix to their local display name.
The diagram below shows how to connect and message a contact:
<div align="center">
<img align="center" src="images/how-to-use-simplex.svg">
</div>
Once you've set up your local profile, enter `/c` (for `/connect`) to create a new connection and generate an invitation. Send this invitation to your contact via any other channel.
You are able to create multiple invitations by entering `/connect` multiple times and sending these invitations to the corresponding contacts you'd like to connect with.
The invitation can only be used once and even if this is intercepted, the attacker would not be able to use it to send you the messages via this queue once your contact confirms that the connection is established. See agent protocol for explanation of [invitation format](https://github.com/simplex-chat/simplexmq/blob/master/protocol/agent-protocol.md#connection-request).
The contact who received the invitation should enter `/c <invitation>` to accept the connection. This establishes the connection, and both parties are notified.
They would then use `@<name> <message>` commands to send messages. You may also just start typing a message to send it to the contact that was the last.
Use `/help` in chat to see the list of available commands.
### Groups
To create a group use `/g <group>`, then add contacts to it with `/a <group> <name>`. You can then send messages to the group by entering `#<group> <message>`. Use `/help groups` for other commands.
![simplex-chat](./images/groups.gif)
> **Please note**: the groups are not stored on any server, they are maintained as a list of members in the app database to whom the messages will be sent.
### Sending files
You can send a file to your contact with `/f @<contact> <file_path>` - the recipient will have to accept it before it is sent. Use `/help files` for other commands.
![simplex-chat](./images/files.gif)
You can send files to a group with `/f #<group> <file_path>`.
### User contact addresses
As an alternative to one-time invitation links, you can create a long-term address with `/ad` (for `/address`). The created address can then be shared via any channel, and used by other users as a link to make a contact request with `/c <user_contact_address>`.
You can accept or reject incoming requests with `/ac <name>` and `/rc <name>` commands.
User address is "long-term" in a sense that it is a multiple-use connection link - it can be used until it is deleted by the user, in which case all established connections would still remain active (unlike how it works with email, when changing the address results in people not being able to message you).
Use `/help address` for other commands.
![simplex-chat](./images/user-addresses.gif)
### Access chat history
SimpleX chat stores all your contacts and conversations in a local SQLite database, making it private and portable by design, owned and controlled by user.
You can view and search your chat history by querying your database. Run the below script to create message views in your database.
```sh
curl -o- https://raw.githubusercontent.com/simplex-chat/simplex-chat/master/message_views.sql | sqlite3 ~/.simplex/simplex_v1_chat.db
```
Open SQLite Command Line Shell:
```sh
sqlite3 ~/.simplex/simplex_v1_chat.db
```
See [Message queries](./message_queries.md) for examples.
> **Please note:** SQLite foreign key constraints are disabled by default, and must be **[enabled separately for each database connection](https://sqlite.org/foreignkeys.html#fk_enable)**. The latter can be achieved by running `PRAGMA foreign_keys = ON;` command on an open database connection. By running data altering queries without enabling foreign keys prior to that, you may risk putting your database in an inconsistent state.
**Convenience queries**
Get all messages from today (`chat_dt` is in UTC):
```sql
select * from all_messages_plain where date(chat_dt) > date('now', '-1 day') order by chat_dt;
```
Get overnight messages in the morning:
```sql
select * from all_messages_plain where chat_dt > datetime('now', '-15 hours') order by chat_dt;
```
See [SimpleX whitepaper](https://github.com/simplex-chat/simplexmq/blob/master/protocol/overview-tjr.md) for more information on platform objectives and technical design.
## Roadmap
1. Mobile and desktop apps (in progress).
2. SMP protocol improvements:
- SMP queue redundancy and rotation.
- Message delivery confirmation.
- Support multiple devices.
3. Privacy-preserving identity server for optional DNS-based contact/group addresses to simplify connection and discovery, but not used to deliver messages:
- keep all your contacts and groups even if you lose the domain.
- the server doesn't have information about your contacts and groups.
4. Media server to optimize sending large files to groups.
5. Channels server for large groups and broadcast channels.
- ✅ Easy to deploy SimpleX server with in-memory message storage, without any dependencies.
- ✅ Terminal (console) client with groups and files support.
- ✅ One-click SimpleX server deployment on Linode.
- ✅ End-to-end encryption using double-ratchet protocol with additional encryption layer.
- ✅ Mobile apps v1 for Android and iOS.
- Private instant notifications for Android using background service.
- ✅ Haskell chat bot templates
- 🏗 Privacy preserving instant notifications for iOS using Apple Push Notification service (in progress).
- 🏗 Mobile app v2 - supporting files, images and groups etc. (in progress).
- 🏗 Chat server and TypeScript client SDK to develop chat interfaces, integrations and chat bots (in progress).
- Chat database portability and encryption.
- End-to-end encrypted audio and video calls via the mobile apps.
- Web widgets for custom interactivity in the chats.
- SMP protocol improvements:
- SMP queue redundancy and rotation.
- Message delivery confirmation.
- Supporting the same profile on multiple devices.
- Privacy-preserving identity server for optional DNS-based contact/group addresses to simplify connection and discovery, but not used to deliver messages:
- keep all your contacts and groups even if you lose the domain.
- the server doesn't have information about your contacts and groups.
- Media server to optimize sending large files to groups.
- Channels server for large groups and broadcast channels.
## Disclaimer
[SimpleX protocols and security model](https://github.com/simplex-chat/simplexmq/blob/master/protocol/overview-tjr.md) was reviewed and had many improvements in v1.0.0; we are currently arranging for the independent implementation audit.
You are likely to discover some bugs - we would really appreciate if you use it and let us know anything that needs to be fixed or improved.
## License
+2 -2
View File
@@ -11,8 +11,8 @@ android {
applicationId "chat.simplex.app"
minSdk 29
targetSdk 32
versionCode 22
versionName "1.5"
versionCode 23
versionName "1.5.1"
testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
ndk {
@@ -100,9 +100,14 @@ fun processIntent(intent: Intent?, chatModel: ChatModel) {
Log.d(TAG, "processIntent: OpenChatAction $chatId")
if (chatId != null) {
val cInfo = chatModel.getChat(chatId)?.chatInfo
chatModel.clearOverlays.value = true
if (cInfo != null) withApi { openChat(chatModel, cInfo) }
}
}
NtfManager.ShowChatsAction -> {
Log.d(TAG, "processIntent: ShowChatsAction")
chatModel.clearOverlays.value = true
}
"android.intent.action.VIEW" -> {
val uri = intent.data
if (uri != null) connectIfOpenedViaUri(uri, chatModel)
@@ -27,6 +27,10 @@ class ChatModel(val controller: ChatController) {
var terminalItems = mutableStateListOf<TerminalItem>()
var userAddress = mutableStateOf<String?>(null)
var userSMPServers = mutableStateOf<(List<String>)?>(null)
// set when app opened from external intent
var clearOverlays = mutableStateOf<Boolean>(false)
// set when app is opened via contact or invitation URI
var appOpenUrl = mutableStateOf<Uri?>(null)
var runServiceInBackground = mutableStateOf(true)
@@ -14,6 +14,7 @@ class NtfManager(val context: Context) {
const val MessageChannel: String = "chat.simplex.app.MESSAGE_NOTIFICATION"
const val MessageGroup: String = "chat.simplex.app.MESSAGE_NOTIFICATION"
const val OpenChatAction: String = "chat.simplex.app.OPEN_CHAT"
const val ShowChatsAction: String = "chat.simplex.app.SHOW_CHATS"
}
private val manager: NotificationManager = context.getSystemService(Context.NOTIFICATION_SERVICE) as NotificationManager
@@ -28,6 +29,18 @@ class NtfManager(val context: Context) {
))
}
fun cancelNotificationsForChat(chatId: String) {
prevNtfTime.remove(chatId)
manager.cancel(chatId.hashCode())
val msgNtfs = manager.activeNotifications.filter {
ntf -> ntf.notification.channelId == MessageChannel
}
if (msgNtfs.count() == 1) {
// Have a group notification with no children so cancel it
manager.cancel(0)
}
}
fun notifyMessageReceived(cInfo: ChatInfo, cItem: ChatItem) {
Log.d(TAG, "notifyMessageReceived ${cInfo.id}")
val now = Clock.System.now().toEpochMilliseconds()
@@ -53,6 +66,7 @@ class NtfManager(val context: Context) {
.setGroup(MessageGroup)
.setGroupAlertBehavior(NotificationCompat.GROUP_ALERT_CHILDREN)
.setGroupSummary(true)
.setContentIntent(getSummaryNtfIntent())
.build()
with(NotificationManagerCompat.from(context)) {
@@ -62,7 +76,7 @@ class NtfManager(val context: Context) {
}
}
private fun hideSecrets(cItem: ChatItem): String {
private fun hideSecrets(cItem: ChatItem) : String {
val md = cItem.formattedText
return if (md == null) {
cItem.content.text
@@ -87,4 +101,16 @@ class NtfManager(val context: Context) {
getPendingIntent(uniqueInt, PendingIntent.FLAG_IMMUTABLE)
}
}
private fun getSummaryNtfIntent() : PendingIntent{
Log.d(TAG, "getSummaryNtfIntent")
val uniqueInt = (System.currentTimeMillis() and 0xfffffff).toInt()
val intent = Intent(context, MainActivity::class.java)
.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK or Intent.FLAG_ACTIVITY_SINGLE_TOP or Intent.FLAG_ACTIVITY_CLEAR_TOP)
.setAction(ShowChatsAction)
return TaskStackBuilder.create(context).run {
addNextIntentWithParentStack(intent)
getPendingIntent(uniqueInt, PendingIntent.FLAG_IMMUTABLE)
}
}
}
@@ -29,7 +29,7 @@ typealias ChatCtrl = Long
open class ChatController(private val ctrl: ChatCtrl, private val ntfManager: NtfManager, val appContext: Context) {
var chatModel = ChatModel(this)
private val sharedPreferences: SharedPreferences = appContext.getSharedPreferences(SHARED_PREFS_ID, Context.MODE_PRIVATE)
private val sharedPreferences: SharedPreferences = appContext.getSharedPreferences(SHARED_PREFS_ID, Context.MODE_PRIVATE)
init {
chatModel.runServiceInBackground.value = getRunServiceInBackground()
@@ -72,6 +72,10 @@ open class ChatController(private val ctrl: ChatCtrl, private val ntfManager: Nt
return false
}
fun cancelNotificationsForChat(chatId: String) {
ntfManager.cancelNotificationsForChat(chatId)
}
suspend fun sendCmd(cmd: CC): CR {
return withContext(Dispatchers.IO) {
val c = cmd.cmdString
@@ -62,6 +62,7 @@ fun ChatView(chatModel: ChatModel) {
delay(1000L)
if (chat.chatItems.count() > 0) {
chatModel.markChatItemsRead(chat.chatInfo)
chatModel.controller.cancelNotificationsForChat(chat.id)
withApi {
chatModel.controller.apiChatRead(
chat.chatInfo.chatType,
@@ -4,43 +4,62 @@ import androidx.compose.foundation.layout.*
import androidx.compose.material.Icon
import androidx.compose.material.Text
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Edit
import androidx.compose.material.icons.filled.*
import androidx.compose.runtime.Composable
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.tooling.preview.Preview
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import chat.simplex.app.model.CIDirection
import chat.simplex.app.model.ChatItem
import chat.simplex.app.model.*
import chat.simplex.app.ui.theme.HighOrLowlight
import chat.simplex.app.ui.theme.SimplexBlue
import kotlinx.datetime.Clock
@Composable
fun CIMetaView(chatItem: ChatItem) {
Row(
horizontalArrangement = Arrangement.spacedBy(4.dp),
verticalAlignment = Alignment.CenterVertically
) {
Row(verticalAlignment = Alignment.CenterVertically) {
if (!chatItem.isDeletedContent) {
if (chatItem.meta.itemEdited) {
Icon(
Icons.Filled.Edit,
modifier = Modifier.height(12.dp),
modifier = Modifier.height(12.dp).padding(end = 1.dp),
contentDescription = "Edited",
tint = HighOrLowlight,
)
}
// TODO status
CIStatusView(chatItem.meta.itemStatus)
}
Text(
chatItem.timestampText,
color = HighOrLowlight,
fontSize = 14.sp
fontSize = 14.sp,
modifier = Modifier.padding(start = 3.dp)
)
}
}
@Composable
fun CIStatusView(status: CIStatus) {
when (status) {
is CIStatus.SndSent -> {
Icon(Icons.Filled.Check, "sent", Modifier.height(12.dp), tint = HighOrLowlight)
}
is CIStatus.SndErrorAuth -> {
Icon(Icons.Filled.Close, "unauthorized send", Modifier.height(12.dp), tint = Color.Red)
}
is CIStatus.SndError -> {
Icon(Icons.Filled.WarningAmber, "send failed", Modifier.height(12.dp), tint = Color.Yellow)
}
is CIStatus.RcvNew -> {
Icon(Icons.Filled.Circle, "unread", Modifier.height(12.dp), tint = SimplexBlue)
}
else -> {}
}
}
@Preview
@Composable
fun PreviewCIMetaView() {
@@ -51,6 +70,48 @@ fun PreviewCIMetaView() {
)
}
@Preview
@Composable
fun PreviewCIMetaViewUnread() {
CIMetaView(
chatItem = ChatItem.getSampleData(
1, CIDirection.DirectSnd(), Clock.System.now(), "hello",
status = CIStatus.RcvNew()
)
)
}
@Preview
@Composable
fun PreviewCIMetaViewSendFailed() {
CIMetaView(
chatItem = ChatItem.getSampleData(
1, CIDirection.DirectSnd(), Clock.System.now(), "hello",
status = CIStatus.SndError(AgentErrorType.CMD(CommandErrorType.SYNTAX()))
)
)
}
@Preview
@Composable
fun PreviewCIMetaViewSendNoAuth() {
CIMetaView(
chatItem = ChatItem.getSampleData(
1, CIDirection.DirectSnd(), Clock.System.now(), "hello", status = CIStatus.SndErrorAuth()
)
)
}
@Preview
@Composable
fun PreviewCIMetaViewSendSent() {
CIMetaView(
chatItem = ChatItem.getSampleData(
1, CIDirection.DirectSnd(), Clock.System.now(), "hello", status = CIStatus.SndSent()
)
)
}
@Preview
@Composable
fun PreviewCIMetaViewEdited() {
@@ -62,6 +123,30 @@ fun PreviewCIMetaViewEdited() {
)
}
@Preview
@Composable
fun PreviewCIMetaViewEditedUnread() {
CIMetaView(
chatItem = ChatItem.getSampleData(
1, CIDirection.DirectRcv(), Clock.System.now(), "hello",
itemEdited = true,
status=CIStatus.RcvNew()
)
)
}
@Preview
@Composable
fun PreviewCIMetaViewEditedSent() {
CIMetaView(
chatItem = ChatItem.getSampleData(
1, CIDirection.DirectSnd(), Clock.System.now(), "hello",
itemEdited = true,
status=CIStatus.SndSent()
)
)
}
@Preview
@Composable
fun PreviewCIMetaViewDeletedContent() {
@@ -47,7 +47,7 @@ fun MarkdownText (
senderBold: Boolean = false,
modifier: Modifier = Modifier
) {
val reserve = if (edited) " " else " "
val reserve = if (edited) " " else " "
if (formattedText == null) {
val annotatedText = buildAnnotatedString {
appendSender(this, sender, senderBold)
@@ -18,6 +18,7 @@ import chat.simplex.app.model.ChatModel
import chat.simplex.app.ui.theme.ToolbarDark
import chat.simplex.app.ui.theme.ToolbarLight
import chat.simplex.app.views.chat.ChatHelpView
import chat.simplex.app.views.newchat.ModalManager
import chat.simplex.app.views.newchat.NewChatSheet
import chat.simplex.app.views.usersettings.SettingsView
import kotlinx.coroutines.CoroutineScope
@@ -63,6 +64,11 @@ fun scaffoldController(): ScaffoldController {
@Composable
fun ChatListView(chatModel: ChatModel) {
val scaffoldCtrl = scaffoldController()
if (chatModel.clearOverlays.value) {
scaffoldCtrl.collapse()
ModalManager.shared.closeModal()
chatModel.clearOverlays.value = false
}
BottomSheetScaffold(
scaffoldState = scaffoldCtrl.state,
drawerContent = { SettingsView(chatModel) },
@@ -43,7 +43,7 @@ class ModalManager {
}
fun closeModal() {
if (!modalViews.isEmpty()) {
if (modalViews.isNotEmpty()) {
modalViews.removeAt(modalViews.count() - 1)
}
modalCount.value = modalViews.count()
@@ -231,6 +231,8 @@
files = (
5C764E85279C748C000C6508 /* libz.tbd in Frameworks */,
5C764E84279C748C000C6508 /* libiconv.tbd in Frameworks */,
5C411599280048E90054D6CB /* libHSsimplex-chat-1.5.0-3uBn0HoMpg08OGLfasXsOx.a in Frameworks */,
5C41159B280048E90054D6CB /* libffi.a in Frameworks */,
);
runOnlyForDeploymentPostprocessing = 0;
};
+1 -1
View File
@@ -3,7 +3,7 @@ packages: .
source-repository-package
type: git
location: https://github.com/simplex-chat/simplexmq.git
tag: a638486bb3a9b212d15775d0bc63de2e619e0518
tag: d38303d5f1d55a90dd95d951ba8d798e17699269
source-repository-package
type: git
+224
View File
@@ -0,0 +1,224 @@
# SimpleX Chat terminal (console) app for Linux/MacOS/Windows
## Table of contents
- [Terminal chat features](#terminal-chat-features)
- [Installation](#🚀-installation)
- [Download chat client](#download-chat-client)
- [Linux and MacOS](#linux-and-macos)
- [Windows](#windows)
- [Build from source](#build-from-source)
- [Using Docker](#using-docker)
- [Using Haskell stack](#using-haskell-stack)
- [Usage](#usage)
- [Running the chat client](#running-the-chat-client)
- [How to use SimpleX chat](#how-to-use-simplex-chat)
- [Groups](#groups)
- [Sending files](#sending-files)
- [User contact addresses](#user-contact-addresses)
- [Access chat history](#access-chat-history)
## Terminal chat features
- 1-to-1 chat with multiple people in the same terminal window.
- Group messaging.
- Sending files to contacts and groups.
- User contact addresses - establish connections via multiple-use contact links.
- Messages persisted in a local SQLite database.
- Auto-populated recipient name - just type your messages to reply to the sender once the connection is established.
- Demo SMP servers available and pre-configured in the app - or you can [deploy your own server](https://github.com/simplex-chat/simplexmq#using-smp-server-and-smp-agent).
- No global identity or any names visible to the server(s), ensuring full privacy of your contacts and conversations.
- Two layers of E2E encryption (double-ratchet for duplex connections, using X3DH key agreement with ephemeral Curve448 keys, and NaCl crypto_box for SMP queues, using Curve25519 keys) and out-of-band passing of recipient keys (see [How to use SimpleX chat](#how-to-use-simplex-chat)).
- Message integrity validation (via including the digests of the previous messages).
- Authentication of each command/message by SMP servers with automatically generated Ed448 keys.
- TLS 1.3 transport encryption.
- Additional encryption of messages from SMP server to recipient to reduce traffic correlation.
Public keys involved in key exchange are not used as identity, they are randomly generated for each contact.
See [Encryption Primitives Used](https://github.com/simplex-chat/simplexmq/blob/master/protocol/overview-tjr.md#encryption-primitives-used) for technical details.
<a name="🚀-installation"></a>
## 🚀 Installation
### Download chat client
#### Linux and MacOS
To **install** or **update** `simplex-chat`, you should run the install script. To do that, use the following cURL or Wget command:
```sh
curl -o- https://raw.githubusercontent.com/simplex-chat/simplex-chat/stable/install.sh | bash
```
```sh
wget -qO- https://raw.githubusercontent.com/simplex-chat/simplex-chat/stable/install.sh | bash
```
Once the chat client downloads, you can run it with `simplex-chat` command in your terminal.
Alternatively, you can manually download the chat binary for your system from the [latest stable release](https://github.com/simplex-chat/simplex-chat/releases) and make it executable as shown below.
```sh
chmod +x <binary>
mv <binary> ~/.local/bin/simplex-chat
```
(or any other preferred location on `PATH`).
On MacOS you also need to [allow Gatekeeper to run it](https://support.apple.com/en-us/HT202491).
#### Windows
```sh
move <binary> %APPDATA%/local/bin/simplex-chat.exe
```
### Build from source
> **Please note:** to build the app use source code from [stable branch](https://github.com/simplex-chat/simplex-chat/tree/stable).
#### Using Docker
On Linux, you can build the chat executable using [docker build with custom output](https://docs.docker.com/engine/reference/commandline/build/#custom-build-outputs):
```shell
$ git clone git@github.com:simplex-chat/simplex-chat.git
$ cd simplex-chat
$ git checkout stable
$ DOCKER_BUILDKIT=1 docker build --output ~/.local/bin .
```
> **Please note:** If you encounter `` version `GLIBC_2.28' not found `` error, rebuild it with `haskell:8.10.4-stretch` base image (change it in your local [Dockerfile](Dockerfile)).
#### Using Haskell stack
Install [Haskell stack](https://docs.haskellstack.org/en/stable/README/):
```shell
curl -sSL https://get.haskellstack.org/ | sh
```
and build the project:
```shell
$ git clone git@github.com:simplex-chat/simplex-chat.git
$ cd simplex-chat
$ git checkout stable
$ stack install
```
## Usage
### Running the chat client
To start the chat client, run `simplex-chat` from the terminal.
By default, app data directory is created in the home directory (`~/.simplex`, or `%APPDATA%/simplex` on Windows), and two SQLite database files `simplex_v1_chat.db` and `simplex_v1_agent.db` are initialized in it.
To specify a different file path prefix for the database files use `-d` command line option:
```shell
$ simplex-chat -d alice
```
Running above, for example, would create `alice_v1_chat.db` and `alice_v1_agent.db` database files in current directory.
Three default SMP servers are hosted on Linode - they are [pre-configured in the app](https://github.com/simplex-chat/simplex-chat/blob/stable/src/Simplex/Chat/Options.hs#L42).
If you deployed your own SMP server(s) you can configure client via `-s` option:
```shell
$ simplex-chat -s smp://LcJUMfVhwD8yxjAiSaDzzGF3-kLG4Uh0Fl_ZIjrRwjI=@smp.example.com
```
Base64url encoded string preceding the server address is the server's offline certificate fingerprint which is validated by client during TLS handshake.
You can still talk to people using default or any other server - it only affects the location of the message queue when you initiate the connection (and the reply queue can be on another server, as set by the other party's client).
Run `simplex-chat -h` to see all available options.
### How to use SimpleX chat
Once you have started the chat, you will be prompted to specify your "display name" and an optional "full name" to create a local chat profile. Your display name is an alias for your contacts to refer to you by - it is not unique and does not serve as a global identity. If some of your contacts chose the same display name, the chat client adds a numeric suffix to their local display name.
The diagram below shows how to connect and message a contact:
<div align="center">
<img align="center" src="images/how-to-use-simplex.svg">
</div>
Once you've set up your local profile, enter `/c` (for `/connect`) to create a new connection and generate an invitation. Send this invitation to your contact via any other channel.
You are able to create multiple invitations by entering `/connect` multiple times and sending these invitations to the corresponding contacts you'd like to connect with.
The invitation can only be used once and even if this is intercepted, the attacker would not be able to use it to send you the messages via this queue once your contact confirms that the connection is established. See agent protocol for explanation of [invitation format](https://github.com/simplex-chat/simplexmq/blob/master/protocol/agent-protocol.md#connection-request).
The contact who received the invitation should enter `/c <invitation>` to accept the connection. This establishes the connection, and both parties are notified.
They would then use `@<name> <message>` commands to send messages. You may also just start typing a message to send it to the contact that was the last.
Use `/help` in chat to see the list of available commands.
### Groups
To create a group use `/g <group>`, then add contacts to it with `/a <group> <name>`. You can then send messages to the group by entering `#<group> <message>`. Use `/help groups` for other commands.
![simplex-chat](./images/groups.gif)
> **Please note**: the groups are not stored on any server, they are maintained as a list of members in the app database to whom the messages will be sent.
### Sending files
You can send a file to your contact with `/f @<contact> <file_path>` - the recipient will have to accept it before it is sent. Use `/help files` for other commands.
![simplex-chat](./images/files.gif)
You can send files to a group with `/f #<group> <file_path>`.
### User contact addresses
As an alternative to one-time invitation links, you can create a long-term address with `/ad` (for `/address`). The created address can then be shared via any channel, and used by other users as a link to make a contact request with `/c <user_contact_address>`.
You can accept or reject incoming requests with `/ac <name>` and `/rc <name>` commands.
User address is "long-term" in a sense that it is a multiple-use connection link - it can be used until it is deleted by the user, in which case all established connections would still remain active (unlike how it works with email, when changing the address results in people not being able to message you).
Use `/help address` for other commands.
![simplex-chat](./images/user-addresses.gif)
### Access chat history
SimpleX chat stores all your contacts and conversations in a local SQLite database, making it private and portable by design, owned and controlled by user.
You can view and search your chat history by querying your database. Run the below script to create message views in your database.
```sh
curl -o- https://raw.githubusercontent.com/simplex-chat/simplex-chat/stable/scripts/message_views.sql | sqlite3 ~/.simplex/simplex_v1_chat.db
```
Open SQLite Command Line Shell:
```sh
sqlite3 ~/.simplex/simplex_v1_chat.db
```
See [Message queries](./SQL.md) for examples.
> **Please note:** SQLite foreign key constraints are disabled by default, and must be **[enabled separately for each database connection](https://sqlite.org/foreignkeys.html#fk_enable)**. The latter can be achieved by running `PRAGMA foreign_keys = ON;` command on an open database connection. By running data altering queries without enabling foreign keys prior to that, you may risk putting your database in an inconsistent state.
**Convenience queries**
Get all messages from today (`chat_dt` is in UTC):
```sql
select * from all_messages_plain where date(chat_dt) > date('now', '-1 day') order by chat_dt;
```
Get overnight messages in the morning:
```sql
select * from all_messages_plain where chat_dt > datetime('now', '-15 hours') order by chat_dt;
```
+95
View File
@@ -0,0 +1,95 @@
# SimpleX platform - motivation and comparison
## Problems
Existing chat platforms and protocols have some or all of the following problems:
- Lack of privacy of the user profile and contacts (meta-data privacy).
- No protection (or only optional protection) of [E2EE][1] implementations from MITM attacks via provider.
- Unsolicited messages (spam and abuse).
- Lack of data ownership and protection.
- Complexity of usage for all non-centralized protocols to non-technical users.
The concentration of the communication in a small number of centralized platforms makes resolving these problems quite difficult.
## Proposed solution
Proposed stack of protocols solves these problems by making both messages and contacts stored only on client devices, reducing the role of the servers to simple message relays that only require authorization of messages sent to the queues, but do NOT require user authentication - not only the messages but also the metadata is protected because users do not have any identifiers assigned to them - unlike with any other platforms.
See [SimpleX whitepaper](https://github.com/simplex-chat/simplexmq/blob/master/protocol/overview-tjr.md) for more information on platform objectives and technical design.
## Why use SimpleX
## SimpleX unique approach to privacy and security
Everyone should care about privacy and security of their communications - even ordinary conversations can put you in danger.
### Full privacy of your identity, profile, contacts and metadata
**Unlike any other existing messaging platform, SimpleX has no identifiers assigned to the users** - it does not use phone numbers (like Signal or WhatsApp), domain-based addresses (like email, XMPP or Matrix), usernames (like Telegram), public keys or even random numbers (like all other messengers) to identify its users - we do not even know how many people use SimpleX.
To deliver the messages instead of user identifiers that all other platforms use, SimpleX uses the addresses of unidirectional (simplex) message queues. Using SimpleX is like having a different email address or a phone number for each contact you have, but without the hassle of managing all these addresses. In the near future SimpleX apps will also change the message queues automatically, moving the conversations from one server to another, to provide even better privacy to the users.
This approach protects the privacy of who are you communicating with, hiding it from SimpleX platform servers and from any observers. You can further improve your privacy by configuring your network access to connect to SimpleX servers via some overlay transport network, e.g. Tor.
### The best protection against spam and abuse
As you have no identifier on SimpleX platform, you cannot be contacted unless you share a one-time invitation link or an optional temporary user address. Even with the optional user addresses, while they can be used to send spam contact requests, you can change or completely delete it without losing any of your connections.
### Complete ownership, control and security of your data
SimpleX stores all user data on client devices, the messages are only held temporarily on SimpleX relay servers until they are received.
We use portable database format that can be used on all supported devices - we will soon add the ability to export the chat database from the mobile app so it can be used on another device.
Unlike servers of federated networks (email, XMPP or Matrix), SimpleX servers do not store user accounts, they simply relay messages to the recipients, protecting the privacy of both parties. There are no identifiers or encrypted messages in common between sent and received traffic of the server, thanks to the additional encryption layer for delivered messages. So if anybody is observing server traffic, they cannot easily determine who is communicating with whom (see [SimpleX whitepaper](https://github.com/simplex-chat/simplexmq/blob/master/protocol/overview-tjr.md) for the known traffic correlation attacks).
### Users own SimpleX network
You can use SimpleX with your own servers and still communicate with people using the servers that are pre-configured in the apps or any other SimpleX servers.
SimpleX platform uses an open protocol and provides SDK to create chat bots, allowing implementation of services that users can interact with via SimpleX Chat apps we are really looking forward to see what SimpleX services can be built.
If you are considering developing with the SimpleX platform, whether for chat bot services for SimpleX app users or to integrate the SimpleX Chat library into your mobile apps, please get in touch for any advice and support.
## Comparison with other protocols
| | SimpleX chat | Signal, big platforms | XMPP, Matrix | P2P protocols |
| :--------------------------------------------- | :----------------: | :-------------------: | :-------------: | :-------------: |
| Requires user identifiers | No = private | Yes<sup>1</sup> | Yes<sup>2</sup> | Yes<sup>3</sup> |
| Possibility of MITM | No = secure | Yes<sup>4</sup> | Yes | Yes |
| Dependence on DNS | No = resilient | Yes | Yes | No |
| Single operator or network | No = decentralized | Yes | No | Yes<sup>5</sup> |
| Central component or other network-wide attack | No = resilient | Yes | Yes<sup>2</sup> | Yes<sup>6</sup> |
1. Usually based on a phone number, in some cases on usernames.
2. DNS based.
3. Public key or some other globally unique ID.
4. If operators servers are compromised.
5. While P2P networks and cryptocurrency-based networks are distributed, they are not decentralized - they operate as a single network, with a single namespace of user addresses.
6. P2P networks either have a central authority or the whole network can be compromised - see the next section.
## Comparison with [P2P][9] messaging protocols
There are several P2P chat/messaging protocols and implementations that aim to solve privacy and centralisation problem, but they have their own set of problems that makes them less reliable than the proposed design, more complex to implement and analyse and more vulnerable to attacks.
1. [P2P][9] networks use some variant of [DHT][10] to route messages/requests through the network. DHT implementations have complex designs that have to balance reliability, delivery guarantee and latency. The proposed design has both better delivery guarantees and lower latency (the message is passed multiple times in parallel, through one node each time, using servers chosen by the recipient, while in P2P networks the message is passed through `O(log N)` nodes sequentially, using nodes chosen by the algorithm).
2. The proposed design, unlike most P2P networks, has no global user identifiers of any kind, even temporary.
3. P2P itself does not solve [MITM attack][2] problem, and most existing solutions do not use out-of-band messages for the initial key exchange. The proposed design uses out-of-band messages or, in some cases, pre-existing secure and trusted connections for the initial key exchange.
4. P2P implementations can be blocked by some Internet providers (like [BitTorrent][11]). The proposed design is transport agnostic - it can work over standard web protocols, and the servers can be deployed on the same domains as the websites.
5. All known P2P networks are likely to be vulnerable to [Sybil attack][12], because each node is discoverable, and the network operates as a whole. Known measures to reduce the probability of the Sybil attack either require a centralized component or expensive [proof of work][13]. The proposed design, on the opposite, has no server discoverability - servers are not connected, not known to each other and to all clients. The SimpleX network is fragmented and operates as multiple isolated connections. It makes network-wide attacks on SimpleX network impossible - even if some servers are compromised, other parts of the network can operate normally, and affected clients can switch to using other servers without losing contacts or messages.
6. P2P networks are likely to be vulnerable to [DRDoS attack][14]. In the proposed design clients only relay traffic from known trusted connection and cannot be used to reflect and amplify the traffic in the whole network.
[1]: https://en.wikipedia.org/wiki/End-to-end_encryption
[2]: https://en.wikipedia.org/wiki/Man-in-the-middle_attack
[9]: https://en.wikipedia.org/wiki/Peer-to-peer
[10]: https://en.wikipedia.org/wiki/Distributed_hash_table
[11]: https://en.wikipedia.org/wiki/BitTorrent
[12]: https://en.wikipedia.org/wiki/Sybil_attack
[13]: https://en.wikipedia.org/wiki/Proof_of_work
[14]: https://www.usenix.org/conference/woot15/workshop-program/presentation/p2p-file-sharing-hell-exploiting-bittorrent
+2 -2
View File
@@ -1,4 +1,4 @@
# Message queries
# Accessing message history via SQL queries
You can run queries against `direct_messages`, `group_messages` and `all_messages` (or their simpler alternatives `direct_messages_plain`, `group_messages_plain` and `all_messages_plain`), for example:
@@ -21,7 +21,7 @@ select * from all_messages_plain;
-- files you offered for sending
select * from direct_messages where msg_sent = 1 and chat_msg_event = 'x.file';
-- everything catherine sent related to cats
select * from direct_messages where msg_sent = 0 and contact = 'catherine' and msg_body like '%cats%';
select * from direct_messages where msg_sent = 0 and contact = 'catherine' and msg_body like '%cats%';
-- all correspondence with alice in #team
select * from group_messages where group_name = 'team' and contact = 'alice';
+4 -4
View File
@@ -18,10 +18,10 @@
name = "simplex-chat";
src = ./.;
};
sha256map = import ./sha256map.nix;
sha256map = import ./scripts/nix/sha256map.nix;
modules = [{
packages.direct-sqlite.patches = [ ./direct-sqlite-2.3.26.patch ];
packages.entropy.patches = [ ./entropy.patch ];
packages.direct-sqlite.patches = [ ./scripts/nix/direct-sqlite-2.3.26.patch ];
packages.entropy.patches = [ ./scripts/nix/entropy.patch ];
}
({ pkgs,lib, ... }: lib.mkIf (pkgs.stdenv.hostPlatform.isAndroid) {
packages.simplex-chat.components.library.ghcOptions = [ "-pie" ];
@@ -329,7 +329,7 @@
name = "update-sha256map";
runtimeInputs = [ pkgs.nix-prefetch-git pkgs.jq pkgs.gawk ];
text = ''
gawk -f update-sha256.awk cabal.project > sha256map.nix
gawk -f ./scripts/nix/update-sha256.awk cabal.project > ./scripts/nix/sha256map.nix
'';
}; in
pkgs.mkShell {
@@ -3,8 +3,8 @@
# the binaries folders should be in ~/Downloads folder
rm -rf ./apps/ios/Libraries/mac-aarch64 ./apps/ios/Libraries/mac-x86_64 ./apps/ios/Libraries/ios ./apps/ios/Libraries/sim
mkdir -p ./apps/ios/Libraries/mac-aarch64 ./apps/ios/Libraries/mac-x86_64 ./apps/ios/Libraries/ios ./apps/ios/Libraries/sim
cp ~/Downloads/pkg-ios-aarch64/* ./apps/ios/Libraries/mac-aarch64
cp ~/Downloads/pkg-ios-x86_64/* ./apps/ios/Libraries/mac-x86_64
cp ~/Downloads/pkg-ios-aarch64-swift-json/* ./apps/ios/Libraries/mac-aarch64
cp ~/Downloads/pkg-ios-x86_64-swift-json/* ./apps/ios/Libraries/mac-x86_64
chmod +w ./apps/ios/Libraries/mac-aarch64/*
chmod +w ./apps/ios/Libraries/mac-x86_64/*
cp ./apps/ios/Libraries/mac-aarch64/* ./apps/ios/Libraries/ios
@@ -3,7 +3,7 @@
# the binaries folder should be in ~/Downloads folder
rm -rf ./apps/ios/Libraries/mac ./apps/ios/Libraries/ios ./apps/ios/Libraries/sim
mkdir -p ./apps/ios/Libraries/mac ./apps/ios/Libraries/ios ./apps/ios/Libraries/sim
cp ~/Downloads/pkg-ios-aarch64/* ./apps/ios/Libraries/mac
cp ~/Downloads/pkg-ios-aarch64-swift-json/* ./apps/ios/Libraries/mac
chmod +w ./apps/ios/Libraries/mac/*
cp ./apps/ios/Libraries/mac/* ./apps/ios/Libraries/ios
cp ./apps/ios/Libraries/mac/* ./apps/ios/Libraries/sim
+1 -1
View File
@@ -1,5 +1,5 @@
{
"https://github.com/simplex-chat/simplexmq.git"."a638486bb3a9b212d15775d0bc63de2e619e0518" = "14m6ljmkfvcqnv53lp4ygaqnn2vydhws1hc3ah8zh5l5g4hkif36";
"https://github.com/simplex-chat/simplexmq.git"."d38303d5f1d55a90dd95d951ba8d798e17699269" = "0gx9wlk0v8ml6aid9hzm1nqy357plk8sbnhfcnchna12dgm8v0sd";
"https://github.com/simplex-chat/aeson.git"."3eb66f9a68f103b5f1489382aad89f5712a64db7" = "0kilkx59fl6c3qy3kjczqvm8c3f4n3p0bdk9biyflf51ljnzp4yp";
"https://github.com/simplex-chat/haskell-terminal.git"."f708b00009b54890172068f168bf98508ffcd495" = "0zmq7lmfsk8m340g47g5963yba7i88n4afa6z93sg9px5jv1mijj";
"https://github.com/zw3rk/android-support.git"."3c3a5ab0b8b137a072c98d3d0937cbdc96918ddb" = "1r6jyxbim3dsvrmakqfyxbd6ms6miaghpbwyl0sr6dzwpgaprz97";
-84
View File
@@ -1,84 +0,0 @@
# Federated chat system with [E2EE][1] and low risk of [MITM attack][2]
## Problems
Existing chat platforms and protocols have some or all of the following problems:
- Lack of privacy of the user profile and connections (meta-data privacy).
- No protection (or only optional protection) of [E2EE][1] implementations from MITM attacks.
- Unsolicited messages (spam and abuse).
- Lack of data ownership and protection.
- Complexity of usage for all non-centralized protocols to non-technical users.
The concentration of the communication in a small number of centralized platforms makes resolving these problems quite difficult.
## Proposed solution
Proposed stack of protocols solves these and other problems by making both messages and contacts accessible only on client devices, reducing the role of the servers to simple message brokers that only require authorization of messages sent to the queues, but do NOT require user authentication - not only the messages but also the metadata is protected.
See [SMP protocol][6] and [SMP agent protocol][8].
## Comparison with other protocols
| | SimpleX chat | Signal, big platforms | XMPP, Matrix | P2P protocols |
|:-------- |:------------:|:---------------------:|:------------:|:-------------:|
| Requires global identity | No = private | Yes<sup>1</sup> | Yes<sup>2</sup> | Yes<sup>3</sup> |
| Possibility of MITM | No = secure | Yes<sup>4</sup> | Yes | Yes |
| Dependence on DNS | No = resilient | Yes | Yes | No |
| Federation | Yes | No | Yes | No<sup>5</sup> |
| Central component or other network-wide attack | No = resilient | Yes | Yes<sup>2</sup> | Yes<sup>6</sup> |
1. Usually based on a phone number, in some cases on usernames.
2. DNS based.
3. Public key or some other globally unique ID.
4. If operators servers are compromised.
5. While P2P networks are distributed, they are not federated - they operate as a single network.
6. P2P networks either have a central authority or the whole network can be compromised - see the next section.
## Comparison with [P2P][9] messaging protocols
There are several P2P chat/messaging protocols and implementations that aim to solve privacy and centralisation problem, but they have their own set of problems that makes them less reliable than the proposed chat system design, more complex to implement and analyse and more vulnerable to attacks.
1. [P2P][9] networks either have some centralized component, which makes them highly vulnerable, or, more commonly, use some variant of [DHT][10] to route messages/requests through the network. DHT implementations have complex designs that have to balance reliability, delivery guarantee and latency, and also have some other problems. The proposed chat system design has both higher delivery guarantee and low latency (the message is passed multiple times in parallel, through one node each time, using servers chosen by the recipient, while in P2P networks the message is passed through `O(log N)` nodes sequentially, using nodes chosen by the algorithm).
2. The proposed design, unlike most P2P networks, has no global identity of any form, even temporary.
3. P2P itself does not solve [MITM attack][2] problem, but most existing solutions do not use out-of-band messages for the initial key exchange. The proposed design uses out-of-band messages or, in some cases, pre-existing secure and trusted connections for the initial key exchange.
4. P2P implementations can be blocked by some Internet providers (like [BitTorrent][11]). The proposed design is transport agnostic - it can work over standard web protocols, and the servers can be deployed on the same domains as the websites.
5. All known P2P networks are likely to be vulnerable to [Sybil attack][12], because each node is discoverable, and the network operates as a whole. Known measures to reduce the probability of the Sybil attack either require a vulnerable centralized component or expensive [proof of work][13]. The proposed design, on the opposite, has no server discoverability - servers are not connected, not known to each other and to all clients. The chat network is fragmented and operates as multiple isolated connections. It makes Sybil attack on the whole simplex messaging network impossible - even if some servers are compromised, other parts of the network can operate normally, and affected clients can always switch to using other servers without losing contacts or messages.
6. P2P networks are likely to be vulnerable to [DRDoS attack][14]. In the proposed design clients only relay traffic from known trusted connection and cannot be used to reflect and amplify the traffic in the whole network.
## Network features
- No user identity known to system servers - no phone numbers, user names and no DNS are needed to authorize users to the network.
- Each user can be connected to multiple servers to ensure message delivery, even if some of the servers are compromised.
- No single server in the system has visibility of all connections or messages of any user, as user profiles are identified by multiple rotating public keys, using separate key for each profile connection.
- Uses standard asymmetric cryptographic protocols, so that system users can create independent server and client implementations complying with the protocols.
- Open-source server implementations that can be easily deployed by any user with minimal technical expertise (e.g. on Heroku via web UI).
- Open-source client implementations so that system users can independently assess system security model.
- Only client applications store user profiles, contacts of other user profiles, messages; servers do NOT have access to any of this information and (unless compromised) do NOT store encrypted messages or any logs.
- Multiple client applications and devices can be used by each user profile to communicate and to share connections and message history - the devices are not known to the servers.
- Initial key exchange and establishing connections between user profiles is done by sharing the invitation (e.g. QR code via any independent communication channel (or directly via screen and camera), system servers are NOT used for key exchange - to reduce risk of key substitution in [MITM attack][2]. QR code contains the connection-specific public key and other information needed to establish the connection.
- Connections between users can be established via shared trusted connections to simplify key exchange.
- Servers do NOT communicate with each other, they only communicate with client applications.
- Unique public key is used for each user profile connection in order to:
- reduce the risk of attacker posing as user's connection;
- avoid exposing all user connections to the servers.
- Unique public key is used to identify each connection participant to each server.
- Public keys used between connections are regularly rotated to prevent decryption of the full message history ([forward secrecy][4]) in case when some servers or middlemen preserve message history and the current key is compromised.
- Users can repeat key exchange using QR code and alternative channel at any point to increase communication security and trust.
[1]: https://en.wikipedia.org/wiki/End-to-end_encryption
[2]: https://en.wikipedia.org/wiki/Man-in-the-middle_attack
[4]: https://en.wikipedia.org/wiki/Forward_secrecy
[6]: https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md
[8]: https://github.com/simplex-chat/simplexmq/blob/master/protocol/agent-protocol.md
[9]: https://en.wikipedia.org/wiki/Peer-to-peer
[10]: https://en.wikipedia.org/wiki/Distributed_hash_table
[11]: https://en.wikipedia.org/wiki/BitTorrent
[12]: https://en.wikipedia.org/wiki/Sybil_attack
[13]: https://en.wikipedia.org/wiki/Proof_of_work
[14]: https://www.usenix.org/conference/woot15/workshop-program/presentation/p2p-file-sharing-hell-exploiting-bittorrent
+103 -38
View File
@@ -64,7 +64,7 @@ import System.IO (Handle, IOMode (..), SeekMode (..), hFlush, openFile, stdout)
import Text.Read (readMaybe)
import UnliftIO.Async
import UnliftIO.Concurrent (forkIO, threadDelay)
import UnliftIO.Directory (doesDirectoryExist, doesFileExist, getFileSize, getHomeDirectory, getTemporaryDirectory)
import UnliftIO.Directory
import qualified UnliftIO.Exception as E
import UnliftIO.IO (hClose, hSeek, hTell)
import UnliftIO.STM
@@ -118,7 +118,8 @@ newChatController chatStore user cfg@ChatConfig {agentConfig = aCfg, tbqSize} Ch
chatLock <- newTMVarIO ()
sndFiles <- newTVarIO M.empty
rcvFiles <- newTVarIO M.empty
pure ChatController {activeTo, firstTime, currentUser, smpAgent, agentAsync, chatStore, idsDrg, inputQ, outputQ, notifyQ, chatLock, sndFiles, rcvFiles, config, sendNotification}
filesFolder <- newTVarIO Nothing
pure ChatController {activeTo, firstTime, currentUser, smpAgent, agentAsync, chatStore, idsDrg, inputQ, outputQ, notifyQ, chatLock, sndFiles, rcvFiles, config, sendNotification, filesFolder}
where
resolveServers :: IO (NonEmpty SMPServer)
resolveServers = case user of
@@ -172,6 +173,11 @@ processChatCommand = \case
asks agentAsync >>= readTVarIO >>= \case
Just _ -> pure CRChatRunning
_ -> startChatController user $> CRChatStarted
SetFilesFolder filesFolder' -> withUser $ \_ -> do
createDirectoryIfMissing True filesFolder'
ff <- asks filesFolder
atomically . writeTVar ff $ Just filesFolder'
pure CRCmdOk
APIGetChats -> CRApiChats <$> withUser (\user -> withStore (`getChatPreviews` user))
APIGetChat cType cId pagination -> withUser $ \user -> case cType of
CTDirect -> CRApiChat . AChat SCTDirect <$> withStore (\st -> getDirectChat st user cId pagination)
@@ -291,13 +297,15 @@ processChatCommand = \case
CTContactRequest -> pure $ chatCmdError "not supported"
APIDeleteChatItem cType chatId itemId mode -> withUser $ \user@User {userId} -> withChatLock $ case cType of
CTDirect -> do
(ct@Contact {localDisplayName = c}, CChatItem msgDir deletedItem@ChatItem {meta = CIMeta {itemSharedMsgId}}) <- withStore $ \st -> (,) <$> getContact st userId chatId <*> getDirectChatItem st userId chatId itemId
(ct@Contact {localDisplayName = c}, CChatItem msgDir deletedItem@ChatItem {meta = CIMeta {itemSharedMsgId}, file}) <- withStore $ \st -> (,) <$> getContact st userId chatId <*> getDirectChatItem st userId chatId itemId
case (mode, msgDir, itemSharedMsgId) of
(CIDMInternal, _, _) -> do
deleteFile userId file
toCi <- withStore $ \st -> deleteDirectChatItemInternal st userId ct itemId
pure $ CRChatItemDeleted (AChatItem SCTDirect msgDir (DirectChat ct) deletedItem) toCi
(CIDMBroadcast, SMDSnd, Just itemSharedMId) -> do
SndMessage {msgId} <- sendDirectContactMessage ct (XMsgDel itemSharedMId)
deleteFile userId file
toCi <- withStore $ \st -> deleteDirectChatItemSndBroadcast st userId ct itemId msgId
setActive $ ActiveC c
pure $ CRChatItemDeleted (AChatItem SCTDirect msgDir (DirectChat ct) deletedItem) toCi
@@ -305,18 +313,27 @@ processChatCommand = \case
CTGroup -> do
Group gInfo@GroupInfo {localDisplayName = gName, membership} ms <- withStore $ \st -> getGroup st user chatId
unless (memberActive membership) $ throwChatError CEGroupMemberUserRemoved
CChatItem msgDir deletedItem@ChatItem {meta = CIMeta {itemSharedMsgId}} <- withStore $ \st -> getGroupChatItem st user chatId itemId
CChatItem msgDir deletedItem@ChatItem {meta = CIMeta {itemSharedMsgId}, file} <- withStore $ \st -> getGroupChatItem st user chatId itemId
case (mode, msgDir, itemSharedMsgId) of
(CIDMInternal, _, _) -> do
deleteFile userId file
toCi <- withStore $ \st -> deleteGroupChatItemInternal st user gInfo itemId
pure $ CRChatItemDeleted (AChatItem SCTGroup msgDir (GroupChat gInfo) deletedItem) toCi
(CIDMBroadcast, SMDSnd, Just itemSharedMId) -> do
SndMessage {msgId} <- sendGroupMessage gInfo ms (XMsgDel itemSharedMId)
deleteFile userId file
toCi <- withStore $ \st -> deleteGroupChatItemSndBroadcast st user gInfo itemId msgId
setActive $ ActiveG gName
pure $ CRChatItemDeleted (AChatItem SCTGroup msgDir (GroupChat gInfo) deletedItem) toCi
(CIDMBroadcast, _, _) -> throwChatError CEInvalidChatItemDelete
CTContactRequest -> pure $ chatCmdError "not supported"
where
deleteFile :: MsgDirectionI d => UserId -> Maybe (CIFile d) -> m ()
deleteFile userId file =
forM_ file $ \CIFile {fileId, filePath, fileStatus} -> do
cancelFiles userId [(fileId, AFS msgDirection fileStatus)]
withFilesFolder $ \filesFolder ->
deleteFiles filesFolder [filePath]
APIChatRead cType chatId fromToIds -> withChatLock $ case cType of
CTDirect -> withStore (\st -> updateDirectChatItemsRead st chatId fromToIds) $> CRCmdOk
CTGroup -> withStore (\st -> updateGroupChatItemsRead st chatId fromToIds) $> CRCmdOk
@@ -326,8 +343,12 @@ processChatCommand = \case
ct@Contact {localDisplayName} <- withStore $ \st -> getContact st userId chatId
withStore (\st -> getContactGroupNames st userId ct) >>= \case
[] -> do
files <- withStore $ \st -> getContactFiles st userId ct
conns <- withStore $ \st -> getContactConnections st userId ct
withChatLock . procCmd $ do
cancelFiles userId (map (\(fId, fStatus, _) -> (fId, fStatus)) files)
withFilesFolder $ \filesFolder -> do
deleteFiles filesFolder (map (\(_, _, fPath) -> fPath) files)
withAgent $ \a -> forM_ conns $ \conn ->
deleteConnection a (aConnId conn) `catchError` \(_ :: AgentErrorType) -> pure ()
withStore $ \st -> deleteContact st userId ct
@@ -578,24 +599,8 @@ processChatCommand = \case
ChatErrorAgent (CONN DUPLICATE) -> pure $ CRRcvFileAcceptedSndCancelled ft
e -> throwError e
CancelFile fileId -> withUser $ \User {userId} -> do
ft' <- withStore (\st -> getFileTransfer st userId fileId)
withChatLock . procCmd $ do
case ft' of
FTSnd ftm fts -> do
cancelFileTransfer userId ft' CIFSSndCancelled
forM_ fts $ \ft -> cancelSndFileTransfer ft
pure $ CRSndGroupFileCancelled ftm fts
FTRcv ft -> do
cancelFileTransfer userId ft' CIFSRcvCancelled
cancelRcvFileTransfer ft
pure $ CRRcvFileCancelled ft
where
cancelFileTransfer :: MsgDirectionI d => UserId -> FileTransfer -> CIFileStatus d -> m ()
cancelFileTransfer userId ft ciFileStatus =
unless (fileTransferCancelled ft) $
withStore $ \st -> do
updateFileCancelled st userId fileId
updateCIFileStatus st userId fileId ciFileStatus
ft <- withStore (\st -> getFileTransfer st userId fileId)
withChatLock . procCmd $ cancelFile userId fileId ft
FileStatus fileId ->
CRFileTransferStatus <$> withUser (\User {userId} -> withStore $ \st -> getFileTransferProgress st userId fileId)
ShowProfile -> withUser $ \User {profile} -> pure $ CRUserProfile profile
@@ -641,8 +646,9 @@ processChatCommand = \case
cId == Just contactId && s /= GSMemRemoved && s /= GSMemLeft
checkSndFile :: FilePath -> m (Integer, Integer)
checkSndFile f = do
unlessM (doesFileExist f) . throwChatError $ CEFileNotFound f
(,) <$> getFileSize f <*> asks (fileChunkSize . config)
fsFilePath <- toFSFilePath f
unlessM (doesFileExist fsFilePath) . throwChatError $ CEFileNotFound f
(,) <$> getFileSize fsFilePath <*> asks (fileChunkSize . config)
updateProfile :: User -> Profile -> m ChatResponse
updateProfile user@User {profile = p} p'@Profile {displayName}
| p' == p = pure CRUserProfileNoChange
@@ -659,6 +665,53 @@ processChatCommand = \case
isReady ct =
let s = connStatus $ activeConn (ct :: Contact)
in s == ConnReady || s == ConnSndReady
-- perform an action only if filesFolder is set (i.e. on mobile devices)
withFilesFolder :: (FilePath -> m ()) -> m ()
withFilesFolder action = asks filesFolder >>= readTVarIO >>= mapM_ action
deleteFiles :: FilePath -> [Maybe FilePath] -> m ()
deleteFiles filesFolder filePaths =
forM_ filePaths $ \filePath_ ->
forM_ filePath_ $ \filePath -> do
let fsFilePath = filesFolder <> "/" <> filePath
removeFile fsFilePath `E.catch` \(_ :: E.SomeException) ->
removePathForcibly fsFilePath `E.catch` \(_ :: E.SomeException) -> pure ()
cancelFiles :: UserId -> [(Int64, ACIFileStatus)] -> m ()
cancelFiles userId files =
forM_ files $ \(fileId, status) -> do
case status of
AFS _ CIFSSndStored -> cancelById fileId
AFS _ CIFSRcvInvitation -> cancelById fileId
AFS _ CIFSRcvTransfer -> cancelById fileId
_ -> pure ()
where
cancelById fileId = do
ft <- withStore (\st -> getFileTransfer st userId fileId)
void $ cancelFile userId fileId ft
cancelFile :: UserId -> Int64 -> FileTransfer -> m ChatResponse
cancelFile userId fileId ft =
case ft of
FTSnd ftm fts -> do
cancelFileTransfer CIFSSndCancelled
forM_ fts $ \ft' -> cancelSndFileTransfer ft'
pure $ CRSndGroupFileCancelled ftm fts
FTRcv ftr -> do
cancelFileTransfer CIFSRcvCancelled
cancelRcvFileTransfer ftr
pure $ CRRcvFileCancelled ftr
where
cancelFileTransfer :: MsgDirectionI d => CIFileStatus d -> m ()
cancelFileTransfer ciFileStatus =
unless (fileTransferCancelled ft) $
withStore $ \st -> do
updateFileCancelled st userId fileId
updateCIFileStatus st userId fileId ciFileStatus
-- mobile clients use file paths relative to app directory (e.g. for the reason ios app directory changes on updates),
-- so we have to differentiate between the file path stored in db and communicated with frontend, and the file path
-- used during file transfer for actual operations with file system
toFSFilePath :: ChatMonad m => FilePath -> m FilePath
toFSFilePath f =
maybe f (<> "/" <> f) <$> (readTVarIO =<< asks filesFolder)
acceptFileReceive :: forall m. ChatMonad m => User -> RcvFileTransfer -> Maybe FilePath -> m FilePath
acceptFileReceive user@User {userId} RcvFileTransfer {fileId, fileInvitation = FileInvitation {fileName = fName, fileConnReq}, fileStatus, senderDisplayName, grpMemberId} filePath_ = do
@@ -696,11 +749,17 @@ acceptFileReceive user@User {userId} RcvFileTransfer {fileId, fileInvitation = F
where
getRcvFilePath :: Maybe FilePath -> String -> m FilePath
getRcvFilePath fPath_ fn = case fPath_ of
Nothing -> do
dir <- (`combine` "Downloads") <$> getHomeDirectory
ifM (doesDirectoryExist dir) (pure dir) getTemporaryDirectory
>>= (`uniqueCombine` fn)
>>= createEmptyFile
Nothing ->
asks filesFolder >>= readTVarIO >>= \case
Nothing -> do
dir <- (`combine` "Downloads") <$> getHomeDirectory
ifM (doesDirectoryExist dir) (pure dir) getTemporaryDirectory
>>= (`uniqueCombine` fn)
>>= createEmptyFile
Just filesFolder ->
filesFolder `uniqueCombine` fn
>>= createEmptyFile
>>= pure <$> takeFileName
Just fPath ->
ifM
(doesDirectoryExist fPath)
@@ -1109,11 +1168,12 @@ processAgentMessage (Just user@User {userId, profile}) agentConnId agentMessage
then badRcvFileChunk ft "incorrect chunk size"
else do
appendFileChunk ft chunkNo chunk
withStore $ \st -> do
ci <- withStore $ \st -> do
updateRcvFileStatus st ft FSComplete
updateCIFileStatus st userId fileId CIFSRcvComplete
deleteRcvFileChunks st ft
toView $ CRRcvFileComplete ft
getChatItemByFileId st user fileId
toView $ CRRcvFileComplete ci
closeFileHandle fileId rcvFiles
withAgent (`deleteConnection` agentConnId)
RcvChunkDuplicate -> pure ()
@@ -1239,6 +1299,7 @@ processAgentMessage (Just user@User {userId, profile}) agentConnId agentMessage
CChatItem msgDir deletedItem@ChatItem {meta = CIMeta {itemId}} <- withStore $ \st -> getDirectChatItemBySharedMsgId st userId contactId sharedMsgId
case msgDir of
SMDRcv -> do
-- TODO allow to locally delete items that were broadcast deleted by sender
toCi <- withStore $ \st -> deleteDirectChatItemRcvBroadcast st userId ct itemId msgId
toView $ CRChatItemDeleted (AChatItem SCTDirect SMDRcv (DirectChat ct) deletedItem) toCi
checkIntegrity msgMeta $ toView . CRMsgIntegrityError
@@ -1516,11 +1577,12 @@ sendFileChunkNo ft@SndFileTransfer {agentConnId = AgentConnId acId} chunkNo = do
withStore $ \st -> updateSndFileChunkMsg st ft chunkNo msgId
readFileChunk :: ChatMonad m => SndFileTransfer -> Integer -> m ByteString
readFileChunk SndFileTransfer {fileId, filePath, chunkSize} chunkNo =
read_ `E.catch` (throwChatError . CEFileRead filePath . (show :: E.SomeException -> String))
readFileChunk SndFileTransfer {fileId, filePath, chunkSize} chunkNo = do
fsFilePath <- toFSFilePath filePath
read_ fsFilePath `E.catch` (throwChatError . CEFileRead filePath . (show :: E.SomeException -> String))
where
read_ = do
h <- getFileHandle fileId filePath sndFiles ReadMode
read_ fsFilePath = do
h <- getFileHandle fileId fsFilePath sndFiles ReadMode
pos <- hTell h
let pos' = (chunkNo - 1) * chunkSize
when (pos /= pos') $ hSeek h AbsoluteSeek pos'
@@ -1548,12 +1610,14 @@ parseFileChunk msg =
appendFileChunk :: ChatMonad m => RcvFileTransfer -> Integer -> ByteString -> m ()
appendFileChunk ft@RcvFileTransfer {fileId, fileStatus} chunkNo chunk =
case fileStatus of
RFSConnected RcvFileInfo {filePath} -> append_ filePath
RFSConnected RcvFileInfo {filePath} -> do
fsFilePath <- toFSFilePath filePath
append_ filePath fsFilePath
RFSCancelled _ -> pure ()
_ -> throwChatError $ CEFileInternal "receiving file transfer not in progress"
where
append_ fPath = do
h <- getFileHandle fileId fPath rcvFiles AppendMode
append_ fPath fPathUsed = do
h <- getFileHandle fileId fPathUsed rcvFiles AppendMode
E.try (liftIO $ B.hPut h chunk >> hFlush h) >>= \case
Left (e :: E.SomeException) -> throwChatError . CEFileWrite fPath $ show e
Right () -> withStore $ \st -> updatedRcvFileChunkStored st ft chunkNo
@@ -1815,6 +1879,7 @@ chatCommandP =
("/user " <|> "/u ") *> (CreateActiveUser <$> userProfile)
<|> ("/user" <|> "/u") $> ShowActiveUser
<|> "/_start" $> StartChat
<|> "/_files_folder " *> (SetFilesFolder <$> filePath)
<|> "/_get chats" $> APIGetChats
<|> "/_get chat " *> (APIGetChat <$> chatTypeP <*> A.decimal <* A.space <*> chatPaginationP)
<|> "/_get items count=" *> (APIGetChatItems <$> A.decimal)
+4 -2
View File
@@ -77,7 +77,8 @@ data ChatController = ChatController
chatLock :: TMVar (),
sndFiles :: TVar (Map Int64 Handle),
rcvFiles :: TVar (Map Int64 Handle),
config :: ChatConfig
config :: ChatConfig,
filesFolder :: TVar (Maybe FilePath) -- path to files folder for mobile apps
}
data HelpSection = HSMain | HSFiles | HSGroups | HSMyAddress | HSMarkdown | HSMessages
@@ -91,6 +92,7 @@ data ChatCommand
= ShowActiveUser
| CreateActiveUser Profile
| StartChat
| SetFilesFolder FilePath
| APIGetChats
| APIGetChat ChatType Int64 ChatPagination
| APIGetChatItems Int
@@ -200,7 +202,7 @@ data ChatResponse
| CRRcvFileAccepted {fileTransfer :: RcvFileTransfer, filePath :: FilePath}
| CRRcvFileAcceptedSndCancelled {rcvFileTransfer :: RcvFileTransfer}
| CRRcvFileStart {rcvFileTransfer :: RcvFileTransfer}
| CRRcvFileComplete {rcvFileTransfer :: RcvFileTransfer}
| CRRcvFileComplete {chatItem :: AChatItem}
| CRRcvFileCancelled {rcvFileTransfer :: RcvFileTransfer}
| CRRcvFileSndCancelled {rcvFileTransfer :: RcvFileTransfer}
| CRSndFileStart {sndFileTransfer :: SndFileTransfer}
+1 -1
View File
@@ -48,7 +48,7 @@ cChatRecvMsg cc = deRefStablePtr cc >>= chatRecvMsg >>= newCAString
mobileChatOpts :: ChatOpts
mobileChatOpts =
ChatOpts
{ dbFilePrefix = "simplex_v1", -- two database files will be created: simplex_v1_chat.db and simplex_v1_agent.db
{ dbFilePrefix = undefined,
smpServers = [],
logConnections = False,
logAgent = False,
+45
View File
@@ -115,6 +115,7 @@ module Simplex.Chat.Store
updateFileTransferChatItemId,
getFileTransfer,
getFileTransferProgress,
getContactFiles,
createNewSndMessage,
createSndMsgDelivery,
createNewMessageAndRcvMsgDelivery,
@@ -135,6 +136,7 @@ module Simplex.Chat.Store
getGroupChatItemBySharedMsgId,
getDirectChatItemIdByText,
getGroupChatItemIdByText,
getChatItemByFileId,
updateDirectChatItemStatus,
updateDirectChatItem,
deleteDirectChatItemInternal,
@@ -2214,6 +2216,19 @@ getFileTransferMeta_ db userId fileId =
fileTransferMeta (fileName, fileSize, chunkSize, filePath, cancelled_) =
FileTransferMeta {fileId, fileName, filePath, fileSize, chunkSize, cancelled = fromMaybe False cancelled_}
getContactFiles :: MonadUnliftIO m => SQLiteStore -> UserId -> Contact -> m [(Int64, ACIFileStatus, Maybe FilePath)]
getContactFiles st userId Contact {contactId} =
liftIO . withTransaction st $ \db ->
DB.query
db
[sql|
SELECT f.file_id, f.ci_file_status, f.file_path
FROM chat_items i
JOIN files f ON f.chat_item_id = i.chat_item_id
WHERE i.user_id = ? AND i.contact_id = ?
|]
(userId, contactId)
createNewSndMessage :: StoreMonad m => SQLiteStore -> TVar ChaChaDRG -> ConnOrGroupId -> (SharedMsgId -> NewMessage) -> m SndMessage
createNewSndMessage st gVar connOrGroupId mkMessage =
liftIOEither . withTransaction st $ \db ->
@@ -3367,6 +3382,35 @@ getGroupChatItemIdByText st User {userId, localDisplayName = userName} groupId c
|]
(userId, groupId, cName, quotedMsg <> "%")
getChatItemByFileId :: StoreMonad m => SQLiteStore -> User -> Int64 -> m AChatItem
getChatItemByFileId st user@User {userId} fileId = do
liftIOEither . withTransaction st $ \db -> runExceptT $ do
r <- ExceptT $ getChatItemIdByFileId_ db userId fileId
case r of
(itemId, Just contactId, Nothing) -> do
ct <- ExceptT $ getContact_ db userId contactId
(CChatItem msgDir ci) <- ExceptT $ getDirectChatItem_ db userId contactId itemId
pure $ AChatItem SCTDirect msgDir (DirectChat ct) ci
(itemId, Nothing, Just groupId) -> do
gInfo <- ExceptT $ getGroupInfo_ db user groupId
(CChatItem msgDir ci) <- ExceptT $ getGroupChatItem_ db user groupId itemId
pure $ AChatItem SCTGroup msgDir (GroupChat gInfo) ci
_ -> throwError $ SEChatItemNotFoundByFileId fileId
getChatItemIdByFileId_ :: DB.Connection -> UserId -> Int64 -> IO (Either StoreError (ChatItemId, Maybe Int64, Maybe Int64))
getChatItemIdByFileId_ db userId fileId =
firstRow id (SEChatItemNotFoundByFileId fileId) $
DB.query
db
[sql|
SELECT i.chat_item_id, i.contact_id, i.group_id
FROM chat_items i
JOIN files f ON f.chat_item_id = i.chat_item_id
WHERE f.user_id = ? AND f.file_id = ?
LIMIT 1
|]
(userId, fileId)
updateDirectChatItemsRead :: (StoreMonad m) => SQLiteStore -> Int64 -> (ChatItemId, ChatItemId) -> m ()
updateDirectChatItemsRead st contactId (fromItemId, toItemId) = do
currentTs <- liftIO getCurrentTime
@@ -3608,6 +3652,7 @@ data StoreError
| SEChatItemNotFound {itemId :: ChatItemId}
| SEQuotedChatItemNotFound
| SEChatItemSharedMsgIdNotFound {sharedMsgId :: SharedMsgId}
| SEChatItemNotFoundByFileId {fileId :: FileTransferId}
deriving (Show, Exception, Generic)
instance ToJSON StoreError where
+8 -1
View File
@@ -100,7 +100,7 @@ responseToView testView = \case
CRContactsMerged intoCt mergedCt -> viewContactsMerged intoCt mergedCt
CRReceivedContactRequest UserContactRequest {localDisplayName = c, profile} -> viewReceivedContactRequest c profile
CRRcvFileStart ft -> receivingFile_ "started" ft
CRRcvFileComplete ft -> receivingFile_ "completed" ft
CRRcvFileComplete ci -> receivingFile_' "completed" ci
CRRcvFileSndCancelled ft -> viewRcvFileSndCancelled ft
CRSndFileStart ft -> sendingFile_ "started" ft
CRSndFileComplete ft -> sendingFile_ "completed" ft
@@ -548,6 +548,13 @@ humanReadableSize size
mB = kB * 1024
gB = mB * 1024
receivingFile_' :: StyledString -> AChatItem -> [StyledString]
receivingFile_' status (AChatItem _ _ (DirectChat Contact {localDisplayName = c}) ChatItem {file = Just CIFile {fileId, fileName}, chatDir = CIDirectRcv}) =
[status <> " receiving " <> fileTransferStr fileId fileName <> " from " <> ttyContact c]
receivingFile_' status (AChatItem _ _ _ ChatItem {file = Just CIFile {fileId, fileName}, chatDir = CIGroupRcv GroupMember {localDisplayName = m}}) =
[status <> " receiving " <> fileTransferStr fileId fileName <> " from " <> ttyContact m]
receivingFile_' status _ = [status <> " receiving file"] -- shouldn't happen
receivingFile_ :: StyledString -> RcvFileTransfer -> [StyledString]
receivingFile_ status ft@RcvFileTransfer {senderDisplayName = c} =
[status <> " receiving " <> rcvFile ft <> " from " <> ttyContact c]
+1 -1
View File
@@ -49,7 +49,7 @@ extra-deps:
# - simplexmq-1.0.0@sha256:34b2004728ae396e3ae449cd090ba7410781e2b3cefc59259915f4ca5daa9ea8,8561
# - ../simplexmq
- github: simplex-chat/simplexmq
commit: a638486bb3a9b212d15775d0bc63de2e619e0518
commit: d38303d5f1d55a90dd95d951ba8d798e17699269
# - terminal-0.2.0.0@sha256:de6770ecaae3197c66ac1f0db5a80cf5a5b1d3b64a66a05b50f442de5ad39570,2977
- github: simplex-chat/aeson
commit: 3eb66f9a68f103b5f1489382aad89f5712a64db7
+106 -5
View File
@@ -15,7 +15,7 @@ import qualified Data.Text as T
import Simplex.Chat.Controller (ChatController (..))
import Simplex.Chat.Types (ImageData (..), Profile (..), User (..))
import Simplex.Chat.Util (unlessM)
import System.Directory (doesFileExist)
import System.Directory (copyFile, doesFileExist)
import Test.Hspec
aliceProfile :: Profile
@@ -66,6 +66,9 @@ chatTests = do
describe "messages with files" $ do
it "send and receive message with file" testMessageWithFile
it "send and receive image" testSendImage
it "files folder: send and receive image" testFilesFoldersSendImage
it "files folder: sender deleted file during transfer" testFilesFoldersImageSndDelete
it "files folder: recipient deleted file during transfer" testFilesFoldersImageRcvDelete
it "send and receive image with text and quote" testSendImageWithTextAndQuote
it "send and receive image to group" testGroupSendImage
it "send and receive image with text and quote to group" testGroupSendImageWithTextAndQuote
@@ -1037,8 +1040,6 @@ testFileRcvCancel =
alice <## "sending file 1 (test.jpg) cancelled: bob"
]
checkPartialTransfer
where
waitFileExists f = unlessM (doesFileExist f) $ waitFileExists f
testGroupFileTransfer :: IO ()
testGroupFileTransfer =
@@ -1166,8 +1167,6 @@ testFileRcvCancelV2 =
alice <## "sending file 1 (test.jpg) cancelled: bob"
]
checkPartialTransfer
where
waitFileExists f = unlessM (doesFileExist f) $ waitFileExists f
testGroupFileTransferV2 :: IO ()
testGroupFileTransferV2 =
@@ -1261,6 +1260,97 @@ testSendImage =
dest `shouldBe` src
alice #$> ("/_get chat @2 count=100", chatF, [((1, ""), Just "./tests/fixtures/test.jpg")])
bob #$> ("/_get chat @2 count=100", chatF, [((0, ""), Just "./tests/tmp/test.jpg")])
-- deleting contact without files folder set should not remove file
bob ##> "/d alice"
bob <## "alice: contact is deleted"
fileExists <- doesFileExist "./tests/tmp/test.jpg"
fileExists `shouldBe` True
testFilesFoldersSendImage :: IO ()
testFilesFoldersSendImage =
testChat2 aliceProfile bobProfile $
\alice bob -> do
connectUsers alice bob
alice #$> ("/_files_folder ./tests/fixtures", id, "ok")
bob #$> ("/_files_folder ./tests/tmp/app_files", id, "ok")
alice ##> "/_send @2 file test.jpg json {\"text\":\"\",\"type\":\"image\",\"image\":\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAIAQMAAAD+wSzIAAAABlBMVEX///+/v7+jQ3Y5AAAADklEQVQI12P4AIX8EAgALgAD/aNpbtEAAAAASUVORK5CYII=\"}"
alice <# "/f @bob test.jpg"
alice <## "use /fc 1 to cancel sending"
bob <# "alice> sends file test.jpg (136.5 KiB / 139737 bytes)"
bob <## "use /fr 1 [<dir>/ | <path>] to receive it"
bob ##> "/fr 1"
bob <## "saving file 1 from alice to test.jpg"
concurrently_
(bob <## "started receiving file 1 (test.jpg) from alice")
(alice <## "started sending file 1 (test.jpg) to bob")
concurrently_
(bob <## "completed receiving file 1 (test.jpg) from alice")
(alice <## "completed sending file 1 (test.jpg) to bob")
src <- B.readFile "./tests/fixtures/test.jpg"
dest <- B.readFile "./tests/tmp/app_files/test.jpg"
dest `shouldBe` src
alice #$> ("/_get chat @2 count=100", chatF, [((1, ""), Just "test.jpg")])
bob #$> ("/_get chat @2 count=100", chatF, [((0, ""), Just "test.jpg")])
-- deleting contact with files folder set should remove file
checkActionDeletesFile "./tests/tmp/app_files/test.jpg" $ do
bob ##> "/d alice"
bob <## "alice: contact is deleted"
testFilesFoldersImageSndDelete :: IO ()
testFilesFoldersImageSndDelete =
testChat2 aliceProfile bobProfile $
\alice bob -> do
connectUsers alice bob
alice #$> ("/_files_folder ./tests/tmp/alice_app_files", id, "ok")
copyFile "./tests/fixtures/test.jpg" "./tests/tmp/alice_app_files/test.jpg"
bob #$> ("/_files_folder ./tests/tmp/bob_app_files", id, "ok")
alice ##> "/_send @2 file test.jpg json {\"text\":\"\",\"type\":\"image\",\"image\":\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAIAQMAAAD+wSzIAAAABlBMVEX///+/v7+jQ3Y5AAAADklEQVQI12P4AIX8EAgALgAD/aNpbtEAAAAASUVORK5CYII=\"}"
alice <# "/f @bob test.jpg"
alice <## "use /fc 1 to cancel sending"
bob <# "alice> sends file test.jpg (136.5 KiB / 139737 bytes)"
bob <## "use /fr 1 [<dir>/ | <path>] to receive it"
bob ##> "/fr 1"
bob <## "saving file 1 from alice to test.jpg"
concurrently_
(bob <## "started receiving file 1 (test.jpg) from alice")
(alice <## "started sending file 1 (test.jpg) to bob")
-- deleting contact should cancel and remove file
checkActionDeletesFile "./tests/tmp/alice_app_files/test.jpg" $ do
alice ##> "/d bob"
alice <## "bob: contact is deleted"
bob <## "alice cancelled sending file 1 (test.jpg)"
bob ##> "/fs 1"
bob <## "receiving file 1 (test.jpg) cancelled, received part path: test.jpg"
-- deleting contact should remove cancelled file
checkActionDeletesFile "./tests/tmp/bob_app_files/test.jpg" $ do
bob ##> "/d alice"
bob <## "alice: contact is deleted"
testFilesFoldersImageRcvDelete :: IO ()
testFilesFoldersImageRcvDelete =
testChat2 aliceProfile bobProfile $
\alice bob -> do
connectUsers alice bob
alice #$> ("/_files_folder ./tests/fixtures", id, "ok")
bob #$> ("/_files_folder ./tests/tmp/app_files", id, "ok")
alice ##> "/_send @2 file test.jpg json {\"text\":\"\",\"type\":\"image\",\"image\":\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAIAQMAAAD+wSzIAAAABlBMVEX///+/v7+jQ3Y5AAAADklEQVQI12P4AIX8EAgALgAD/aNpbtEAAAAASUVORK5CYII=\"}"
alice <# "/f @bob test.jpg"
alice <## "use /fc 1 to cancel sending"
bob <# "alice> sends file test.jpg (136.5 KiB / 139737 bytes)"
bob <## "use /fr 1 [<dir>/ | <path>] to receive it"
bob ##> "/fr 1"
bob <## "saving file 1 from alice to test.jpg"
concurrently_
(bob <## "started receiving file 1 (test.jpg) from alice")
(alice <## "started sending file 1 (test.jpg) to bob")
-- deleting contact should cancel and remove file
waitFileExists "./tests/tmp/app_files/test.jpg"
checkActionDeletesFile "./tests/tmp/app_files/test.jpg" $ do
bob ##> "/d alice"
bob <## "alice: contact is deleted"
alice <## "bob cancelled receiving file 1 (test.jpg)"
alice ##> "/fs 1"
alice <## "sending file 1 (test.jpg) cancelled: bob"
testSendImageWithTextAndQuote :: IO ()
testSendImageWithTextAndQuote =
@@ -1672,6 +1762,17 @@ checkPartialTransfer = do
B.unpack src `shouldStartWith` B.unpack dest
B.length src > B.length dest `shouldBe` True
checkActionDeletesFile :: FilePath -> IO () -> IO ()
checkActionDeletesFile file action = do
fileExistsBefore <- doesFileExist file
fileExistsBefore `shouldBe` True
action
fileExistsAfter <- doesFileExist file
fileExistsAfter `shouldBe` False
waitFileExists :: FilePath -> IO ()
waitFileExists f = unlessM (doesFileExist f) $ waitFileExists f
connectUsers :: TestCC -> TestCC -> IO ()
connectUsers cc1 cc2 = do
name1 <- showName cc1