From 78c0fe73a7b590a8a9f0dd9f4bacd62d4f55a37f Mon Sep 17 00:00:00 2001 From: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com> Date: Wed, 21 Jun 2023 21:54:12 +0100 Subject: [PATCH 1/4] readme: add simplex-devs group --- README.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/README.md b/README.md index 31ad125c26..904f1dd2b8 100644 --- a/README.md +++ b/README.md @@ -66,6 +66,13 @@ Messages not following these rules will be deleted, the right to send messages m You can join an English-speaking users group if you want to ask any questions: [#SimpleX-Group-4](https://simplex.chat/contact#/?v=1-2&smp=smp%3A%2F%2Fu2dS9sG8nMNURyZwqASV4yROM28Er0luVTx5X1CsMrU%3D%40smp4.simplex.im%2Fw2GlucRXtRVgYnbt_9ZP-kmt76DekxxS%23%2F%3Fv%3D1-2%26dh%3DMCowBQYDK2VuAyEA0tJhTyMGUxznwmjb7aT24P1I1Wry_iURTuhOFlMb1Eo%253D%26srv%3Do5vmywmrnaxalvz6wi3zicyftgio6psuvyniis6gco6bp6ekl4cqj4id.onion&data=%7B%22type%22%3A%22group%22%2C%22groupLinkId%22%3A%22WoPxjFqGEDlVazECOSi2dg%3D%3D%22%7D) +There is also a group [#simplex-devs](https://simplex.chat/contact#/?v=1-2&smp=smp%3A%2F%2Fu2dS9sG8nMNURyZwqASV4yROM28Er0luVTx5X1CsMrU%3D%40smp4.simplex.im%2F6eHqy7uAbZPOcA6qBtrQgQquVlt4Ll91%23%2F%3Fv%3D1-2%26dh%3DMCowBQYDK2VuAyEAqV_pg3FF00L98aCXp4D3bOs4Sxv_UmSd-gb0juVoQVs%253D%26srv%3Do5vmywmrnaxalvz6wi3zicyftgio6psuvyniis6gco6bp6ekl4cqj4id.onion&data=%7B%22type%22%3A%22group%22%2C%22groupLinkId%22%3A%22XonlixcHBIb2ijCehbZoiw%3D%3D%22%7D) for developers who build on SimpleX platform: + +- chat bots and automations +- integrations with other apps +- social apps and services +- etc. + There are groups in other languages, that we have the apps interface translated into. These groups are for testing, and asking questions to other SimpleX Chat users: [\#SimpleX-DE](https://simplex.chat/contact#/?v=1-2&smp=smp%3A%2F%2FPQUV2eL0t7OStZOoAsPEV2QYWt4-xilbakvGUGOItUo%3D%40smp6.simplex.im%2FkIEl7OQzcp-J6aDmjdlQbRJwqkcZE7XR%23%2F%3Fv%3D1-2%26dh%3DMCowBQYDK2VuAyEAR16PCu02MobRmKAsjzhDWMZcWP9hS8l5AUZi-Gs8z18%253D%26srv%3Dbylepyau3ty4czmn77q4fglvperknl4bi2eb2fdy2bh4jxtf32kf73yd.onion&data=%7B%22type%22%3A%22group%22%2C%22groupLinkId%22%3A%22puYPMCQt11yPUvgmI5jCiw%3D%3D%22%7D) (German-speaking), [\#SimpleX-ES](https://simplex.chat/contact#/?v=1-2&smp=smp%3A%2F%2FPQUV2eL0t7OStZOoAsPEV2QYWt4-xilbakvGUGOItUo%3D%40smp6.simplex.im%2FaJ8O1O8A8GbeoaHTo_V8dcefaCl7ouPb%23%2F%3Fv%3D1-2%26dh%3DMCowBQYDK2VuAyEA034qWTA3sWcTsi6aWhNf9BA34vKVCFaEBdP2R66z6Ao%253D%26srv%3Dbylepyau3ty4czmn77q4fglvperknl4bi2eb2fdy2bh4jxtf32kf73yd.onion&data=%7B%22type%22%3A%22group%22%2C%22groupLinkId%22%3A%22wiZ1v_wNjLPlT-nCSB-bRA%3D%3D%22%7D) (Spanish-speaking), [\#SimpleX-FR](https://simplex.chat/contact#/?v=1-2&smp=smp%3A%2F%2Fhpq7_4gGJiilmz5Rf-CswuU5kZGkm_zOIooSw6yALRg%3D%40smp5.simplex.im%2FvIHQDxTor53nwnWWTy5cHNwQQAdWN5Hw%23%2F%3Fv%3D1-2%26dh%3DMCowBQYDK2VuAyEAPdgK1eBnETmgiqEQufbUkydKBJafoRx4iRrtrC2NAGc%253D%26srv%3Djjbyvoemxysm7qxap7m5d5m35jzv5qq6gnlv7s4rsn7tdwwmuqciwpid.onion&data=%7B%22type%22%3A%22group%22%2C%22groupLinkId%22%3A%221FyUryBPza-1ZFFE80Ekbg%3D%3D%22%7D) (French-speaking), [\#SimpleX-RU](https://simplex.chat/contact#/?v=1-2&smp=smp%3A%2F%2FPQUV2eL0t7OStZOoAsPEV2QYWt4-xilbakvGUGOItUo%3D%40smp6.simplex.im%2FXZyt3hJmWsycpN7Dqve_wbrAqb6myk1R%23%2F%3Fv%3D1-2%26dh%3DMCowBQYDK2VuAyEAMFVIoytozTEa_QXOgoZFq_oe0IwZBYKvW50trSFXzXo%253D%26srv%3Dbylepyau3ty4czmn77q4fglvperknl4bi2eb2fdy2bh4jxtf32kf73yd.onion&data=%7B%22type%22%3A%22group%22%2C%22groupLinkId%22%3A%22xz05ngjA3pNIxLZ32a8Vxg%3D%3D%22%7D) (Russian-speaking), [\#SimpleX-IT](https://simplex.chat/contact#/?v=1-2&smp=smp%3A%2F%2Fu2dS9sG8nMNURyZwqASV4yROM28Er0luVTx5X1CsMrU%3D%40smp4.simplex.im%2F0weR-ZgDUl7ruOtI_8TZwEsnJP6UiImA%23%2F%3Fv%3D1-2%26dh%3DMCowBQYDK2VuAyEAq4PSThO9Fvb5ydF48wB0yNbpzCbuQJCW3vZ9BGUfcxk%253D%26srv%3Do5vmywmrnaxalvz6wi3zicyftgio6psuvyniis6gco6bp6ekl4cqj4id.onion&data=%7B%22type%22%3A%22group%22%2C%22groupLinkId%22%3A%22e-iceLA0SctC62eARgYDWg%3D%3D%22%7D) (Italian-speaking). @@ -283,6 +290,8 @@ You can: If you are considering developing with SimpleX platform please get in touch for any advice and support. +Please also join [#simplex-devs](https://simplex.chat/contact#/?v=1-2&smp=smp%3A%2F%2Fu2dS9sG8nMNURyZwqASV4yROM28Er0luVTx5X1CsMrU%3D%40smp4.simplex.im%2F6eHqy7uAbZPOcA6qBtrQgQquVlt4Ll91%23%2F%3Fv%3D1-2%26dh%3DMCowBQYDK2VuAyEAqV_pg3FF00L98aCXp4D3bOs4Sxv_UmSd-gb0juVoQVs%253D%26srv%3Do5vmywmrnaxalvz6wi3zicyftgio6psuvyniis6gco6bp6ekl4cqj4id.onion&data=%7B%22type%22%3A%22group%22%2C%22groupLinkId%22%3A%22XonlixcHBIb2ijCehbZoiw%3D%3D%22%7D) group to ask any questions and share your success stories. + ## Roadmap - ✅ Easy to deploy SimpleX server with in-memory message storage, without any dependencies. From da2622f00e12f5bb0bdf219b7e2fe73c84cf4b61 Mon Sep 17 00:00:00 2001 From: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com> Date: Thu, 22 Jun 2023 20:38:09 +0400 Subject: [PATCH 2/4] core: moderate messages that have arrived after the event of moderation (#2604) * core: moderate messages that have arrived after the event of moderation * remove index * test, delete moderation * unused selector * rework * refactor * change error * parameter * fix syntax * refactor * Nothing --------- Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com> --- simplex-chat.cabal | 1 + src/Simplex/Chat.hs | 33 +++++-- src/Simplex/Chat/Messages.hs | 8 ++ .../M20230621_chat_item_moderations.hs | 41 ++++++++ src/Simplex/Chat/Migrations/chat_schema.sql | 25 +++++ src/Simplex/Chat/Store/Groups.hs | 4 +- src/Simplex/Chat/Store/Messages.hs | 44 +++++++++ src/Simplex/Chat/Store/Migrations.hs | 4 +- src/Simplex/Chat/Store/Shared.hs | 1 + src/Simplex/Chat/View.hs | 1 + tests/ChatTests/Groups.hs | 99 +++++++++++++++++-- 11 files changed, 245 insertions(+), 16 deletions(-) create mode 100644 src/Simplex/Chat/Migrations/M20230621_chat_item_moderations.hs diff --git a/simplex-chat.cabal b/simplex-chat.cabal index 8620f881d7..162a57c237 100644 --- a/simplex-chat.cabal +++ b/simplex-chat.cabal @@ -102,6 +102,7 @@ library Simplex.Chat.Migrations.M20230529_indexes Simplex.Chat.Migrations.M20230608_deleted_contacts Simplex.Chat.Migrations.M20230618_favorite_chats + Simplex.Chat.Migrations.M20230621_chat_item_moderations Simplex.Chat.Mobile Simplex.Chat.Mobile.WebRTC Simplex.Chat.Options diff --git a/src/Simplex/Chat.hs b/src/Simplex/Chat.hs index b823a4bccb..908547fe38 100644 --- a/src/Simplex/Chat.hs +++ b/src/Simplex/Chat.hs @@ -3533,15 +3533,34 @@ processAgentMessageConn user@User {userId} corrId agentConnId agentMessage = do e -> throwError e newGroupContentMessage :: GroupInfo -> GroupMember -> MsgContainer -> RcvMessage -> MsgMeta -> m () - newGroupContentMessage gInfo m@GroupMember {localDisplayName = c, memberId} mc msg@RcvMessage {sharedMsgId_} msgMeta = do + newGroupContentMessage gInfo m@GroupMember {localDisplayName = c, memberId, memberRole} mc msg@RcvMessage {sharedMsgId_} msgMeta = do -- TODO integrity message check - let (ExtMsgContent content fInv_ _ _) = mcExtMsgContent mc if isVoice content && not (groupFeatureAllowed SGFVoice gInfo) then void $ newChatItem (CIRcvGroupFeatureRejected GFVoice) Nothing Nothing False else do - let ExtMsgContent _ _ itemTTL live_ = mcExtMsgContent mc - timed_ = rcvGroupCITimed gInfo itemTTL + -- check if message moderation event was received ahead of message + let timed_ = rcvGroupCITimed gInfo itemTTL live = fromMaybe False live_ + withStore' (\db -> getCIModeration db user gInfo memberId sharedMsgId_) >>= \case + Just ciModeration -> do + applyModeration timed_ live ciModeration + withStore' $ \db -> deleteCIModeration db gInfo memberId sharedMsgId_ + Nothing -> createItem timed_ live + where + ExtMsgContent content fInv_ itemTTL live_ = mcExtMsgContent mc + applyModeration timed_ live CIModeration {moderatorMember = moderator@GroupMember {memberRole = moderatorRole}, createdByMsgId, moderatedAt} + | moderatorRole < GRAdmin || moderatorRole < memberRole = + createItem timed_ live + | groupFeatureAllowed SGFFullDelete gInfo = do + ci <- saveRcvChatItem' user (CDGroupRcv gInfo m) msg sharedMsgId_ msgMeta CIRcvModerated Nothing timed_ False + ci' <- withStore' $ \db -> updateGroupChatItemModerated db user gInfo (CChatItem SMDRcv ci) moderator moderatedAt + toView $ CRNewChatItem user ci' + | otherwise = do + file_ <- processFileInvitation fInv_ content $ \db -> createRcvGroupFileTransfer db userId m + ci <- saveRcvChatItem' user (CDGroupRcv gInfo m) msg sharedMsgId_ msgMeta (CIRcvMsgContent content) (snd <$> file_) timed_ False + cr <- markGroupCIDeleted user gInfo (CChatItem SMDRcv ci) createdByMsgId False (Just moderator) moderatedAt + toView cr + createItem timed_ live = do file_ <- processFileInvitation fInv_ content $ \db -> createRcvGroupFileTransfer db userId m ChatItem {formattedText} <- newChatItem (CIRcvMsgContent content) (snd <$> file_) timed_ live autoAcceptFile file_ @@ -3549,7 +3568,6 @@ processAgentMessageConn user@User {userId} corrId agentConnId agentMessage = do whenGroupNtfs user gInfo $ do showMsgToast ("#" <> g <> " " <> c <> "> ") content formattedText setActive $ ActiveG g - where newChatItem ciContent ciFile_ timed_ live = do ci <- saveRcvChatItem' user (CDGroupRcv gInfo m) msg sharedMsgId_ msgMeta ciContent ciFile_ timed_ live reactions <- maybe (pure []) (\sharedMsgId -> withStore' $ \db -> getGroupCIReactions db gInfo memberId sharedMsgId) sharedMsgId_ @@ -3602,7 +3620,10 @@ processAgentMessageConn user@User {userId} corrId agentConnId agentMessage = do | sameMemberId memberId mem && msgMemberId == memberId -> delete ci Nothing >>= toView | otherwise -> deleteMsg mem ci CIGroupSnd -> deleteMsg membership ci - Left e -> messageError $ "x.msg.del: message not found, " <> tshow e + Left e + | msgMemberId == memberId -> messageError $ "x.msg.del: message not found, " <> tshow e + | senderRole < GRAdmin -> messageError $ "x.msg.del: message not found, message of another member with insufficient member permissions, " <> tshow e + | otherwise -> withStore' $ \db -> createCIModeration db gInfo m msgMemberId sharedMsgId msgId brokerTs where deleteMsg :: GroupMember -> CChatItem 'CTGroup -> m () deleteMsg mem ci = case sndMemberId_ of diff --git a/src/Simplex/Chat/Messages.hs b/src/Simplex/Chat/Messages.hs index 05cc6909b1..2c5f4755f3 100644 --- a/src/Simplex/Chat/Messages.hs +++ b/src/Simplex/Chat/Messages.hs @@ -907,3 +907,11 @@ mkItemVersion ChatItem {content, meta} = version <$> ciMsgContent content itemVersionTs = itemTs, createdAt = createdAt } + +data CIModeration = CIModeration + { moderationId :: Int64, + moderatorMember :: GroupMember, + createdByMsgId :: MessageId, + moderatedAt :: UTCTime + } + deriving (Show) diff --git a/src/Simplex/Chat/Migrations/M20230621_chat_item_moderations.hs b/src/Simplex/Chat/Migrations/M20230621_chat_item_moderations.hs new file mode 100644 index 0000000000..449e21e209 --- /dev/null +++ b/src/Simplex/Chat/Migrations/M20230621_chat_item_moderations.hs @@ -0,0 +1,41 @@ +{-# LANGUAGE QuasiQuotes #-} + +module Simplex.Chat.Migrations.M20230621_chat_item_moderations where + +import Database.SQLite.Simple (Query) +import Database.SQLite.Simple.QQ (sql) + +-- moderations that could not be applied - for messages that haven't been received at the time of moderation +m20230621_chat_item_moderations :: Query +m20230621_chat_item_moderations = + [sql| +CREATE TABLE chat_item_moderations ( + chat_item_moderation_id INTEGER PRIMARY KEY, + group_id INTEGER NOT NULL REFERENCES groups ON DELETE CASCADE, + moderator_member_id INTEGER NOT NULL REFERENCES group_members ON DELETE CASCADE, + item_member_id BLOB NOT NULL, + shared_msg_id BLOB NOT NULL, + created_by_msg_id INTEGER REFERENCES messages(message_id) ON DELETE SET NULL, + moderated_at TEXT NOT NULL, -- broker_ts of creating message + created_at TEXT NOT NULL DEFAULT(datetime('now')), + updated_at TEXT NOT NULL DEFAULT(datetime('now')) +); + +CREATE INDEX idx_chat_item_moderations_group_id ON chat_item_moderations(group_id); +CREATE INDEX idx_chat_item_moderations_moderator_member_id ON chat_item_moderations(moderator_member_id); +CREATE INDEX idx_chat_item_moderations_created_by_msg_id ON chat_item_moderations(created_by_msg_id); + +CREATE INDEX idx_chat_item_moderations_group ON chat_item_moderations(group_id, item_member_id, shared_msg_id); +|] + +down_m20230621_chat_item_moderations :: Query +down_m20230621_chat_item_moderations = + [sql| +DROP INDEX idx_chat_item_moderations_group; + +DROP INDEX idx_chat_item_moderations_created_by_msg_id; +DROP INDEX idx_chat_item_moderations_moderator_member_id; +DROP INDEX idx_chat_item_moderations_group_id; + +DROP TABLE chat_item_moderations; +|] diff --git a/src/Simplex/Chat/Migrations/chat_schema.sql b/src/Simplex/Chat/Migrations/chat_schema.sql index 05c7857a24..176397ddca 100644 --- a/src/Simplex/Chat/Migrations/chat_schema.sql +++ b/src/Simplex/Chat/Migrations/chat_schema.sql @@ -481,6 +481,17 @@ CREATE TABLE chat_item_reactions( created_at TEXT NOT NULL DEFAULT(datetime('now')), updated_at TEXT NOT NULL DEFAULT(datetime('now')) ); +CREATE TABLE chat_item_moderations( + chat_item_moderation_id INTEGER PRIMARY KEY, + group_id INTEGER NOT NULL REFERENCES groups ON DELETE CASCADE, + moderator_member_id INTEGER NOT NULL REFERENCES group_members ON DELETE CASCADE, + item_member_id BLOB NOT NULL, + shared_msg_id BLOB NOT NULL, + created_by_msg_id INTEGER REFERENCES messages(message_id) ON DELETE SET NULL, + moderated_at TEXT NOT NULL, -- broker_ts of creating message + created_at TEXT NOT NULL DEFAULT(datetime('now')), + updated_at TEXT NOT NULL DEFAULT(datetime('now')) +); CREATE INDEX contact_profiles_index ON contact_profiles( display_name, full_name @@ -658,3 +669,17 @@ CREATE INDEX idx_msg_deliveries_agent_ack_cmd_id ON msg_deliveries( CREATE INDEX msg_delivery_events_msg_delivery_id ON msg_delivery_events( msg_delivery_id ); +CREATE INDEX idx_chat_item_moderations_group_id ON chat_item_moderations( + group_id +); +CREATE INDEX idx_chat_item_moderations_moderator_member_id ON chat_item_moderations( + moderator_member_id +); +CREATE INDEX idx_chat_item_moderations_created_by_msg_id ON chat_item_moderations( + created_by_msg_id +); +CREATE INDEX idx_chat_item_moderations_group ON chat_item_moderations( + group_id, + item_member_id, + shared_msg_id +); diff --git a/src/Simplex/Chat/Store/Groups.hs b/src/Simplex/Chat/Store/Groups.hs index acf722a3ac..1829192dfc 100644 --- a/src/Simplex/Chat/Store/Groups.hs +++ b/src/Simplex/Chat/Store/Groups.hs @@ -17,6 +17,7 @@ module Simplex.Chat.Store.Groups toGroupInfo, toGroupMember, toMaybeGroupMember, + -- * Group functions createGroupLink, getGroupLinkConnection, @@ -1061,9 +1062,6 @@ getGroupMemberIdByName db User {userId} groupId groupMemberName = ExceptT . firstRow fromOnly (SEGroupMemberNameNotFound groupId groupMemberName) $ DB.query db "SELECT group_member_id FROM group_members WHERE user_id = ? AND group_id = ? AND local_display_name = ?" (userId, groupId, groupMemberName) - - - getMatchingContacts :: DB.Connection -> User -> Contact -> IO [Contact] getMatchingContacts db user@User {userId} Contact {contactId, profile = LocalProfile {displayName, fullName, image}} = do contactIds <- diff --git a/src/Simplex/Chat/Store/Messages.hs b/src/Simplex/Chat/Store/Messages.hs index 1b1df64a25..0cc17320ed 100644 --- a/src/Simplex/Chat/Store/Messages.hs +++ b/src/Simplex/Chat/Store/Messages.hs @@ -14,6 +14,7 @@ module Simplex.Chat.Store.Messages ( getContactConnIds_, getDirectChatReactions_, toDirectChatItem, + -- * Message and chat item functions deleteContactCIs, getGroupFileInfo, @@ -83,6 +84,9 @@ module Simplex.Chat.Store.Messages deleteContactExpiredCIs, getGroupExpiredFileInfo, deleteGroupExpiredCIs, + createCIModeration, + getCIModeration, + deleteCIModeration, ) where @@ -1803,3 +1807,43 @@ deleteGroupExpiredCIs db User {userId} GroupInfo {groupId} expirationDate create DB.execute db "DELETE FROM messages WHERE group_id = ? AND created_at <= ?" (groupId, min expirationDate createdAtCutoff) DB.execute db "DELETE FROM chat_item_reactions WHERE group_id = ? AND reaction_ts <= ? AND created_at <= ?" (groupId, expirationDate, createdAtCutoff) DB.execute db "DELETE FROM chat_items WHERE user_id = ? AND group_id = ? AND item_ts <= ? AND created_at <= ?" (userId, groupId, expirationDate, createdAtCutoff) + +createCIModeration :: DB.Connection -> GroupInfo -> GroupMember -> MemberId -> SharedMsgId -> MessageId -> UTCTime -> IO () +createCIModeration db GroupInfo {groupId} moderatorMember itemMemberId itemSharedMId msgId moderatedAtTs = + DB.execute + db + [sql| + INSERT INTO chat_item_moderations + (group_id, moderator_member_id, item_member_id, shared_msg_id, created_by_msg_id, moderated_at) + VALUES (?,?,?,?,?,?) + |] + (groupId, groupMemberId' moderatorMember, itemMemberId, itemSharedMId, msgId, moderatedAtTs) + +getCIModeration :: DB.Connection -> User -> GroupInfo -> MemberId -> Maybe SharedMsgId -> IO (Maybe CIModeration) +getCIModeration _ _ _ _ Nothing = pure Nothing +getCIModeration db user GroupInfo {groupId} itemMemberId (Just sharedMsgId) = do + r_ <- + maybeFirstRow id $ + DB.query + db + [sql| + SELECT chat_item_moderation_id, moderator_member_id, created_by_msg_id, moderated_at + FROM chat_item_moderations + WHERE group_id = ? AND item_member_id = ? AND shared_msg_id = ? + LIMIT 1 + |] + (groupId, itemMemberId, sharedMsgId) + case r_ of + Just (moderationId, moderatorId, createdByMsgId, moderatedAt) -> do + runExceptT (getGroupMember db user groupId moderatorId) >>= \case + Right moderatorMember -> pure (Just CIModeration {moderationId, moderatorMember, createdByMsgId, moderatedAt}) + _ -> pure Nothing + _ -> pure Nothing + +deleteCIModeration :: DB.Connection -> GroupInfo -> MemberId -> Maybe SharedMsgId -> IO () +deleteCIModeration _ _ _ Nothing = pure () +deleteCIModeration db GroupInfo {groupId} itemMemberId (Just sharedMsgId) = + DB.execute + db + "DELETE FROM chat_item_moderations WHERE group_id = ? AND item_member_id = ? AND shared_msg_id = ?" + (groupId, itemMemberId, sharedMsgId) diff --git a/src/Simplex/Chat/Store/Migrations.hs b/src/Simplex/Chat/Store/Migrations.hs index a155841b84..f1294de64a 100644 --- a/src/Simplex/Chat/Store/Migrations.hs +++ b/src/Simplex/Chat/Store/Migrations.hs @@ -72,6 +72,7 @@ import Simplex.Chat.Migrations.M20230526_indexes import Simplex.Chat.Migrations.M20230529_indexes import Simplex.Chat.Migrations.M20230608_deleted_contacts import Simplex.Chat.Migrations.M20230618_favorite_chats +import Simplex.Chat.Migrations.M20230621_chat_item_moderations import Simplex.Messaging.Agent.Store.SQLite.Migrations (Migration (..)) schemaMigrations :: [(String, Query, Maybe Query)] @@ -143,7 +144,8 @@ schemaMigrations = ("20230526_indexes", m20230526_indexes, Just down_m20230526_indexes), ("20230529_indexes", m20230529_indexes, Just down_m20230529_indexes), ("20230608_deleted_contacts", m20230608_deleted_contacts, Just down_m20230608_deleted_contacts), - ("20230618_favorite_chats", m20230618_favorite_chats, Just down_m20230618_favorite_chats) + ("20230618_favorite_chats", m20230618_favorite_chats, Just down_m20230618_favorite_chats), + ("20230621_chat_item_moderations", m20230621_chat_item_moderations, Just down_m20230621_chat_item_moderations) ] -- | The list of migrations in ascending order by date diff --git a/src/Simplex/Chat/Store/Shared.hs b/src/Simplex/Chat/Store/Shared.hs index 0d49915899..28c70c03ca 100644 --- a/src/Simplex/Chat/Store/Shared.hs +++ b/src/Simplex/Chat/Store/Shared.hs @@ -56,6 +56,7 @@ data StoreError | SEGroupNotFoundByName {groupName :: GroupName} | SEGroupMemberNameNotFound {groupId :: GroupId, groupMemberName :: ContactName} | SEGroupMemberNotFound {groupMemberId :: GroupMemberId} + | SEGroupMemberNotFoundByMemberId {memberId :: MemberId} | SEGroupWithoutUser | SEDuplicateGroupMember | SEGroupAlreadyJoined diff --git a/src/Simplex/Chat/View.hs b/src/Simplex/Chat/View.hs index 1c3eeafe02..e67746a2c8 100644 --- a/src/Simplex/Chat/View.hs +++ b/src/Simplex/Chat/View.hs @@ -404,6 +404,7 @@ viewChatItem chat ci@ChatItem {chatDir, meta = meta, content, quotedItem, file} CIRcvIntegrityError err -> viewRcvIntegrityError from err ts tz meta CIRcvDecryptionError err n -> viewRcvDecryptionError from err n ts tz meta CIRcvGroupInvitation {} -> showRcvItemProhibited from + CIRcvModerated {} -> receivedWithTime_ ts tz (ttyFromGroup g m) quote meta [plainContent content] False _ -> showRcvItem from where from = ttyFromGroup g m diff --git a/tests/ChatTests/Groups.hs b/tests/ChatTests/Groups.hs index 85b9c720b1..5e9a419231 100644 --- a/tests/ChatTests/Groups.hs +++ b/tests/ChatTests/Groups.hs @@ -40,8 +40,10 @@ chatGroupTests = do it "update member role" testUpdateMemberRole it "unused contacts are deleted after all their groups are deleted" testGroupDeleteUnusedContacts it "group description is shown as the first message to new members" testGroupDescription - it "delete message of another group member" testGroupMemberMessageDelete - it "full delete message of another group member" testGroupMemberMessageFullDelete + it "moderate message of another group member" testGroupModerate + it "moderate message of another group member (full delete)" testGroupModerateFullDelete + it "moderate message that arrives after the event of moderation" testGroupDelayedModeration + it "moderate message that arrives after the event of moderation (full delete)" testGroupDelayedModerationFullDelete describe "async group connections" $ do xit "create and join group when clients go offline" testGroupAsync describe "group links" $ do @@ -1308,8 +1310,8 @@ testGroupDescription = testChat4 aliceProfile bobProfile cathProfile danProfile cc <## "#team: bob added dan (Daniel) to the group (connecting...)" cc <## "#team: new member dan is connected" -testGroupMemberMessageDelete :: HasCallStack => FilePath -> IO () -testGroupMemberMessageDelete = +testGroupModerate :: HasCallStack => FilePath -> IO () +testGroupModerate = testChat3 aliceProfile bobProfile cathProfile $ \alice bob cath -> do createGroup3 "team" alice bob cath @@ -1339,8 +1341,8 @@ testGroupMemberMessageDelete = bob #$> ("/_get chat #1 count=1", chat, [(0, "hi [marked deleted by you]")]) cath #$> ("/_get chat #1 count=1", chat, [(1, "hi [marked deleted by bob]")]) -testGroupMemberMessageFullDelete :: HasCallStack => FilePath -> IO () -testGroupMemberMessageFullDelete = +testGroupModerateFullDelete :: HasCallStack => FilePath -> IO () +testGroupModerateFullDelete = testChat3 aliceProfile bobProfile cathProfile $ \alice bob cath -> do createGroup3 "team" alice bob cath @@ -1377,6 +1379,91 @@ testGroupMemberMessageFullDelete = bob #$> ("/_get chat #1 count=1", chat, [(0, "moderated [deleted by you]")]) cath #$> ("/_get chat #1 count=1", chat, [(1, "moderated [deleted by bob]")]) +testGroupDelayedModeration :: HasCallStack => FilePath -> IO () +testGroupDelayedModeration tmp = do + withNewTestChat tmp "alice" aliceProfile $ \alice -> do + withNewTestChat tmp "bob" bobProfile $ \bob -> do + createGroup2 "team" alice bob + withNewTestChat tmp "cath" cathProfile $ \cath -> do + connectUsers alice cath + addMember "team" alice cath GRMember + cath ##> "/j team" + concurrentlyN_ + [ alice <## "#team: cath joined the group", + cath <## "#team: you joined the group" + ] + threadDelay 1000000 + cath #> "#team hi" -- message is pending for bob + alice <# "#team cath> hi" + alice ##> "\\\\ #team @cath hi" + alice <## "message marked deleted by you" + cath <# "#team cath> [marked deleted by alice] hi" + withTestChat tmp "bob" $ \bob -> do + bob <## "1 contacts connected (use /cs for the list)" + bob <## "#team: connected to server(s)" + bob <## "#team: alice added cath (Catherine) to the group (connecting...)" + withTestChat tmp "cath" $ \cath -> do + cath <## "2 contacts connected (use /cs for the list)" + cath <## "#team: connected to server(s)" + cath <## "#team: member bob (Bob) is connected" + bob + <### [ "#team: new member cath is connected", + EndsWith "#team cath> [marked deleted by alice] hi" + ] + alice #$> ("/_get chat #1 count=1", chat, [(0, "hi [marked deleted by you]")]) + cath #$> ("/_get chat #1 count=2", chat, [(1, "hi [marked deleted by alice]"), (0, "connected")]) + bob ##> "/_get chat #1 count=2" + r <- chat <$> getTermLine bob + r `shouldMatchList` [(0, "connected"), (0, "hi [marked deleted by alice]")] + +testGroupDelayedModerationFullDelete :: HasCallStack => FilePath -> IO () +testGroupDelayedModerationFullDelete tmp = do + withNewTestChat tmp "alice" aliceProfile $ \alice -> do + withNewTestChat tmp "bob" bobProfile $ \bob -> do + createGroup2 "team" alice bob + withNewTestChat tmp "cath" cathProfile $ \cath -> do + connectUsers alice cath + addMember "team" alice cath GRMember + cath ##> "/j team" + concurrentlyN_ + [ alice <## "#team: cath joined the group", + cath <## "#team: you joined the group" + ] + threadDelay 1000000 + cath #> "#team hi" -- message is pending for bob + alice <# "#team cath> hi" + alice ##> "\\\\ #team @cath hi" + alice <## "message marked deleted by you" + cath <# "#team cath> [marked deleted by alice] hi" + -- if full deletion was enabled at time of moderation, cath would delete pending message as well, + -- that's why we set it afterwards to test delayed moderation for bob + alice ##> "/set delete #team on" + alice <## "updated group preferences:" + alice <## "Full deletion: on" + cath <## "alice updated group #team:" + cath <## "updated group preferences:" + cath <## "Full deletion: on" + withTestChat tmp "bob" $ \bob -> do + bob <## "1 contacts connected (use /cs for the list)" + bob <## "#team: connected to server(s)" + bob <## "#team: alice added cath (Catherine) to the group (connecting...)" + bob <## "alice updated group #team:" + bob <## "updated group preferences:" + bob <## "Full deletion: on" + withTestChat tmp "cath" $ \cath -> do + cath <## "2 contacts connected (use /cs for the list)" + cath <## "#team: connected to server(s)" + cath <## "#team: member bob (Bob) is connected" + bob + <### [ "#team: new member cath is connected", + EndsWith "#team cath> moderated [deleted by alice]" + ] + alice #$> ("/_get chat #1 count=2", chat, [(0, "hi [marked deleted by you]"), (1, "Full deletion: on")]) + cath #$> ("/_get chat #1 count=3", chat, [(1, "hi [marked deleted by alice]"), (0, "Full deletion: on"), (0, "connected")]) + bob ##> "/_get chat #1 count=3" + r <- chat <$> getTermLine bob + r `shouldMatchList` [(0, "Full deletion: on"), (0, "connected"), (0, "moderated [deleted by alice]")] + testGroupAsync :: HasCallStack => FilePath -> IO () testGroupAsync tmp = do print (0 :: Integer) From 6da18d9b2a4b563a639fa513b1f1ec34bd4367b2 Mon Sep 17 00:00:00 2001 From: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com> Date: Sat, 24 Jun 2023 12:36:07 +0100 Subject: [PATCH 3/4] core: group permision to allow files and media (#2610) * core: group permision to allow files and media * test --- src/Simplex/Chat.hs | 59 +++++++++++++++++++++------------------ src/Simplex/Chat/Help.hs | 1 + src/Simplex/Chat/Types.hs | 43 ++++++++++++++++++++++------ tests/ChatTests/Files.hs | 25 +++++++++++++++++ tests/ChatTests/Groups.hs | 1 + tests/ChatTests/Utils.hs | 3 +- tests/ProtocolTests.hs | 2 +- 7 files changed, 97 insertions(+), 37 deletions(-) diff --git a/src/Simplex/Chat.hs b/src/Simplex/Chat.hs index 908547fe38..6d7ab60e35 100644 --- a/src/Simplex/Chat.hs +++ b/src/Simplex/Chat.hs @@ -551,22 +551,25 @@ processChatCommand = \case quoteData ChatItem {content = CIRcvMsgContent qmc} = pure (qmc, CIQDirectRcv, False) quoteData _ = throwChatError CEInvalidQuote CTGroup -> do - g@(Group gInfo@GroupInfo {groupId, membership, localDisplayName = gName} ms) <- withStore $ \db -> getGroup db user chatId + g@(Group gInfo _) <- withStore $ \db -> getGroup db user chatId assertUserGroupRole gInfo GRAuthor - if isVoice mc && not (groupFeatureAllowed SGFVoice gInfo) - then pure $ chatCmdError (Just user) ("feature not allowed " <> T.unpack (groupFeatureNameText GFVoice)) - else do - (fInv_, ciFile_, ft_) <- unzipMaybe3 <$> setupSndFileTransfer g (length $ filter memberCurrent ms) - timed_ <- sndGroupCITimed live gInfo itemTTL - (msgContainer, quotedItem_) <- prepareMsg fInv_ timed_ membership - msg@SndMessage {sharedMsgId} <- sendGroupMessage user gInfo ms (XMsgNew msgContainer) - mapM_ (sendGroupFileInline ms sharedMsgId) ft_ - ci <- saveSndChatItem' user (CDGroupSnd gInfo) msg (CISndMsgContent mc) ciFile_ quotedItem_ timed_ live - forM_ (timed_ >>= timedDeleteAt') $ - startProximateTimedItemThread user (ChatRef CTGroup groupId, chatItemId' ci) - setActive $ ActiveG gName - pure $ CRNewChatItem user (AChatItem SCTGroup SMDSnd (GroupChat gInfo) ci) + send g where + send g@(Group gInfo@GroupInfo {groupId, membership, localDisplayName = gName} ms) + | isVoice mc && not (groupFeatureAllowed SGFVoice gInfo) = notAllowedError GFVoice + | not (isVoice mc) && isJust file_ && not (groupFeatureAllowed SGFFiles gInfo) = notAllowedError GFFiles + | otherwise = do + (fInv_, ciFile_, ft_) <- unzipMaybe3 <$> setupSndFileTransfer g (length $ filter memberCurrent ms) + timed_ <- sndGroupCITimed live gInfo itemTTL + (msgContainer, quotedItem_) <- prepareMsg fInv_ timed_ membership + msg@SndMessage {sharedMsgId} <- sendGroupMessage user gInfo ms (XMsgNew msgContainer) + mapM_ (sendGroupFileInline ms sharedMsgId) ft_ + ci <- saveSndChatItem' user (CDGroupSnd gInfo) msg (CISndMsgContent mc) ciFile_ quotedItem_ timed_ live + forM_ (timed_ >>= timedDeleteAt') $ + startProximateTimedItemThread user (ChatRef CTGroup groupId, chatItemId' ci) + setActive $ ActiveG gName + pure $ CRNewChatItem user (AChatItem SCTGroup SMDSnd (GroupChat gInfo) ci) + notAllowedError f = pure $ chatCmdError (Just user) ("feature not allowed " <> T.unpack (groupFeatureNameText f)) setupSndFileTransfer :: Group -> Int -> m (Maybe (FileInvitation, CIFile 'MDSnd, FileTransferMeta)) setupSndFileTransfer g@(Group gInfo _) n = forM file_ $ \file -> do (fileSize, fileMode) <- checkSndFile mc file $ fromIntegral n @@ -3533,20 +3536,21 @@ processAgentMessageConn user@User {userId} corrId agentConnId agentMessage = do e -> throwError e newGroupContentMessage :: GroupInfo -> GroupMember -> MsgContainer -> RcvMessage -> MsgMeta -> m () - newGroupContentMessage gInfo m@GroupMember {localDisplayName = c, memberId, memberRole} mc msg@RcvMessage {sharedMsgId_} msgMeta = do - -- TODO integrity message check - if isVoice content && not (groupFeatureAllowed SGFVoice gInfo) - then void $ newChatItem (CIRcvGroupFeatureRejected GFVoice) Nothing Nothing False - else do - -- check if message moderation event was received ahead of message - let timed_ = rcvGroupCITimed gInfo itemTTL - live = fromMaybe False live_ - withStore' (\db -> getCIModeration db user gInfo memberId sharedMsgId_) >>= \case - Just ciModeration -> do - applyModeration timed_ live ciModeration - withStore' $ \db -> deleteCIModeration db gInfo memberId sharedMsgId_ - Nothing -> createItem timed_ live + newGroupContentMessage gInfo m@GroupMember {localDisplayName = c, memberId, memberRole} mc msg@RcvMessage {sharedMsgId_} msgMeta + | isVoice content && not (groupFeatureAllowed SGFVoice gInfo) = rejected GFVoice + | not (isVoice content) && isJust fInv_ && not (groupFeatureAllowed SGFFiles gInfo) = rejected GFFiles + | otherwise = do + -- TODO integrity message check + -- check if message moderation event was received ahead of message + let timed_ = rcvGroupCITimed gInfo itemTTL + live = fromMaybe False live_ + withStore' (\db -> getCIModeration db user gInfo memberId sharedMsgId_) >>= \case + Just ciModeration -> do + applyModeration timed_ live ciModeration + withStore' $ \db -> deleteCIModeration db gInfo memberId sharedMsgId_ + Nothing -> createItem timed_ live where + rejected f = void $ newChatItem (CIRcvGroupFeatureRejected f) Nothing Nothing False ExtMsgContent content fInv_ itemTTL live_ = mcExtMsgContent mc applyModeration timed_ live CIModeration {moderatorMember = moderator@GroupMember {memberRole = moderatorRole}, createdByMsgId, moderatedAt} | moderatorRole < GRAdmin || moderatorRole < memberRole = @@ -5018,6 +5022,7 @@ chatCommandP = "/set voice #" *> (SetGroupFeature (AGF SGFVoice) <$> displayName <*> (A.space *> strP)), "/set voice @" *> (SetContactFeature (ACF SCFVoice) <$> displayName <*> optional (A.space *> strP)), "/set voice " *> (SetUserFeature (ACF SCFVoice) <$> strP), + "/set files #" *> (SetGroupFeature (AGF SGFFiles) <$> displayName <*> (A.space *> strP)), "/set calls @" *> (SetContactFeature (ACF SCFCalls) <$> displayName <*> optional (A.space *> strP)), "/set calls " *> (SetUserFeature (ACF SCFCalls) <$> strP), "/set delete #" *> (SetGroupFeature (AGF SGFFullDelete) <$> displayName <*> (A.space *> strP)), diff --git a/src/Simplex/Chat/Help.hs b/src/Simplex/Chat/Help.hs index 8997f150d6..c83e81a9ef 100644 --- a/src/Simplex/Chat/Help.hs +++ b/src/Simplex/Chat/Help.hs @@ -153,6 +153,7 @@ groupsHelpInfo = "", green "Group chat preferences:", indent <> highlight "/set voice # on/off " <> " - enable/disable voice messages", + -- indent <> highlight "/set files # on/off " <> " - enable/disable files and media (other than voice)", indent <> highlight "/set delete # on/off " <> " - enable/disable full message deletion", indent <> highlight "/set direct # on/off " <> " - enable/disable direct messages to other members", indent <> highlight "/set disappear # on