diff options
80 files changed, 1887 insertions, 449 deletions
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 9e9107ce3..be0dd4773 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -194,7 +194,7 @@ amd64: variables: &release-variables MIX_ENV: prod before_script: &before-release - - apt install cmake -y + - apt-get update && apt-get install -y cmake - echo "import Mix.Config" > config/prod.secret.exs - mix local.hex --force - mix local.rebar --force diff --git a/CHANGELOG.md b/CHANGELOG.md index c462833d7..d0eba0f79 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,17 +6,20 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). ## [unreleased] ### Changed + +- **Breaking:** The default descriptions on uploads are now empty. The old behavior (filename as default) can be configured, see the cheat sheet. - **Breaking:** Added the ObjectAgePolicy to the default set of MRFs. This will delist and strip the follower collection of any message received that is older than 7 days. This will stop users from seeing very old messages in the timelines. The messages can still be viewed on the user's page and in conversations. They also still trigger notifications. - **Breaking:** Elixir >=1.9 is now required (was >= 1.8) - **Breaking:** Configuration: `:auto_linker, :opts` moved to `:pleroma, Pleroma.Formatter`. Old config namespace is deprecated. +- **Breaking:** Configuration: `:instance, welcome_user_nickname` moved to `:welcome, :direct_message, :sender_nickname`, `:instance, :welcome_message` moved to `:welcome, :direct_message, :message`. Old config namespace is deprecated. +- **Breaking:** LDAP: Fallback to local database authentication has been removed for security reasons and lack of a mechanism to ensure the passwords are synchronized when LDAP passwords are updated. +- **Breaking** Changed defaults for `:restrict_unauthenticated` so that when `:instance, :public` is set to `false` then all `:restrict_unauthenticated` items be effectively set to `true`. If you'd like to allow unauthenticated access to specific API endpoints on a private instance, please explicitly set `:restrict_unauthenticated` to non-default value in `config/prod.secret.exs`. - In Conversations, return only direct messages as `last_status` - Using the `only_media` filter on timelines will now exclude reblog media - MFR policy to set global expiration for all local Create activities - OGP rich media parser merged with TwitterCard - Configuration: `:instance, rewrite_policy` moved to `:mrf, policies`, `:instance, :mrf_transparency` moved to `:mrf, :transparency`, `:instance, :mrf_transparency_exclusions` moved to `:mrf, :transparency_exclusions`. Old config namespace is deprecated. - Configuration: `:media_proxy, whitelist` format changed to host with scheme (e.g. `http://example.com` instead of `example.com`). Domain format is deprecated. -- **Breaking:** Configuration: `:instance, welcome_user_nickname` moved to `:welcome, :direct_message, :sender_nickname`, `:instance, :welcome_message` moved to `:welcome, :direct_message, :message`. Old config namespace is deprecated. -- **Breaking:** LDAP: Fallback to local database authentication has been removed for security reasons and lack of a mechanism to ensure the passwords are synchronized when LDAP passwords are updated. <details> <summary>API Changes</summary> @@ -24,34 +27,37 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). - **Breaking:** Pleroma API: The routes to update avatar, banner and background have been removed. - **Breaking:** Image description length is limited now. - **Breaking:** Emoji API: changed methods and renamed routes. +- **Breaking:** Notification Settings API for suppressing notifications has been simplified down to `block_from_strangers`. +- **Breaking:** Notification Settings API option for hiding push notification contents has been renamed to `hide_notification_contents`. - MastodonAPI: Allow removal of avatar, banner and background. - Streaming: Repeats of a user's posts will no longer be pushed to the user's stream. - Mastodon API: Added `pleroma.metadata.fields_limits` to /api/v1/instance - Mastodon API: On deletion, returns the original post text. - Mastodon API: Add `pleroma.unread_count` to the Marker entity. -- **Breaking:** Notification Settings API for suppressing notifications - has been simplified down to `block_from_strangers`. -- **Breaking:** Notification Settings API option for hiding push notification - contents has been renamed to `hide_notification_contents` - Mastodon API: Added `pleroma.metadata.post_formats` to /api/v1/instance - Mastodon API (legacy): Allow query parameters for `/api/v1/domain_blocks`, e.g. `/api/v1/domain_blocks?domain=badposters.zone` - Mastodon API: Make notifications about statuses from muted users and threads read automatically - Pleroma API: `/api/pleroma/captcha` responses now include `seconds_valid` with an integer value. + </details> <details> <summary>Admin API Changes</summary> +- **Breaking** Changed relay `/api/pleroma/admin/relay` endpoints response format. - Status visibility stats: now can return stats per instance. - - Mix task to refresh counter cache (`mix pleroma.refresh_counter_cache`) + </details> ### Removed + - **Breaking:** removed `with_move` parameter from notifications timeline. ### Added +- Frontends: Add mix task to install frontends. +- Frontends: Add configurable frontends for primary and admin fe. - Configuration: Added a blacklist for email servers. - Chats: Added `accepts_chat_messages` field to user, exposed in APIs and federation. - Chats: Added support for federated chats. For details, see the docs. @@ -106,6 +112,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). - Fix edge case where MediaProxy truncates media, usually caused when Caddy is serving content for the other Federated instance. - Emoji Packs could not be listed when instance was set to `public: false` - Fix whole_word always returning false on filter get requests +- Migrations not working on OTP releases if the database was connected over ssl +- Fix relay following ## [Unreleased (patch)] diff --git a/config/config.exs b/config/config.exs index eb85a6ed4..246712b9f 100644 --- a/config/config.exs +++ b/config/config.exs @@ -72,7 +72,8 @@ config :pleroma, Pleroma.Upload, pool: :upload ] ], - filename_display_max_length: 30 + filename_display_max_length: 30, + default_description: nil config :pleroma, Pleroma.Uploaders.Local, uploads: "uploads" @@ -670,7 +671,50 @@ config :pleroma, :static_fe, enabled: false # With no frontend configuration, the bundled files from the `static` directory will # be used. # -# config :pleroma, :frontends, primary: %{"name" => "pleroma", "ref" => "develop"} +# config :pleroma, :frontends, +# primary: %{"name" => "pleroma-fe", "ref" => "develop"}, +# admin: %{"name" => "admin-fe", "ref" => "stable"}, +# available: %{...} + +config :pleroma, :frontends, + available: %{ + "kenoma" => %{ + "name" => "kenoma", + "git" => "https://git.pleroma.social/lambadalambda/kenoma", + "build_url" => + "https://git.pleroma.social/lambadalambda/kenoma/-/jobs/artifacts/${ref}/download?job=build", + "ref" => "master" + }, + "pleroma-fe" => %{ + "name" => "pleroma-fe", + "git" => "https://git.pleroma.social/pleroma/pleroma-fe", + "build_url" => + "https://git.pleroma.social/pleroma/pleroma-fe/-/jobs/artifacts/${ref}/download?job=build", + "ref" => "develop" + }, + "fedi-fe" => %{ + "name" => "fedi-fe", + "git" => "https://git.pleroma.social/pleroma/fedi-fe", + "build_url" => + "https://git.pleroma.social/pleroma/fedi-fe/-/jobs/artifacts/${ref}/download?job=build", + "ref" => "master" + }, + "admin-fe" => %{ + "name" => "admin-fe", + "git" => "https://git.pleroma.social/pleroma/admin-fe", + "build_url" => + "https://git.pleroma.social/pleroma/admin-fe/-/jobs/artifacts/${ref}/download?job=build", + "ref" => "develop" + }, + "soapbox-fe" => %{ + "name" => "soapbox-fe", + "git" => "https://gitlab.com/soapbox-pub/soapbox-fe", + "build_url" => + "https://gitlab.com/soapbox-pub/soapbox-fe/-/jobs/artifacts/${ref}/download?job=build-production", + "ref" => "v1.0.0", + "build_dir" => "static" + } + } config :pleroma, :web_cache_ttl, activity_pub: nil, @@ -725,10 +769,12 @@ config :pleroma, :hackney_pools, timeout: 300_000 ] +private_instance? = :if_instance_is_private + config :pleroma, :restrict_unauthenticated, - timelines: %{local: false, federated: false}, - profiles: %{local: false, remote: false}, - activities: %{local: false, remote: false} + timelines: %{local: private_instance?, federated: private_instance?}, + profiles: %{local: private_instance?, remote: private_instance?}, + activities: %{local: private_instance?, remote: private_instance?} config :pleroma, Pleroma.Web.ApiSpec.CastAndValidate, strict: false diff --git a/config/description.exs b/config/description.exs index 7734ff7a1..29a657333 100644 --- a/config/description.exs +++ b/config/description.exs @@ -12,6 +12,55 @@ websocket_config = [ compress: false ] +installed_frontend_options = [ + %{ + key: "name", + label: "Name", + type: :string, + description: + "Name of the installed frontend. Valid config must include both `Name` and `Reference` values." + }, + %{ + key: "ref", + label: "Reference", + type: :string, + description: + "Reference of the installed frontend to be used. Valid config must include both `Name` and `Reference` values." + } +] + +frontend_options = [ + %{ + key: "name", + label: "Name", + type: :string, + description: "Name of the frontend." + }, + %{ + key: "ref", + label: "Reference", + type: :string, + description: "Reference of the frontend to be used." + }, + %{ + key: "git", + type: :string, + description: "URL of the git repository of the frontend" + }, + %{ + key: "build_url", + type: :string, + description: + "Either an url to a zip file containing the frontend or a template to build it by inserting the `ref`. The string `${ref}` will be replaced by the configured `ref`.", + example: "https://some.url/builds/${ref}.zip" + }, + %{ + key: "build_dir", + type: :string, + description: "The directory inside the zip file " + } +] + config :pleroma, :config_description, [ %{ group: :pleroma, @@ -1810,12 +1859,12 @@ config :pleroma, :config_description, [ %{ key: :enabled, type: :boolean, - description: "Enables invalidate media cache" + description: "Enables media cache object invalidation." }, %{ key: :provider, type: :module, - description: "Module which will be used to cache purge.", + description: "Module which will be used to purge objects from the cache.", suggestions: [ Pleroma.Web.MediaProxy.Invalidation.Script, Pleroma.Web.MediaProxy.Invalidation.Http @@ -3553,21 +3602,40 @@ config :pleroma, :config_description, [ key: :primary, type: :map, description: "Primary frontend, the one that is served for all pages by default", + children: installed_frontend_options + }, + %{ + key: :admin, + type: :map, + description: "Admin frontend", + children: installed_frontend_options + }, + %{ + key: :available, + type: :map, + description: + "A map containing available frontends and parameters for their installation.", children: [ - %{ - key: "name", - label: "Name", - type: :string, - description: - "Name of the installed primary frontend. Valid config must include both `Name` and `Reference` values." - }, - %{ - key: "ref", - label: "Reference", - type: :string, - description: - "Reference of the installed primary frontend to be used. Valid config must include both `Name` and `Reference` values." - } + frontend_options + ] + } + ] + }, + %{ + group: :pleroma, + key: Pleroma.Web.Preload, + type: :group, + description: "Preload-related settings", + children: [ + %{ + key: :providers, + type: {:list, :module}, + description: "List of preload providers to enable", + suggestions: [ + Pleroma.Web.Preload.Providers.Instance, + Pleroma.Web.Preload.Providers.User, + Pleroma.Web.Preload.Providers.Timelines, + Pleroma.Web.Preload.Providers.StatusNet ] } ] diff --git a/config/test.exs b/config/test.exs index 413c7f0b9..f0358e384 100644 --- a/config/test.exs +++ b/config/test.exs @@ -21,7 +21,10 @@ config :logger, :console, config :pleroma, :auth, oauth_consumer_strategies: [] -config :pleroma, Pleroma.Upload, filters: [], link_name: false +config :pleroma, Pleroma.Upload, + filters: [], + link_name: false, + default_description: :filename config :pleroma, Pleroma.Uploaders.Local, uploads: "test/uploads" diff --git a/docs/API/admin_api.md b/docs/API/admin_api.md index 4b143e4ee..c0ea074f0 100644 --- a/docs/API/admin_api.md +++ b/docs/API/admin_api.md @@ -313,31 +313,53 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret - On failure: `Not found` - On success: JSON array of user's latest statuses +## `GET /api/pleroma/admin/relay` + +### List Relays + +Params: none +Response: + +* On success: JSON array of relays + +```json +[ + {"actor": "https://example.com/relay", "followed_back": true}, + {"actor": "https://example2.com/relay", "followed_back": false} +] +``` + ## `POST /api/pleroma/admin/relay` ### Follow a Relay -- Params: - - `relay_url` -- Response: - - On success: URL of the followed relay +Params: + +* `relay_url` + +Response: + +* On success: relay json object + +```json +{"actor": "https://example.com/relay", "followed_back": true} +``` ## `DELETE /api/pleroma/admin/relay` ### Unfollow a Relay -- Params: - - `relay_url` -- Response: - - On success: URL of the unfollowed relay +Params: -## `GET /api/pleroma/admin/relay` +* `relay_url` -### List Relays +Response: -- Params: none -- Response: - - On success: JSON array of relays +* On success: URL of the unfollowed relay + +```json +{"https://example.com/relay"} +``` ## `POST /api/pleroma/admin/users/invite_token` @@ -1266,11 +1288,14 @@ Loads json generated from `config/descriptions.exs`. - Params: - *optional* `page`: **integer** page number - *optional* `page_size`: **integer** number of log entries per page (default is `50`) +- *optional* `query`: **string** search term - Response: ``` json { + "page_size": integer, + "count": integer, "urls": [ "http://example.com/media/a688346.jpg", "http://example.com/media/fb1f4d.jpg" @@ -1290,12 +1315,7 @@ Loads json generated from `config/descriptions.exs`. - Response: ``` json -{ - "urls": [ - "http://example.com/media/a688346.jpg", - "http://example.com/media/fb1f4d.jpg" - ] -} +{ } ``` @@ -1311,11 +1331,6 @@ Loads json generated from `config/descriptions.exs`. - Response: ``` json -{ - "urls": [ - "http://example.com/media/a688346.jpg", - "http://example.com/media/fb1f4d.jpg" - ] -} +{ } ``` diff --git a/docs/administration/CLI_tasks/frontend.md b/docs/administration/CLI_tasks/frontend.md new file mode 100644 index 000000000..7d1c1e937 --- /dev/null +++ b/docs/administration/CLI_tasks/frontend.md @@ -0,0 +1,69 @@ +# Managing frontends + +`mix pleroma.frontend install <frontend> [--ref <ref>] [--file <file>] [--build-url <build-url>] [--path <path>] [--build-dir <build-dir>]` + +Frontend can be installed either from local zip file, or automatically downloaded from the web. + +You can give all the options directly on the command like, but missing information will be filled out by looking at the data configured under `frontends.available` in the config files. + +Currently known `<frontend>` values are: +- [admin-fe](https://git.pleroma.social/pleroma/admin-fe) +- [kenoma](http://git.pleroma.social/lambadalambda/kenoma) +- [pleroma-fe](http://git.pleroma.social/pleroma/pleroma-fe) +- [fedi-fe](https://git.pleroma.social/pleroma/fedi-fe) +- [soapbox-fe](https://gitlab.com/soapbox-pub/soapbox-fe) + +You can still install frontends that are not configured, see below. + +## Example installations for a known frontend + +For a frontend configured under the `available` key, it's enough to install it by name. + +```sh tab="OTP" +./bin/pleroma_ctl frontend install pleroma +``` + +```sh tab="From Source" +mix pleroma.frontend install pleroma +``` + +This will download the latest build for the the pre-configured `ref` and install it. It can then be configured as the one of the served frontends in the config file (see `primary` or `admin`). + +You can override any of the details. To install a pleroma build from a different url, you could do this: + +```sh tab="OPT" +./bin/pleroma_ctl frontend install pleroma --ref 2hu_edition --build-url https://example.org/raymoo.zip +``` + +```sh tab="From Source" +mix pleroma.frontend install pleroma --ref 2hu_edition --build-url https://example.org/raymoo.zip +``` + +Similarly, you can also install from a local zip file. + +```sh tab="OTP" +./bin/pleroma_ctl frontend install pleroma --ref mybuild --file ~/Downloads/doomfe.zip +``` + +```sh tab="From Source" +mix pleroma.frontend install pleroma --ref mybuild --file ~/Downloads/doomfe.zip +``` + +The resulting frontend will always be installed into a folder of this template: `${instance_static}/frontends/${name}/${ref}` + +Careful: This folder will be completely replaced on installation + +## Example installation for an unknown frontend + +The installation process is the same, but you will have to give all the needed options on the commond line. For example: + +```sh tab="OTP" +./bin/pleroma_ctl frontend install gensokyo --ref master --build-url https://gensokyo.2hu/builds/marisa.zip +``` + +```sh tab="From Source" +mix pleroma.frontend install gensokyo --ref master --build-url https://gensokyo.2hu/builds/marisa.zip +``` + +If you don't have a zip file but just want to install a frontend from a local path, you can simply copy the files over a folder of this template: `${instance_static}/frontends/${name}/${ref}` + diff --git a/docs/clients.md b/docs/clients.md index 2a42c659f..f84295b1f 100644 --- a/docs/clients.md +++ b/docs/clients.md @@ -6,11 +6,11 @@ Feel free to contact us to be added to this list! ### Roma for Desktop - Homepage: <https://www.pleroma.com/#desktopApp> - Source Code: <https://github.com/roma-apps/roma-desktop> -- Platforms: Windows, Mac, (Linux?) +- Platforms: Windows, Mac, Linux - Features: Streaming Ready ### Social -- Source Code: <https://gitlab.gnome.org/BrainBlasted/Social> +- Source Code: <https://gitlab.gnome.org/World/Social> - Contact: [@brainblasted@social.libre.fi](https://social.libre.fi/users/brainblasted) - Platforms: Linux (GNOME) - Note(2019-01-28): Not at a pre-alpha stage yet @@ -35,7 +35,7 @@ Feel free to contact us to be added to this list! - Source Code: <https://framagit.org/tom79/fedilab/> - Contact: [@fedilab@framapiaf.org](https://framapiaf.org/users/fedilab) - Platforms: Android -- Features: Streaming Ready, Moderation, Text Formatting +- Features: Streaming Ready, Moderation, Text Formatting ### Kyclos - Source Code: <https://git.pleroma.social/pleroma/harbour-kyclos> @@ -48,16 +48,9 @@ Feel free to contact us to be added to this list! - Platforms: Android - Features: No Streaming, Emoji Reactions, Text Formatting, FE Stickers -### Nekonium -- Homepage: [F-Droid Repository](https://repo.gdgd.jp.net/), [Google Play](https://play.google.com/store/apps/details?id=com.apps.nekonium), [Amazon](https://www.amazon.co.jp/dp/B076FXPRBC/) -- Source: <https://gogs.gdgd.jp.net/lin/nekonium> -- Contact: [@lin@pleroma.gdgd.jp.net](https://pleroma.gdgd.jp.net/users/lin) -- Platforms: Android -- Features: Streaming Ready - ### Fedi - Homepage: <https://www.fediapp.com/> -- Source Code: Proprietary, but free +- Source Code: Proprietary, but gratis - Platforms: iOS, Android - Features: Pleroma-specific features like Reactions @@ -70,9 +63,9 @@ Feel free to contact us to be added to this list! ### Twidere - Homepage: <https://twidere.mariotaku.org/> -- Source Code: <https://github.com/TwidereProject/Twidere-Android/>, <https://github.com/TwidereProject/Twidere-iOS/> +- Source Code: <https://github.com/TwidereProject/Twidere-Android/> - Contact: <me@mariotaku.org> -- Platform: Android, iOS +- Platform: Android - Features: No Streaming ### Indigenous @@ -89,11 +82,6 @@ Feel free to contact us to be added to this list! - Contact: [@gcupc@glitch.social](https://glitch.social/users/gcupc) - Features: No Streaming -### Feather -- Source Code: <https://github.com/kaniini/feather> -- Contact: [@kaniini@pleroma.site](https://pleroma.site/kaniini) -- Features: No Streaming - ### Halcyon - Source Code: <https://notabug.org/halcyon-suite/halcyon> - Contact: [@halcyon@social.csswg.org](https://social.csswg.org/users/halcyon) @@ -107,6 +95,15 @@ Feel free to contact us to be added to this list! - Features: No Streaming ### Sengi +- Homepage: <https://nicolasconstant.github.io/sengi/> - Source Code: <https://github.com/NicolasConstant/sengi> - Contact: [@sengi_app@mastodon.social](https://mastodon.social/users/sengi_app) -- Note(2019-01-28): The development is currently in a early stage. + +### DashFE +- Source Code: <https://notabug.org/daisuke/DashboardFE> +- Contact: [@dashfe@stereophonic.space](https://stereophonic.space/users/dashfe) + +### BloatFE +- Source Code: <https://git.freesoftwareextremist.com/bloat/> +- Contact: [@r@freesoftwareextremist.com](https://freesoftwareextremist.com/users/r) +- Features: Does not requires JavaScript diff --git a/docs/configuration/cheatsheet.md b/docs/configuration/cheatsheet.md index e5742bc3a..a09d6b6b2 100644 --- a/docs/configuration/cheatsheet.md +++ b/docs/configuration/cheatsheet.md @@ -38,8 +38,8 @@ To add configuration to your config file, you can copy it from the base config. * `federation_incoming_replies_max_depth`: Max. depth of reply-to activities fetching on incoming federation, to prevent out-of-memory situations while fetching very long threads. If set to `nil`, threads of any depth will be fetched. Lower this value if you experience out-of-memory crashes. * `federation_reachability_timeout_days`: Timeout (in days) of each external federation target being unreachable prior to pausing federating to it. * `allow_relay`: Enable Pleroma’s Relay, which makes it possible to follow a whole instance. -* `public`: Makes the client API in authenticated mode-only except for user-profiles. Useful for disabling the Local Timeline and The Whole Known Network. See also: `restrict_unauthenticated`. -* `quarantined_instances`: List of ActivityPub instances where private(DMs, followers-only) activities will not be send. +* `public`: Makes the client API in authenticated mode-only except for user-profiles. Useful for disabling the Local Timeline and The Whole Known Network. Note that there is a dependent setting restricting or allowing unauthenticated access to specific resources, see `restrict_unauthenticated` for more details. +* `quarantined_instances`: List of ActivityPub instances where private (DMs, followers-only) activities will not be send. * `managed_config`: Whenether the config for pleroma-fe is configured in [:frontend_configurations](#frontend_configurations) or in ``static/config.json``. * `allowed_post_formats`: MIME-type list of formats allowed to be posted (transformed into HTML). * `extended_nickname_format`: Set to `true` to use extended local nicknames format (allows underscores/dashes). This will break federation with @@ -552,6 +552,7 @@ the source code is here: [kocaptcha](https://github.com/koto-bank/kocaptcha). Th * `proxy_remote`: If you're using a remote uploader, Pleroma will proxy media requests instead of redirecting to it. * `proxy_opts`: Proxy options, see `Pleroma.ReverseProxy` documentation. * `filename_display_max_length`: Set max length of a filename to display. 0 = no limit. Default: 30. +* `default_description`: Sets which default description an image has if none is set explicitly. Options: nil (default) - Don't set a default, :filename - use the filename of the file, a string (e.g. "attachment") - Use this string !!! warning `strip_exif` has been replaced by `Pleroma.Upload.Filter.Mogrify`. @@ -1051,6 +1052,8 @@ Restrict access for unauthenticated users to timelines (public and federated), u * `local` * `remote` +Note: when `:instance, :public` is set to `false`, all `:restrict_unauthenticated` items be effectively set to `true` by default. If you'd like to allow unauthenticated access to specific API endpoints on a private instance, please explicitly set `:restrict_unauthenticated` to non-default value in `config/prod.secret.exs`. + Note: setting `restrict_unauthenticated/timelines/local` to `true` has no practical sense if `restrict_unauthenticated/timelines/federated` is set to `false` (since local public activities will still be delivered to unauthenticated users as part of federated timeline). ## Pleroma.Web.ApiSpec.CastAndValidate @@ -1067,11 +1070,11 @@ Control favicons for instances. Frontends in Pleroma are swappable - you can specify which one to use here. -For now, you can set a frontend with the key `primary` and the options of `name` and `ref`. This will then make Pleroma serve the frontend from a folder constructed by concatenating the instance static path, `frontends` and the name and ref. +You can set a frontends for the key `primary` and `admin` and the options of `name` and `ref`. This will then make Pleroma serve the frontend from a folder constructed by concatenating the instance static path, `frontends` and the name and ref. -The key `primary` refers to the frontend that will be served by default for general requests. In the future, other frontends like the admin frontend will also be configurable here. +The key `primary` refers to the frontend that will be served by default for general requests. The key `admin` refers to the frontend that will be served at the `/pleroma/admin` path. -If you don't set anything here, the bundled frontend will be used. +If you don't set anything here, the bundled frontends will be used. Example: @@ -1080,6 +1083,10 @@ config :pleroma, :frontends, primary: %{ "name" => "pleroma", "ref" => "stable" + }, + admin: %{ + "name" => "admin", + "ref" => "develop" } ``` diff --git a/docs/installation/freebsd_en.md b/docs/installation/freebsd_en.md new file mode 100644 index 000000000..130d68766 --- /dev/null +++ b/docs/installation/freebsd_en.md @@ -0,0 +1,210 @@ +# Installing on FreeBSD + +This document was written for FreeBSD 12.1, but should be work on future releases. + +## Required software + +This assumes the target system has `pkg(8)`. + +``` +# pkg install elixir postgresql12-server postgresql12-client postgresql12-contrib git-lite sudo nginx gmake acme.sh +``` + +Copy the rc.d scripts to the right directory: + +Setup the required services to automatically start at boot, using `sysrc(8)`. + +``` +# sysrc nginx_enable=YES +# sysrc postgresql_enable=YES +``` + +## Initialize postgres + +``` +# service postgresql initdb +# service postgresql start +``` + +## Configuring Pleroma + +Create a user for Pleroma: + +``` +# pw add user pleroma -m +# echo 'export LC_ALL="en_US.UTF-8"' >> /home/pleroma/.profile +# su -l pleroma +``` + +Clone the repository: + +``` +$ cd $HOME # Should be the same as /home/pleroma +$ git clone -b stable https://git.pleroma.social/pleroma/pleroma.git +``` + +Configure Pleroma. Note that you need a domain name at this point: + +``` +$ cd /home/pleroma/pleroma +$ mix deps.get # Enter "y" when asked to install Hex +$ mix pleroma.instance gen # You will be asked a few questions here. +$ cp config/generated_config.exs config/prod.secret.exs +``` + +Since Postgres is configured, we can now initialize the database. There should +now be a file in `config/setup_db.psql` that makes this easier. Edit it, and +*change the password* to a password of your choice. Make sure it is secure, since +it'll be protecting your database. As root, you can now initialize the database: + +``` +# cd /home/pleroma/pleroma +# sudo -Hu postgres -g postgres psql -f config/setup_db.psql +``` + +Postgres allows connections from all users without a password by default. To +fix this, edit `/var/db/postgres/data12/pg_hba.conf`. Change every `trust` to +`password`. + +Once this is done, restart Postgres with: +``` +# service postgresql restart +``` + +Run the database migrations. + +Back as the pleroma user, run the following to implement any database migrations. + +``` +# su -l pleroma +$ cd /home/pleroma/pleroma +$ MIX_ENV=prod mix ecto.migrate +``` + +You will need to do this whenever you update with `git pull`: + +## Configuring acme.sh + +We'll be using acme.sh in Stateless Mode for TLS certificate renewal. + +First, as root, allow the user `acme` to have access to the acme log file, as follows: + +``` +# touch /var/log/acme.sh.log +# chown acme:acme /var/log/acme.sh.log +# chmod 600 /var/log/acme.sh.log +``` + +Next, obtain your account fingerprint: + +``` +# sudo -Hu acme -g acme acme.sh --register-account +``` + +You need to add the following to your nginx configuration for the server +running on port 80: + +``` + location ~ ^/\.well-known/acme-challenge/([-_a-zA-Z0-9]+)$ { + default_type text/plain; + return 200 "$1.6fXAG9VyG0IahirPEU2ZerUtItW2DHzDzD9wZaEKpqd"; + } +``` + +Replace the string after after `$1.` with your fingerprint. + +Start nginx: + +``` +# service nginx start +``` + +It should now be possible to issue a cert (replace `example.com` +with your domain name): + +``` +# sudo -Hu acme -g acme acme.sh --issue -d example.com --stateless +``` + +Let's add auto-renewal to `/etc/crontab` +(replace `example.com` with your domain): + +``` +/usr/local/bin/sudo -Hu acme -g acme /usr/local/sbin/acme.sh -r -d example.com --stateless +``` + +### Configuring nginx + +FreeBSD's default nginx configuration does not contain an include directive, which is +typically used for multiple sites. Therefore, you will need to first create the required +directory as follows: + + +``` +# mkdir -p /usr/local/etc/nginx/sites-available +``` + +Next, add an `include` directive to `/usr/local/etc/nginx/nginx.conf`, within the `http {}` +block, as follows: + + +``` +http { +... + include /usr/local/etc/nginx/sites-available/*; +} +``` + +As root, copy `/home/pleroma/pleroma/installation/pleroma.nginx` to +`/usr/local/etc/nginx/sites-available/pleroma.nginx`. + +Edit the defaults of `/usr/local/etc/nginx/sites-available/pleroma.nginx`: + +* Change `ssl_trusted_certificate` to `/var/db/acme/certs/example.tld/example.tld.cer`. +* Change `ssl_certificate` to `/var/db/acme/certs/example.tld/fullchain.cer`. +* Change `ssl_certificate_key` to `/var/db/acme/certs/example.tld/example.tld.key`. +* Change all references of `example.tld` to your instance's domain name. + +## Creating a startup script for Pleroma + +Pleroma will need to compile when it initially starts, which typically takes a longer +period of time. Therefore, it is good practice to initially run pleroma from the +command-line before utilizing the rc.d script. That is done as follows: + +``` +# su -l pleroma +$ cd $HOME/pleroma +$ MIX_ENV=prod mix phx.server +``` + +Copy the startup script to the correct location and make sure it's executable: + +``` +# cp /home/pleroma/pleroma/installation/freebsd/rc.d/pleroma /usr/local/etc/rc.d/pleroma +# chmod +x /usr/local/etc/rc.d/pleroma +``` + +Update the `/etc/rc.conf` and start pleroma with the following commands: + +``` +# sysrc pleroma_enable=YES +# service pleroma start +``` + +#### Create your first user + +If your instance is up and running, you can create your first user with administrative rights with the following task: + +```shell +sudo -Hu pleroma MIX_ENV=prod mix pleroma.user new <username> <your@emailaddress> --admin +``` +## Conclusion + +Restart nginx with `# service nginx restart` and you should be up and running. + +Make sure your time is in sync, or other instances will receive your posts with +incorrect timestamps. You should have ntpd running. + +## Questions + +Questions about the installation or didn’t it work as it should be, ask in [#pleroma:matrix.org](https://matrix.heldscal.la/#/room/#freenode_#pleroma:matrix.org) or IRC Channel **#pleroma** on **Freenode**. diff --git a/installation/freebsd/rc.d/pleroma b/installation/freebsd/rc.d/pleroma new file mode 100755 index 000000000..f62aef18d --- /dev/null +++ b/installation/freebsd/rc.d/pleroma @@ -0,0 +1,27 @@ +#!/bin/sh +# $FreeBSD$ +# PROVIDE: pleroma +# REQUIRE: DAEMON postgresql +# KEYWORD: shutdown + +# sudo -u pleroma MIX_ENV=prod elixir --erl \"-detached\" -S mix phx.server + +. /etc/rc.subr + +name=pleroma +rcvar=pleroma_enable + +desc="Pleroma Social Media Platform" + +load_rc_config ${name} + +: ${pleroma_user:=pleroma} +: ${pleroma_home:=$(getent passwd ${pleroma_user} | awk -F: '{print $6}')} +: ${pleroma_chdir:="${pleroma_home}/pleroma"} +: ${pleroma_env:="HOME=${pleroma_home} MIX_ENV=prod"} + +command=/usr/local/bin/elixir +command_args="--erl \"-detached\" -S /usr/local/bin/mix phx.server" +procname="*beam.smp" + +run_rc_command "$1" diff --git a/lib/mix/pleroma.ex b/lib/mix/pleroma.ex index 074492a46..fe9b0d16c 100644 --- a/lib/mix/pleroma.ex +++ b/lib/mix/pleroma.ex @@ -14,7 +14,7 @@ defmodule Mix.Pleroma do :swoosh, :timex ] - @cachex_children ["object", "user"] + @cachex_children ["object", "user", "scrubber"] @doc "Common functions to be reused in mix tasks" def start_pleroma do Pleroma.Config.Holder.save_default() diff --git a/lib/mix/tasks/pleroma/ecto/migrate.ex b/lib/mix/tasks/pleroma/ecto/migrate.ex index bc8ed29fb..e903bd171 100644 --- a/lib/mix/tasks/pleroma/ecto/migrate.ex +++ b/lib/mix/tasks/pleroma/ecto/migrate.ex @@ -41,6 +41,10 @@ defmodule Mix.Tasks.Pleroma.Ecto.Migrate do load_pleroma() {opts, _} = OptionParser.parse!(args, strict: @switches, aliases: @aliases) + if Application.get_env(:pleroma, Pleroma.Repo)[:ssl] do + Application.ensure_all_started(:ssl) + end + opts = if opts[:to] || opts[:step] || opts[:all], do: opts, diff --git a/lib/mix/tasks/pleroma/ecto/rollback.ex b/lib/mix/tasks/pleroma/ecto/rollback.ex index f43bd0b98..3dba952cb 100644 --- a/lib/mix/tasks/pleroma/ecto/rollback.ex +++ b/lib/mix/tasks/pleroma/ecto/rollback.ex @@ -40,6 +40,10 @@ defmodule Mix.Tasks.Pleroma.Ecto.Rollback do load_pleroma() {opts, _} = OptionParser.parse!(args, strict: @switches, aliases: @aliases) + if Application.get_env(:pleroma, Pleroma.Repo)[:ssl] do + Application.ensure_all_started(:ssl) + end + opts = if opts[:to] || opts[:step] || opts[:all], do: opts, diff --git a/lib/mix/tasks/pleroma/emoji.ex b/lib/mix/tasks/pleroma/emoji.ex index f4eaeac98..8f52ee98d 100644 --- a/lib/mix/tasks/pleroma/emoji.ex +++ b/lib/mix/tasks/pleroma/emoji.ex @@ -15,7 +15,7 @@ defmodule Mix.Tasks.Pleroma.Emoji do {options, [], []} = parse_global_opts(args) url_or_path = options[:manifest] || default_manifest() - manifest = fetch_and_decode(url_or_path) + manifest = fetch_and_decode!(url_or_path) Enum.each(manifest, fn {name, info} -> to_print = [ @@ -42,7 +42,7 @@ defmodule Mix.Tasks.Pleroma.Emoji do url_or_path = options[:manifest] || default_manifest() - manifest = fetch_and_decode(url_or_path) + manifest = fetch_and_decode!(url_or_path) for pack_name <- pack_names do if Map.has_key?(manifest, pack_name) do @@ -92,7 +92,7 @@ defmodule Mix.Tasks.Pleroma.Emoji do ]) ) - files = fetch_and_decode(files_loc) + files = fetch_and_decode!(files_loc) IO.puts(IO.ANSI.format(["Unpacking ", :bright, pack_name])) @@ -243,9 +243,11 @@ defmodule Mix.Tasks.Pleroma.Emoji do IO.puts("Emoji packs have been reloaded.") end - defp fetch_and_decode(from) do + defp fetch_and_decode!(from) do with {:ok, json} <- fetch(from) do Jason.decode!(json) + else + {:error, error} -> raise "#{from} cannot be fetched. Error: #{error} occur." end end diff --git a/lib/mix/tasks/pleroma/frontend.ex b/lib/mix/tasks/pleroma/frontend.ex new file mode 100644 index 000000000..2adbf8d72 --- /dev/null +++ b/lib/mix/tasks/pleroma/frontend.ex @@ -0,0 +1,140 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Mix.Tasks.Pleroma.Frontend do + use Mix.Task + + import Mix.Pleroma + + @shortdoc "Manages bundled Pleroma frontends" + + @moduledoc File.read!("docs/administration/CLI_tasks/frontend.md") + + def run(["install", "none" | _args]) do + shell_info("Skipping frontend installation because none was requested") + "none" + end + + def run(["install", frontend | args]) do + log_level = Logger.level() + Logger.configure(level: :warn) + start_pleroma() + + {options, [], []} = + OptionParser.parse( + args, + strict: [ + ref: :string, + static_dir: :string, + build_url: :string, + build_dir: :string, + file: :string + ] + ) + + instance_static_dir = + with nil <- options[:static_dir] do + Pleroma.Config.get!([:instance, :static_dir]) + end + + cmd_frontend_info = %{ + "name" => frontend, + "ref" => options[:ref], + "build_url" => options[:build_url], + "build_dir" => options[:build_dir] + } + + config_frontend_info = Pleroma.Config.get([:frontends, :available, frontend], %{}) + + frontend_info = + Map.merge(config_frontend_info, cmd_frontend_info, fn _key, config, cmd -> + # This only overrides things that are actually set + cmd || config + end) + + ref = frontend_info["ref"] + + unless ref do + raise "No ref given or configured" + end + + dest = + Path.join([ + instance_static_dir, + "frontends", + frontend, + ref + ]) + + fe_label = "#{frontend} (#{ref})" + + tmp_dir = Path.join(dest, "tmp") + + with {_, :ok} <- + {:download_or_unzip, download_or_unzip(frontend_info, tmp_dir, options[:file])}, + shell_info("Installing #{fe_label} to #{dest}"), + :ok <- install_frontend(frontend_info, tmp_dir, dest) do + File.rm_rf!(tmp_dir) + shell_info("Frontend #{fe_label} installed to #{dest}") + + Logger.configure(level: log_level) + else + {:download_or_unzip, _} -> + shell_info("Could not download or unzip the frontend") + + _e -> + shell_info("Could not install the frontend") + end + end + + defp download_or_unzip(frontend_info, temp_dir, file) do + if file do + with {:ok, zip} <- File.read(Path.expand(file)) do + unzip(zip, temp_dir) + end + else + download_build(frontend_info, temp_dir) + end + end + + def unzip(zip, dest) do + with {:ok, unzipped} <- :zip.unzip(zip, [:memory]) do + File.rm_rf!(dest) + File.mkdir_p!(dest) + + Enum.each(unzipped, fn {filename, data} -> + path = filename + + new_file_path = Path.join(dest, path) + + new_file_path + |> Path.dirname() + |> File.mkdir_p!() + + File.write!(new_file_path, data) + end) + + :ok + end + end + + defp download_build(frontend_info, dest) do + shell_info("Downloading pre-built bundle for #{frontend_info["name"]}") + url = String.replace(frontend_info["build_url"], "${ref}", frontend_info["ref"]) + + with {:ok, %{status: 200, body: zip_body}} <- + Pleroma.HTTP.get(url, [], timeout: 120_000, recv_timeout: 120_000) do + unzip(zip_body, dest) + else + e -> {:error, e} + end + end + + defp install_frontend(frontend_info, source, dest) do + from = frontend_info["build_dir"] || "dist" + File.mkdir_p!(dest) + File.cp_r!(Path.join([source, from]), dest) + :ok + end +end diff --git a/lib/mix/tasks/pleroma/relay.ex b/lib/mix/tasks/pleroma/relay.ex index c3312507e..a6d8d6c1c 100644 --- a/lib/mix/tasks/pleroma/relay.ex +++ b/lib/mix/tasks/pleroma/relay.ex @@ -35,10 +35,16 @@ defmodule Mix.Tasks.Pleroma.Relay do def run(["list"]) do start_pleroma() - with {:ok, list} <- Relay.list(true) do - list |> Enum.each(&shell_info(&1)) + with {:ok, list} <- Relay.list() do + Enum.each(list, &print_relay_url/1) else {:error, e} -> shell_error("Error while fetching relay subscription list: #{inspect(e)}") end end + + defp print_relay_url(%{followed_back: false} = relay) do + shell_info("#{relay.actor} - no Accept received (relay didn't follow back)") + end + + defp print_relay_url(relay), do: shell_info(relay.actor) end diff --git a/lib/pleroma/config.ex b/lib/pleroma/config.ex index a8329cc1e..97f877595 100644 --- a/lib/pleroma/config.ex +++ b/lib/pleroma/config.ex @@ -81,6 +81,16 @@ defmodule Pleroma.Config do Application.delete_env(:pleroma, key) end + def restrict_unauthenticated_access?(resource, kind) do + setting = get([:restrict_unauthenticated, resource, kind]) + + if setting in [nil, :if_instance_is_private] do + !get!([:instance, :public]) + else + setting + end + end + def oauth_consumer_strategies, do: get([:auth, :oauth_consumer_strategies], []) def oauth_consumer_enabled?, do: oauth_consumer_strategies() != [] diff --git a/lib/pleroma/emails/user_email.ex b/lib/pleroma/emails/user_email.ex index 313533859..1d8c72ae9 100644 --- a/lib/pleroma/emails/user_email.ex +++ b/lib/pleroma/emails/user_email.ex @@ -107,25 +107,34 @@ defmodule Pleroma.Emails.UserEmail do |> Enum.filter(&(&1.activity.data["type"] == "Create")) |> Enum.map(fn notification -> object = Pleroma.Object.normalize(notification.activity) - object = update_in(object.data["content"], &format_links/1) - %{ - data: notification, - object: object, - from: User.get_by_ap_id(notification.activity.actor) - } + if not is_nil(object) do + object = update_in(object.data["content"], &format_links/1) + + %{ + data: notification, + object: object, + from: User.get_by_ap_id(notification.activity.actor) + } + end end) + |> Enum.filter(& &1) followers = notifications |> Enum.filter(&(&1.activity.data["type"] == "Follow")) |> Enum.map(fn notification -> - %{ - data: notification, - object: Pleroma.Object.normalize(notification.activity), - from: User.get_by_ap_id(notification.activity.actor) - } + from = User.get_by_ap_id(notification.activity.actor) + + if not is_nil(from) do + %{ + data: notification, + object: Pleroma.Object.normalize(notification.activity), + from: User.get_by_ap_id(notification.activity.actor) + } + end end) + |> Enum.filter(& &1) unless Enum.empty?(mentions) do styling = Config.get([__MODULE__, :styling]) diff --git a/lib/pleroma/following_relationship.ex b/lib/pleroma/following_relationship.ex index 83b366dd4..2039a259d 100644 --- a/lib/pleroma/following_relationship.ex +++ b/lib/pleroma/following_relationship.ex @@ -264,4 +264,12 @@ defmodule Pleroma.FollowingRelationship do end end) end + + @spec following_ap_ids(User.t()) :: [String.t()] + def following_ap_ids(%User{} = user) do + user + |> following_query() + |> select([r, u], u.ap_id) + |> Repo.all() + end end diff --git a/lib/pleroma/object/fetcher.ex b/lib/pleroma/object/fetcher.ex index 3ff25118d..6fdbc8efd 100644 --- a/lib/pleroma/object/fetcher.ex +++ b/lib/pleroma/object/fetcher.ex @@ -125,8 +125,8 @@ defmodule Pleroma.Object.Fetcher do defp prepare_activity_params(data) do %{ "type" => "Create", - "to" => data["to"], - "cc" => data["cc"], + "to" => data["to"] || [], + "cc" => data["cc"] || [], # Should we seriously keep this attributedTo thing? "actor" => data["actor"] || data["attributedTo"], "object" => data diff --git a/lib/pleroma/plugs/frontend_static.ex b/lib/pleroma/plugs/frontend_static.ex index f549ca75f..11a0d5382 100644 --- a/lib/pleroma/plugs/frontend_static.ex +++ b/lib/pleroma/plugs/frontend_static.ex @@ -30,6 +30,7 @@ defmodule Pleroma.Plugs.FrontendStatic do opts |> Keyword.put(:from, "__unconfigured_frontend_static_plug") |> Plug.Static.init() + |> Map.put(:frontend_type, opts[:frontend_type]) end def call(conn, opts) do diff --git a/lib/pleroma/upload.ex b/lib/pleroma/upload.ex index 0fa6b89dc..015c87593 100644 --- a/lib/pleroma/upload.ex +++ b/lib/pleroma/upload.ex @@ -56,6 +56,15 @@ defmodule Pleroma.Upload do } defstruct [:id, :name, :tempfile, :content_type, :path] + defp get_description(opts, upload) do + case {opts[:description], Pleroma.Config.get([Pleroma.Upload, :default_description])} do + {description, _} when is_binary(description) -> description + {_, :filename} -> upload.name + {_, str} when is_binary(str) -> str + _ -> "" + end + end + @spec store(source, options :: [option()]) :: {:ok, Map.t()} | {:error, any()} def store(upload, opts \\ []) do opts = get_opts(opts) @@ -63,7 +72,7 @@ defmodule Pleroma.Upload do with {:ok, upload} <- prepare_upload(upload, opts), upload = %__MODULE__{upload | path: upload.path || "#{upload.id}/#{upload.name}"}, {:ok, upload} <- Pleroma.Upload.Filter.filter(opts.filters, upload), - description = Map.get(opts, :description) || upload.name, + description = get_description(opts, upload), {_, true} <- {:description_limit, String.length(description) <= Pleroma.Config.get([:instance, :description_limit])}, diff --git a/lib/pleroma/user.ex b/lib/pleroma/user.ex index d1436a688..d2ad9516f 100644 --- a/lib/pleroma/user.ex +++ b/lib/pleroma/user.ex @@ -247,6 +247,13 @@ defmodule Pleroma.User do end end + defdelegate following_count(user), to: FollowingRelationship + defdelegate following(user), to: FollowingRelationship + defdelegate following?(follower, followed), to: FollowingRelationship + defdelegate following_ap_ids(user), to: FollowingRelationship + defdelegate get_follow_requests(user), to: FollowingRelationship + defdelegate search(query, opts \\ []), to: User.Search + @doc """ Dumps Flake Id to SQL-compatible format (16-byte UUID). E.g. "9pQtDGXuq4p3VlcJEm" -> <<0, 0, 1, 110, 179, 218, 42, 92, 213, 41, 44, 227, 95, 213, 0, 0>> @@ -311,10 +318,12 @@ defmodule Pleroma.User do def visible_for(_, _), do: :invisible - defp restrict_unauthenticated?(%User{local: local}) do - config_key = if local, do: :local, else: :remote + defp restrict_unauthenticated?(%User{local: true}) do + Config.restrict_unauthenticated_access?(:profiles, :local) + end - Config.get([:restrict_unauthenticated, :profiles, config_key], false) + defp restrict_unauthenticated?(%User{local: _}) do + Config.restrict_unauthenticated_access?(:profiles, :remote) end defp visible_account_status(user) do @@ -370,8 +379,6 @@ defmodule Pleroma.User do from(u in query, where: u.deactivated != ^true) end - defdelegate following_count(user), to: FollowingRelationship - defp truncate_fields_param(params) do if Map.has_key?(params, :fields) do Map.put(params, :fields, Enum.map(params[:fields], &truncate_field/1)) @@ -866,8 +873,6 @@ defmodule Pleroma.User do set_cache(follower) end - defdelegate following(user), to: FollowingRelationship - def follow(%User{} = follower, %User{} = followed, state \\ :follow_accept) do deny_follow_blocked = Config.get([:user, :deny_follow_blocked]) @@ -921,8 +926,6 @@ defmodule Pleroma.User do end end - defdelegate following?(follower, followed), to: FollowingRelationship - @doc "Returns follow state as Pleroma.FollowingRelationship.State value" def get_follow_state(%User{} = follower, %User{} = following) do following_relationship = FollowingRelationship.get(follower, following) @@ -1187,8 +1190,6 @@ defmodule Pleroma.User do |> Repo.all() end - defdelegate get_follow_requests(user), to: FollowingRelationship - def increase_note_count(%User{} = user) do User |> where(id: ^user.id) @@ -1581,6 +1582,49 @@ defmodule Pleroma.User do |> update_and_set_cache() end + @spec purge_user_changeset(User.t()) :: Changeset.t() + def purge_user_changeset(user) do + # "Right to be forgotten" + # https://gdpr.eu/right-to-be-forgotten/ + change(user, %{ + bio: nil, + raw_bio: nil, + email: nil, + name: nil, + password_hash: nil, + keys: nil, + public_key: nil, + avatar: %{}, + tags: [], + last_refreshed_at: nil, + last_digest_emailed_at: nil, + banner: %{}, + background: %{}, + note_count: 0, + follower_count: 0, + following_count: 0, + locked: false, + confirmation_pending: false, + password_reset_pending: false, + approval_pending: false, + registration_reason: nil, + confirmation_token: nil, + domain_blocks: [], + deactivated: true, + ap_enabled: false, + is_moderator: false, + is_admin: false, + mastofe_settings: nil, + mascot: nil, + emoji: %{}, + pleroma_settings_store: %{}, + fields: [], + raw_fields: [], + discoverable: false, + also_known_as: [] + }) + end + def delete(users) when is_list(users) do for user <- users, do: delete(user) end @@ -1608,7 +1652,7 @@ defmodule Pleroma.User do _ -> user - |> change(%{deactivated: true, email: nil}) + |> purge_user_changeset() |> update_and_set_cache() end end @@ -2118,8 +2162,6 @@ defmodule Pleroma.User do |> Repo.all() end - defdelegate search(query, opts \\ []), to: User.Search - defp put_password_hash( %Ecto.Changeset{valid?: true, changes: %{password: password}} = changeset ) do diff --git a/lib/pleroma/web/activity_pub/activity_pub.ex b/lib/pleroma/web/activity_pub/activity_pub.ex index bde1fe708..624a508ae 100644 --- a/lib/pleroma/web/activity_pub/activity_pub.ex +++ b/lib/pleroma/web/activity_pub/activity_pub.ex @@ -85,7 +85,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do defp increase_replies_count_if_reply(_create_data), do: :noop - @object_types ["ChatMessage", "Question", "Answer"] + @object_types ~w[ChatMessage Question Answer Audio Event] @spec persist(map(), keyword()) :: {:ok, Activity.t() | Object.t()} def persist(%{"type" => type} = object, meta) when type in @object_types do with {:ok, object} <- Object.create(object) do @@ -1344,9 +1344,8 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do end def maybe_handle_clashing_nickname(data) do - nickname = data[:nickname] - - with %User{} = old_user <- User.get_by_nickname(nickname), + with nickname when is_binary(nickname) <- data[:nickname], + %User{} = old_user <- User.get_by_nickname(nickname), {_, false} <- {:ap_id_comparison, data[:ap_id] == old_user.ap_id} do Logger.info( "Found an old user for #{nickname}, the old ap id is #{old_user.ap_id}, new one is #{ @@ -1360,7 +1359,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do else {:ap_id_comparison, true} -> Logger.info( - "Found an old user for #{nickname}, but the ap id #{data[:ap_id]} is the same as the new user. Race condition? Not changing anything." + "Found an old user for #{data[:nickname]}, but the ap id #{data[:ap_id]} is the same as the new user. Race condition? Not changing anything." ) _ -> diff --git a/lib/pleroma/web/activity_pub/builder.ex b/lib/pleroma/web/activity_pub/builder.ex index f2392ce79..9a7b7d9de 100644 --- a/lib/pleroma/web/activity_pub/builder.ex +++ b/lib/pleroma/web/activity_pub/builder.ex @@ -215,7 +215,7 @@ defmodule Pleroma.Web.ActivityPub.Builder do to = cond do - actor.ap_id == Relay.relay_ap_id() -> + actor.ap_id == Relay.ap_id() -> [actor.follower_address] public? -> diff --git a/lib/pleroma/web/activity_pub/object_validator.ex b/lib/pleroma/web/activity_pub/object_validator.ex index 3f1dffe2b..b77c06395 100644 --- a/lib/pleroma/web/activity_pub/object_validator.ex +++ b/lib/pleroma/web/activity_pub/object_validator.ex @@ -16,12 +16,14 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidator do alias Pleroma.Web.ActivityPub.ObjectValidators.AcceptRejectValidator alias Pleroma.Web.ActivityPub.ObjectValidators.AnnounceValidator alias Pleroma.Web.ActivityPub.ObjectValidators.AnswerValidator + alias Pleroma.Web.ActivityPub.ObjectValidators.AudioValidator alias Pleroma.Web.ActivityPub.ObjectValidators.BlockValidator alias Pleroma.Web.ActivityPub.ObjectValidators.ChatMessageValidator alias Pleroma.Web.ActivityPub.ObjectValidators.CreateChatMessageValidator alias Pleroma.Web.ActivityPub.ObjectValidators.CreateGenericValidator alias Pleroma.Web.ActivityPub.ObjectValidators.DeleteValidator alias Pleroma.Web.ActivityPub.ObjectValidators.EmojiReactValidator + alias Pleroma.Web.ActivityPub.ObjectValidators.EventValidator alias Pleroma.Web.ActivityPub.ObjectValidators.FollowValidator alias Pleroma.Web.ActivityPub.ObjectValidators.LikeValidator alias Pleroma.Web.ActivityPub.ObjectValidators.QuestionValidator @@ -42,6 +44,16 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidator do end end + def validate(%{"type" => "Event"} = object, meta) do + with {:ok, object} <- + object + |> EventValidator.cast_and_validate() + |> Ecto.Changeset.apply_action(:insert) do + object = stringify_keys(object) + {:ok, object, meta} + end + end + def validate(%{"type" => "Follow"} = object, meta) do with {:ok, object} <- object @@ -137,6 +149,16 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidator do end end + def validate(%{"type" => "Audio"} = object, meta) do + with {:ok, object} <- + object + |> AudioValidator.cast_and_validate() + |> Ecto.Changeset.apply_action(:insert) do + object = stringify_keys(object) + {:ok, object, meta} + end + end + def validate(%{"type" => "Answer"} = object, meta) do with {:ok, object} <- object @@ -176,7 +198,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidator do %{"type" => "Create", "object" => %{"type" => objtype} = object} = create_activity, meta ) - when objtype in ["Question", "Answer"] do + when objtype in ~w[Question Answer Audio Event] do with {:ok, object_data} <- cast_and_apply(object), meta = Keyword.put(meta, :object_data, object_data |> stringify_keys), {:ok, create_activity} <- @@ -210,6 +232,14 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidator do AnswerValidator.cast_and_apply(object) end + def cast_and_apply(%{"type" => "Audio"} = object) do + AudioValidator.cast_and_apply(object) + end + + def cast_and_apply(%{"type" => "Event"} = object) do + EventValidator.cast_and_apply(object) + end + def cast_and_apply(o), do: {:error, {:validator_not_set, o}} # is_struct/1 isn't present in Elixir 1.8.x diff --git a/lib/pleroma/web/activity_pub/object_validators/answer_validator.ex b/lib/pleroma/web/activity_pub/object_validators/answer_validator.ex index 323367642..b9fbaf4f6 100644 --- a/lib/pleroma/web/activity_pub/object_validators/answer_validator.ex +++ b/lib/pleroma/web/activity_pub/object_validators/answer_validator.ex @@ -15,16 +15,13 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.AnswerValidator do embedded_schema do field(:id, ObjectValidators.ObjectID, primary_key: true) - field(:to, {:array, :string}, default: []) - field(:cc, {:array, :string}, default: []) - - # is this actually needed? - field(:bto, {:array, :string}, default: []) - field(:bcc, {:array, :string}, default: []) - + field(:to, ObjectValidators.Recipients, default: []) + field(:cc, ObjectValidators.Recipients, default: []) + field(:bto, ObjectValidators.Recipients, default: []) + field(:bcc, ObjectValidators.Recipients, default: []) field(:type, :string) field(:name, :string) - field(:inReplyTo, :string) + field(:inReplyTo, ObjectValidators.ObjectID) field(:attributedTo, ObjectValidators.ObjectID) # TODO: Remove actor on objects diff --git a/lib/pleroma/web/activity_pub/object_validators/attachment_validator.ex b/lib/pleroma/web/activity_pub/object_validators/attachment_validator.ex index f53bb02be..c8b148280 100644 --- a/lib/pleroma/web/activity_pub/object_validators/attachment_validator.ex +++ b/lib/pleroma/web/activity_pub/object_validators/attachment_validator.ex @@ -41,34 +41,34 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.AttachmentValidator do end def fix_media_type(data) do - data = - data - |> Map.put_new("mediaType", data["mimeType"]) + data = Map.put_new(data, "mediaType", data["mimeType"]) if MIME.valid?(data["mediaType"]) do data else - data - |> Map.put("mediaType", "application/octet-stream") + Map.put(data, "mediaType", "application/octet-stream") end end - def fix_url(data) do - case data["url"] do - url when is_binary(url) -> - data - |> Map.put( - "url", - [ - %{ - "href" => url, - "type" => "Link", - "mediaType" => data["mediaType"] - } - ] - ) - - _ -> + defp handle_href(href, mediaType) do + [ + %{ + "href" => href, + "type" => "Link", + "mediaType" => mediaType + } + ] + end + + defp fix_url(data) do + cond do + is_binary(data["url"]) -> + Map.put(data, "url", handle_href(data["url"], data["mediaType"])) + + is_binary(data["href"]) and data["url"] == nil -> + Map.put(data, "url", handle_href(data["href"], data["mediaType"])) + + true -> data end end diff --git a/lib/pleroma/web/activity_pub/object_validators/audio_validator.ex b/lib/pleroma/web/activity_pub/object_validators/audio_validator.ex new file mode 100644 index 000000000..d1869f188 --- /dev/null +++ b/lib/pleroma/web/activity_pub/object_validators/audio_validator.ex @@ -0,0 +1,106 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.ActivityPub.ObjectValidators.AudioValidator do + use Ecto.Schema + + alias Pleroma.EctoType.ActivityPub.ObjectValidators + alias Pleroma.Web.ActivityPub.ObjectValidators.AttachmentValidator + alias Pleroma.Web.ActivityPub.ObjectValidators.CommonFixes + alias Pleroma.Web.ActivityPub.ObjectValidators.CommonValidations + + import Ecto.Changeset + + @primary_key false + @derive Jason.Encoder + + embedded_schema do + field(:id, ObjectValidators.ObjectID, primary_key: true) + field(:to, ObjectValidators.Recipients, default: []) + field(:cc, ObjectValidators.Recipients, default: []) + field(:bto, ObjectValidators.Recipients, default: []) + field(:bcc, ObjectValidators.Recipients, default: []) + # TODO: Write type + field(:tag, {:array, :map}, default: []) + field(:type, :string) + field(:content, :string) + field(:context, :string) + + # TODO: Remove actor on objects + field(:actor, ObjectValidators.ObjectID) + + field(:attributedTo, ObjectValidators.ObjectID) + field(:summary, :string) + field(:published, ObjectValidators.DateTime) + # TODO: Write type + field(:emoji, :map, default: %{}) + field(:sensitive, :boolean, default: false) + embeds_many(:attachment, AttachmentValidator) + field(:replies_count, :integer, default: 0) + field(:like_count, :integer, default: 0) + field(:announcement_count, :integer, default: 0) + field(:inReplyTo, :string) + field(:url, ObjectValidators.Uri) + # short identifier for PleromaFE to group statuses by context + field(:context_id, :integer) + + field(:likes, {:array, :string}, default: []) + field(:announcements, {:array, :string}, default: []) + end + + def cast_and_apply(data) do + data + |> cast_data + |> apply_action(:insert) + end + + def cast_and_validate(data) do + data + |> cast_data() + |> validate_data() + end + + def cast_data(data) do + %__MODULE__{} + |> changeset(data) + end + + defp fix_url(%{"url" => url} = data) when is_list(url) do + attachment = + Enum.find(url, fn x -> is_map(x) and String.starts_with?(x["mimeType"], "audio/") end) + + link_element = Enum.find(url, fn x -> is_map(x) and x["mimeType"] == "text/html" end) + + data + |> Map.put("attachment", [attachment]) + |> Map.put("url", link_element["href"]) + end + + defp fix_url(data), do: data + + defp fix(data) do + data + |> CommonFixes.fix_defaults() + |> CommonFixes.fix_attribution() + |> fix_url() + end + + def changeset(struct, data) do + data = fix(data) + + struct + |> cast(data, __schema__(:fields) -- [:attachment]) + |> cast_embed(:attachment) + end + + def validate_data(data_cng) do + data_cng + |> validate_inclusion(:type, ["Audio"]) + |> validate_required([:id, :actor, :attributedTo, :type, :context, :attachment]) + |> CommonValidations.validate_any_presence([:cc, :to]) + |> CommonValidations.validate_fields_match([:actor, :attributedTo]) + |> CommonValidations.validate_actor_presence() + |> CommonValidations.validate_host_match() + end +end diff --git a/lib/pleroma/web/activity_pub/object_validators/common_fixes.ex b/lib/pleroma/web/activity_pub/object_validators/common_fixes.ex new file mode 100644 index 000000000..721749de0 --- /dev/null +++ b/lib/pleroma/web/activity_pub/object_validators/common_fixes.ex @@ -0,0 +1,22 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.ActivityPub.ObjectValidators.CommonFixes do + alias Pleroma.Web.ActivityPub.Utils + + # based on Pleroma.Web.ActivityPub.Utils.lazy_put_objects_defaults + def fix_defaults(data) do + %{data: %{"id" => context}, id: context_id} = + Utils.create_context(data["context"] || data["conversation"]) + + data + |> Map.put_new("context", context) + |> Map.put_new("context_id", context_id) + end + + def fix_attribution(data) do + data + |> Map.put_new("actor", data["attributedTo"]) + end +end diff --git a/lib/pleroma/web/activity_pub/object_validators/create_generic_validator.ex b/lib/pleroma/web/activity_pub/object_validators/create_generic_validator.ex index 60868eae0..b3dbeea57 100644 --- a/lib/pleroma/web/activity_pub/object_validators/create_generic_validator.ex +++ b/lib/pleroma/web/activity_pub/object_validators/create_generic_validator.ex @@ -61,9 +61,20 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.CreateGenericValidator do end end + defp fix_addressing(data, meta) do + if object = meta[:object_data] do + data + |> Map.put_new("to", object["to"] || []) + |> Map.put_new("cc", object["cc"] || []) + else + data + end + end + defp fix(data, meta) do data |> fix_context(meta) + |> fix_addressing(meta) end def validate_data(cng, meta \\ []) do diff --git a/lib/pleroma/web/activity_pub/object_validators/create_note_validator.ex b/lib/pleroma/web/activity_pub/object_validators/create_note_validator.ex index 316bd0c07..9b9743c4a 100644 --- a/lib/pleroma/web/activity_pub/object_validators/create_note_validator.ex +++ b/lib/pleroma/web/activity_pub/object_validators/create_note_validator.ex @@ -16,11 +16,10 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.CreateNoteValidator do field(:id, ObjectValidators.ObjectID, primary_key: true) field(:actor, ObjectValidators.ObjectID) field(:type, :string) - field(:to, {:array, :string}) - field(:cc, {:array, :string}) - field(:bto, {:array, :string}, default: []) - field(:bcc, {:array, :string}, default: []) - + field(:to, ObjectValidators.Recipients, default: []) + field(:cc, ObjectValidators.Recipients, default: []) + field(:bto, ObjectValidators.Recipients, default: []) + field(:bcc, ObjectValidators.Recipients, default: []) embeds_one(:object, NoteValidator) end diff --git a/lib/pleroma/web/activity_pub/object_validators/emoji_react_validator.ex b/lib/pleroma/web/activity_pub/object_validators/emoji_react_validator.ex index a543af1f8..336c92d35 100644 --- a/lib/pleroma/web/activity_pub/object_validators/emoji_react_validator.ex +++ b/lib/pleroma/web/activity_pub/object_validators/emoji_react_validator.ex @@ -20,8 +20,8 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.EmojiReactValidator do field(:actor, ObjectValidators.ObjectID) field(:context, :string) field(:content, :string) - field(:to, {:array, :string}, default: []) - field(:cc, {:array, :string}, default: []) + field(:to, ObjectValidators.Recipients, default: []) + field(:cc, ObjectValidators.Recipients, default: []) end def cast_and_validate(data) do diff --git a/lib/pleroma/web/activity_pub/object_validators/event_validator.ex b/lib/pleroma/web/activity_pub/object_validators/event_validator.ex new file mode 100644 index 000000000..07e4821a4 --- /dev/null +++ b/lib/pleroma/web/activity_pub/object_validators/event_validator.ex @@ -0,0 +1,96 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.ActivityPub.ObjectValidators.EventValidator do + use Ecto.Schema + + alias Pleroma.EctoType.ActivityPub.ObjectValidators + alias Pleroma.Web.ActivityPub.ObjectValidators.AttachmentValidator + alias Pleroma.Web.ActivityPub.ObjectValidators.CommonFixes + alias Pleroma.Web.ActivityPub.ObjectValidators.CommonValidations + + import Ecto.Changeset + + @primary_key false + @derive Jason.Encoder + + # Extends from NoteValidator + embedded_schema do + field(:id, ObjectValidators.ObjectID, primary_key: true) + field(:to, ObjectValidators.Recipients, default: []) + field(:cc, ObjectValidators.Recipients, default: []) + field(:bto, ObjectValidators.Recipients, default: []) + field(:bcc, ObjectValidators.Recipients, default: []) + # TODO: Write type + field(:tag, {:array, :map}, default: []) + field(:type, :string) + + field(:name, :string) + field(:summary, :string) + field(:content, :string) + + field(:context, :string) + # short identifier for PleromaFE to group statuses by context + field(:context_id, :integer) + + # TODO: Remove actor on objects + field(:actor, ObjectValidators.ObjectID) + + field(:attributedTo, ObjectValidators.ObjectID) + field(:published, ObjectValidators.DateTime) + # TODO: Write type + field(:emoji, :map, default: %{}) + field(:sensitive, :boolean, default: false) + embeds_many(:attachment, AttachmentValidator) + field(:replies_count, :integer, default: 0) + field(:like_count, :integer, default: 0) + field(:announcement_count, :integer, default: 0) + field(:inReplyTo, ObjectValidators.ObjectID) + field(:url, ObjectValidators.Uri) + + field(:likes, {:array, ObjectValidators.ObjectID}, default: []) + field(:announcements, {:array, ObjectValidators.ObjectID}, default: []) + end + + def cast_and_apply(data) do + data + |> cast_data + |> apply_action(:insert) + end + + def cast_and_validate(data) do + data + |> cast_data() + |> validate_data() + end + + def cast_data(data) do + %__MODULE__{} + |> changeset(data) + end + + defp fix(data) do + data + |> CommonFixes.fix_defaults() + |> CommonFixes.fix_attribution() + end + + def changeset(struct, data) do + data = fix(data) + + struct + |> cast(data, __schema__(:fields) -- [:attachment]) + |> cast_embed(:attachment) + end + + def validate_data(data_cng) do + data_cng + |> validate_inclusion(:type, ["Event"]) + |> validate_required([:id, :actor, :attributedTo, :type, :context, :context_id]) + |> CommonValidations.validate_any_presence([:cc, :to]) + |> CommonValidations.validate_fields_match([:actor, :attributedTo]) + |> CommonValidations.validate_actor_presence() + |> CommonValidations.validate_host_match() + end +end diff --git a/lib/pleroma/web/activity_pub/object_validators/note_validator.ex b/lib/pleroma/web/activity_pub/object_validators/note_validator.ex index a65fe2354..20e735619 100644 --- a/lib/pleroma/web/activity_pub/object_validators/note_validator.ex +++ b/lib/pleroma/web/activity_pub/object_validators/note_validator.ex @@ -13,18 +13,24 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.NoteValidator do embedded_schema do field(:id, ObjectValidators.ObjectID, primary_key: true) - field(:to, {:array, :string}, default: []) - field(:cc, {:array, :string}, default: []) - field(:bto, {:array, :string}, default: []) - field(:bcc, {:array, :string}, default: []) + field(:to, ObjectValidators.Recipients, default: []) + field(:cc, ObjectValidators.Recipients, default: []) + field(:bto, ObjectValidators.Recipients, default: []) + field(:bcc, ObjectValidators.Recipients, default: []) # TODO: Write type field(:tag, {:array, :map}, default: []) field(:type, :string) + + field(:name, :string) + field(:summary, :string) field(:content, :string) + field(:context, :string) + # short identifier for PleromaFE to group statuses by context + field(:context_id, :integer) + field(:actor, ObjectValidators.ObjectID) field(:attributedTo, ObjectValidators.ObjectID) - field(:summary, :string) field(:published, ObjectValidators.DateTime) # TODO: Write type field(:emoji, :map, default: %{}) @@ -34,14 +40,11 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.NoteValidator do field(:replies_count, :integer, default: 0) field(:like_count, :integer, default: 0) field(:announcement_count, :integer, default: 0) - field(:inReplyTo, :string) - field(:uri, ObjectValidators.Uri) + field(:inReplyTo, ObjectValidators.ObjectID) + field(:url, ObjectValidators.Uri) field(:likes, {:array, :string}, default: []) field(:announcements, {:array, :string}, default: []) - - # see if needed - field(:context_id, :string) end def cast_and_validate(data) do diff --git a/lib/pleroma/web/activity_pub/object_validators/question_validator.ex b/lib/pleroma/web/activity_pub/object_validators/question_validator.ex index f47acf606..712047424 100644 --- a/lib/pleroma/web/activity_pub/object_validators/question_validator.ex +++ b/lib/pleroma/web/activity_pub/object_validators/question_validator.ex @@ -7,9 +7,9 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.QuestionValidator do alias Pleroma.EctoType.ActivityPub.ObjectValidators alias Pleroma.Web.ActivityPub.ObjectValidators.AttachmentValidator + alias Pleroma.Web.ActivityPub.ObjectValidators.CommonFixes alias Pleroma.Web.ActivityPub.ObjectValidators.CommonValidations alias Pleroma.Web.ActivityPub.ObjectValidators.QuestionOptionsValidator - alias Pleroma.Web.ActivityPub.Utils import Ecto.Changeset @@ -19,10 +19,10 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.QuestionValidator do # Extends from NoteValidator embedded_schema do field(:id, ObjectValidators.ObjectID, primary_key: true) - field(:to, {:array, :string}, default: []) - field(:cc, {:array, :string}, default: []) - field(:bto, {:array, :string}, default: []) - field(:bcc, {:array, :string}, default: []) + field(:to, ObjectValidators.Recipients, default: []) + field(:cc, ObjectValidators.Recipients, default: []) + field(:bto, ObjectValidators.Recipients, default: []) + field(:bcc, ObjectValidators.Recipients, default: []) # TODO: Write type field(:tag, {:array, :map}, default: []) field(:type, :string) @@ -42,8 +42,8 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.QuestionValidator do field(:replies_count, :integer, default: 0) field(:like_count, :integer, default: 0) field(:announcement_count, :integer, default: 0) - field(:inReplyTo, :string) - field(:uri, ObjectValidators.Uri) + field(:inReplyTo, ObjectValidators.ObjectID) + field(:url, ObjectValidators.Uri) # short identifier for PleromaFE to group statuses by context field(:context_id, :integer) @@ -81,27 +81,11 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.QuestionValidator do end end - # based on Pleroma.Web.ActivityPub.Utils.lazy_put_objects_defaults - defp fix_defaults(data) do - %{data: %{"id" => context}, id: context_id} = - Utils.create_context(data["context"] || data["conversation"]) - - data - |> Map.put_new_lazy("published", &Utils.make_date/0) - |> Map.put_new("context", context) - |> Map.put_new("context_id", context_id) - end - - defp fix_attribution(data) do - data - |> Map.put_new("actor", data["attributedTo"]) - end - defp fix(data) do data - |> fix_attribution() + |> CommonFixes.fix_defaults() + |> CommonFixes.fix_attribution() |> fix_closed() - |> fix_defaults() end def changeset(struct, data) do @@ -117,7 +101,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.QuestionValidator do def validate_data(data_cng) do data_cng |> validate_inclusion(:type, ["Question"]) - |> validate_required([:id, :actor, :attributedTo, :type, :context]) + |> validate_required([:id, :actor, :attributedTo, :type, :context, :context_id]) |> CommonValidations.validate_any_presence([:cc, :to]) |> CommonValidations.validate_fields_match([:actor, :attributedTo]) |> CommonValidations.validate_actor_presence() diff --git a/lib/pleroma/web/activity_pub/object_validators/undo_validator.ex b/lib/pleroma/web/activity_pub/object_validators/undo_validator.ex index e8d2d39c1..8cae94467 100644 --- a/lib/pleroma/web/activity_pub/object_validators/undo_validator.ex +++ b/lib/pleroma/web/activity_pub/object_validators/undo_validator.ex @@ -18,8 +18,8 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.UndoValidator do field(:type, :string) field(:object, ObjectValidators.ObjectID) field(:actor, ObjectValidators.ObjectID) - field(:to, {:array, :string}, default: []) - field(:cc, {:array, :string}, default: []) + field(:to, ObjectValidators.Recipients, default: []) + field(:cc, ObjectValidators.Recipients, default: []) end def cast_and_validate(data) do diff --git a/lib/pleroma/web/activity_pub/relay.ex b/lib/pleroma/web/activity_pub/relay.ex index b09764d2b..b65710a94 100644 --- a/lib/pleroma/web/activity_pub/relay.ex +++ b/lib/pleroma/web/activity_pub/relay.ex @@ -10,19 +10,13 @@ defmodule Pleroma.Web.ActivityPub.Relay do alias Pleroma.Web.CommonAPI require Logger - @relay_nickname "relay" + @nickname "relay" - def get_actor do - actor = - relay_ap_id() - |> User.get_or_create_service_actor_by_ap_id(@relay_nickname) + @spec ap_id() :: String.t() + def ap_id, do: "#{Pleroma.Web.Endpoint.url()}/#{@nickname}" - actor - end - - def relay_ap_id do - "#{Pleroma.Web.Endpoint.url()}/relay" - end + @spec get_actor() :: User.t() | nil + def get_actor, do: User.get_or_create_service_actor_by_ap_id(ap_id(), @nickname) @spec follow(String.t()) :: {:ok, Activity.t()} | {:error, any()} def follow(target_instance) do @@ -61,34 +55,38 @@ defmodule Pleroma.Web.ActivityPub.Relay do def publish(_), do: {:error, "Not implemented"} - @spec list(boolean()) :: {:ok, [String.t()]} | {:error, any()} - def list(with_not_accepted \\ false) do + @spec list() :: {:ok, [%{actor: String.t(), followed_back: boolean()}]} | {:error, any()} + def list do with %User{} = user <- get_actor() do accepted = user - |> User.following() - |> Enum.map(fn entry -> URI.parse(entry).host end) - |> Enum.uniq() - - list = - if with_not_accepted do - without_accept = - user - |> Pleroma.Activity.following_requests_for_actor() - |> Enum.map(fn a -> URI.parse(a.data["object"]).host <> " (no Accept received)" end) - |> Enum.uniq() + |> following() + |> Enum.map(fn actor -> %{actor: actor, followed_back: true} end) - accepted ++ without_accept - else - accepted - end + without_accept = + user + |> Pleroma.Activity.following_requests_for_actor() + |> Enum.map(fn activity -> %{actor: activity.data["object"], followed_back: false} end) + |> Enum.uniq() - {:ok, list} + {:ok, accepted ++ without_accept} else error -> format_error(error) end end + @spec following() :: [String.t()] + def following do + get_actor() + |> following() + end + + defp following(user) do + user + |> User.following_ap_ids() + |> Enum.uniq() + end + defp format_error({:error, error}), do: format_error(error) defp format_error(error) do diff --git a/lib/pleroma/web/activity_pub/side_effects.ex b/lib/pleroma/web/activity_pub/side_effects.ex index bcd6fd2fb..a5e2323bd 100644 --- a/lib/pleroma/web/activity_pub/side_effects.ex +++ b/lib/pleroma/web/activity_pub/side_effects.ex @@ -340,7 +340,8 @@ defmodule Pleroma.Web.ActivityPub.SideEffects do end end - def handle_object_creation(%{"type" => "Question"} = object, meta) do + def handle_object_creation(%{"type" => objtype} = object, meta) + when objtype in ~w[Audio Question Event] do with {:ok, object, meta} <- Pipeline.common_pipeline(object, meta) do {:ok, object, meta} end diff --git a/lib/pleroma/web/activity_pub/transmogrifier.ex b/lib/pleroma/web/activity_pub/transmogrifier.ex index 544f3f3b6..76298c4a0 100644 --- a/lib/pleroma/web/activity_pub/transmogrifier.ex +++ b/lib/pleroma/web/activity_pub/transmogrifier.ex @@ -276,13 +276,12 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do Map.put(object, "url", url["href"]) end - def fix_url(%{"type" => object_type, "url" => url} = object) - when object_type in ["Video", "Audio"] and is_list(url) do + def fix_url(%{"type" => "Video", "url" => url} = object) when is_list(url) do attachment = Enum.find(url, fn x -> media_type = x["mediaType"] || x["mimeType"] || "" - is_map(x) and String.starts_with?(media_type, ["audio/", "video/"]) + is_map(x) and String.starts_with?(media_type, "video/") end) link_element = @@ -461,7 +460,7 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do %{"type" => "Create", "object" => %{"type" => objtype} = object} = data, options ) - when objtype in ["Article", "Event", "Note", "Video", "Page", "Audio"] do + when objtype in ~w{Article Note Video Page} do actor = Containment.get_actor(data) with nil <- Activity.get_create_by_object_ap_id(object["id"]), @@ -555,7 +554,7 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do %{"type" => "Create", "object" => %{"type" => objtype}} = data, _options ) - when objtype in ["Question", "Answer", "ChatMessage"] do + when objtype in ~w{Question Answer ChatMessage Audio Event} do with {:ok, %User{}} <- ObjectValidator.fetch_actor(data), {:ok, activity, _} <- Pipeline.common_pipeline(data, local: false) do {:ok, activity} diff --git a/lib/pleroma/web/activity_pub/visibility.ex b/lib/pleroma/web/activity_pub/visibility.ex index 343f41caa..5c349bb7a 100644 --- a/lib/pleroma/web/activity_pub/visibility.ex +++ b/lib/pleroma/web/activity_pub/visibility.ex @@ -59,12 +59,9 @@ defmodule Pleroma.Web.ActivityPub.Visibility do end def visible_for_user?(%{local: local} = activity, nil) do - cfg_key = - if local, - do: :local, - else: :remote + cfg_key = if local, do: :local, else: :remote - if Pleroma.Config.get([:restrict_unauthenticated, :activities, cfg_key]), + if Pleroma.Config.restrict_unauthenticated_access?(:activities, cfg_key), do: false, else: is_public?(activity) end diff --git a/lib/pleroma/web/admin_api/controllers/media_proxy_cache_controller.ex b/lib/pleroma/web/admin_api/controllers/media_proxy_cache_controller.ex index e2759d59f..131e22d78 100644 --- a/lib/pleroma/web/admin_api/controllers/media_proxy_cache_controller.ex +++ b/lib/pleroma/web/admin_api/controllers/media_proxy_cache_controller.ex @@ -26,29 +26,40 @@ defmodule Pleroma.Web.AdminAPI.MediaProxyCacheController do defdelegate open_api_operation(action), to: Spec.MediaProxyCacheOperation def index(%{assigns: %{user: _}} = conn, params) do - cursor = - :banned_urls_cache - |> :ets.table([{:traverse, {:select, Cachex.Query.create(true, :key)}}]) - |> :qlc.cursor() + entries = fetch_entries(params) + urls = paginate_entries(entries, params.page, params.page_size) + + render(conn, "index.json", + urls: urls, + page_size: params.page_size, + count: length(entries) + ) + end + + defp fetch_entries(params) do + MediaProxy.cache_table() + |> Cachex.stream!(Cachex.Query.create(true, :key)) + |> filter_entries(params[:query]) + end - urls = - case params.page do - 1 -> - :qlc.next_answers(cursor, params.page_size) + defp filter_entries(stream, query) when is_binary(query) do + regex = ~r/#{query}/i - _ -> - :qlc.next_answers(cursor, (params.page - 1) * params.page_size) - :qlc.next_answers(cursor, params.page_size) - end + stream + |> Enum.filter(fn url -> String.match?(url, regex) end) + |> Enum.to_list() + end - :qlc.delete_cursor(cursor) + defp filter_entries(stream, _), do: Enum.to_list(stream) - render(conn, "index.json", urls: urls) + defp paginate_entries(entries, page, page_size) do + offset = page_size * (page - 1) + Enum.slice(entries, offset, page_size) end def delete(%{assigns: %{user: _}, body_params: %{urls: urls}} = conn, _) do MediaProxy.remove_from_banned_urls(urls) - render(conn, "index.json", urls: urls) + json(conn, %{}) end def purge(%{assigns: %{user: _}, body_params: %{urls: urls, ban: ban}} = conn, _) do @@ -58,6 +69,6 @@ defmodule Pleroma.Web.AdminAPI.MediaProxyCacheController do MediaProxy.put_in_banned_urls(urls) end - render(conn, "index.json", urls: urls) + json(conn, %{}) end end diff --git a/lib/pleroma/web/admin_api/controllers/relay_controller.ex b/lib/pleroma/web/admin_api/controllers/relay_controller.ex index cf9f3a14b..95d06dde7 100644 --- a/lib/pleroma/web/admin_api/controllers/relay_controller.ex +++ b/lib/pleroma/web/admin_api/controllers/relay_controller.ex @@ -39,7 +39,7 @@ defmodule Pleroma.Web.AdminAPI.RelayController do target: target }) - json(conn, target) + json(conn, %{actor: target, followed_back: target in Relay.following()}) else _ -> conn diff --git a/lib/pleroma/web/admin_api/views/account_view.ex b/lib/pleroma/web/admin_api/views/account_view.ex index 333e72e42..9c477feab 100644 --- a/lib/pleroma/web/admin_api/views/account_view.ex +++ b/lib/pleroma/web/admin_api/views/account_view.ex @@ -79,7 +79,8 @@ defmodule Pleroma.Web.AdminAPI.AccountView do "confirmation_pending" => user.confirmation_pending, "approval_pending" => user.approval_pending, "url" => user.uri || user.ap_id, - "registration_reason" => user.registration_reason + "registration_reason" => user.registration_reason, + "actor_type" => user.actor_type } end diff --git a/lib/pleroma/web/admin_api/views/media_proxy_cache_view.ex b/lib/pleroma/web/admin_api/views/media_proxy_cache_view.ex index c97400beb..a803bda0b 100644 --- a/lib/pleroma/web/admin_api/views/media_proxy_cache_view.ex +++ b/lib/pleroma/web/admin_api/views/media_proxy_cache_view.ex @@ -5,7 +5,11 @@ defmodule Pleroma.Web.AdminAPI.MediaProxyCacheView do use Pleroma.Web, :view - def render("index.json", %{urls: urls}) do - %{urls: urls} + def render("index.json", %{urls: urls, page_size: page_size, count: count}) do + %{ + urls: urls, + count: count, + page_size: page_size + } end end diff --git a/lib/pleroma/web/api_spec/operations/admin/media_proxy_cache_operation.ex b/lib/pleroma/web/api_spec/operations/admin/media_proxy_cache_operation.ex index 20d033f66..ab45d6633 100644 --- a/lib/pleroma/web/api_spec/operations/admin/media_proxy_cache_operation.ex +++ b/lib/pleroma/web/api_spec/operations/admin/media_proxy_cache_operation.ex @@ -22,6 +22,12 @@ defmodule Pleroma.Web.ApiSpec.Admin.MediaProxyCacheOperation do security: [%{"oAuth" => ["read:media_proxy_caches"]}], parameters: [ Operation.parameter( + :query, + :query, + %Schema{type: :string, default: nil}, + "Page" + ), + Operation.parameter( :page, :query, %Schema{type: :integer, default: 1}, @@ -36,7 +42,26 @@ defmodule Pleroma.Web.ApiSpec.Admin.MediaProxyCacheOperation do | admin_api_params() ], responses: %{ - 200 => success_response() + 200 => + Operation.response( + "Array of banned MediaProxy URLs in Cachex", + "application/json", + %Schema{ + type: :object, + properties: %{ + count: %Schema{type: :integer}, + page_size: %Schema{type: :integer}, + urls: %Schema{ + type: :array, + items: %Schema{ + type: :string, + format: :uri, + description: "MediaProxy URLs" + } + } + } + } + ) } } end @@ -61,7 +86,7 @@ defmodule Pleroma.Web.ApiSpec.Admin.MediaProxyCacheOperation do required: true ), responses: %{ - 200 => success_response(), + 200 => empty_object_response(), 400 => Operation.response("Error", "application/json", ApiError) } } @@ -88,25 +113,9 @@ defmodule Pleroma.Web.ApiSpec.Admin.MediaProxyCacheOperation do required: true ), responses: %{ - 200 => success_response(), + 200 => empty_object_response(), 400 => Operation.response("Error", "application/json", ApiError) } } end - - defp success_response do - Operation.response("Array of banned MediaProxy URLs in Cachex", "application/json", %Schema{ - type: :object, - properties: %{ - urls: %Schema{ - type: :array, - items: %Schema{ - type: :string, - format: :uri, - description: "MediaProxy URLs" - } - } - } - }) - end end diff --git a/lib/pleroma/web/api_spec/operations/admin/relay_operation.ex b/lib/pleroma/web/api_spec/operations/admin/relay_operation.ex index 67ee5eee0..e06b2d164 100644 --- a/lib/pleroma/web/api_spec/operations/admin/relay_operation.ex +++ b/lib/pleroma/web/api_spec/operations/admin/relay_operation.ex @@ -27,8 +27,7 @@ defmodule Pleroma.Web.ApiSpec.Admin.RelayOperation do properties: %{ relays: %Schema{ type: :array, - items: %Schema{type: :string}, - example: ["lain.com", "mstdn.io"] + items: relay() } } }) @@ -43,19 +42,9 @@ defmodule Pleroma.Web.ApiSpec.Admin.RelayOperation do operationId: "AdminAPI.RelayController.follow", security: [%{"oAuth" => ["write:follows"]}], parameters: admin_api_params(), - requestBody: - request_body("Parameters", %Schema{ - type: :object, - properties: %{ - relay_url: %Schema{type: :string, format: :uri} - } - }), + requestBody: request_body("Parameters", relay_url()), responses: %{ - 200 => - Operation.response("Status", "application/json", %Schema{ - type: :string, - example: "http://mastodon.example.org/users/admin" - }) + 200 => Operation.response("Status", "application/json", relay()) } } end @@ -67,13 +56,7 @@ defmodule Pleroma.Web.ApiSpec.Admin.RelayOperation do operationId: "AdminAPI.RelayController.unfollow", security: [%{"oAuth" => ["write:follows"]}], parameters: admin_api_params(), - requestBody: - request_body("Parameters", %Schema{ - type: :object, - properties: %{ - relay_url: %Schema{type: :string, format: :uri} - } - }), + requestBody: request_body("Parameters", relay_url()), responses: %{ 200 => Operation.response("Status", "application/json", %Schema{ @@ -83,4 +66,29 @@ defmodule Pleroma.Web.ApiSpec.Admin.RelayOperation do } } end + + defp relay do + %Schema{ + type: :object, + properties: %{ + actor: %Schema{ + type: :string, + example: "https://example.com/relay" + }, + followed_back: %Schema{ + type: :boolean, + description: "Is relay followed back by this actor?" + } + } + } + end + + defp relay_url do + %Schema{ + type: :object, + properties: %{ + relay_url: %Schema{type: :string, format: :uri} + } + } + end end diff --git a/lib/pleroma/web/endpoint.ex b/lib/pleroma/web/endpoint.ex index 527fb288d..8b153763d 100644 --- a/lib/pleroma/web/endpoint.ex +++ b/lib/pleroma/web/endpoint.ex @@ -39,6 +39,18 @@ defmodule Pleroma.Web.Endpoint do } ) + plug(Plug.Static.IndexHtml, at: "/pleroma/admin/") + + plug(Pleroma.Plugs.FrontendStatic, + at: "/pleroma/admin", + frontend_type: :admin, + gzip: true, + cache_control_for_etags: @static_cache_control, + headers: %{ + "cache-control" => @static_cache_control + } + ) + # Serve at "/" the static files from "priv/static" directory. # # You should set gzip to true if you are running phoenix.digest @@ -56,8 +68,6 @@ defmodule Pleroma.Web.Endpoint do } ) - plug(Plug.Static.IndexHtml, at: "/pleroma/admin/") - plug(Plug.Static, at: "/pleroma/admin/", from: {:pleroma, "priv/static/adminfe/"} diff --git a/lib/pleroma/web/mastodon_api/controllers/timeline_controller.ex b/lib/pleroma/web/mastodon_api/controllers/timeline_controller.ex index ab7b1d6aa..9244316ed 100644 --- a/lib/pleroma/web/mastodon_api/controllers/timeline_controller.ex +++ b/lib/pleroma/web/mastodon_api/controllers/timeline_controller.ex @@ -8,6 +8,7 @@ defmodule Pleroma.Web.MastodonAPI.TimelineController do import Pleroma.Web.ControllerHelper, only: [add_link_headers: 2, add_link_headers: 3] + alias Pleroma.Config alias Pleroma.Pagination alias Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug alias Pleroma.Plugs.OAuthScopesPlug @@ -89,11 +90,11 @@ defmodule Pleroma.Web.MastodonAPI.TimelineController do end defp restrict_unauthenticated?(true = _local_only) do - Pleroma.Config.get([:restrict_unauthenticated, :timelines, :local]) + Config.restrict_unauthenticated_access?(:timelines, :local) end defp restrict_unauthenticated?(_) do - Pleroma.Config.get([:restrict_unauthenticated, :timelines, :federated]) + Config.restrict_unauthenticated_access?(:timelines, :federated) end # GET /api/v1/timelines/public diff --git a/lib/pleroma/web/mastodon_api/views/status_view.ex b/lib/pleroma/web/mastodon_api/views/status_view.ex index 91b41ef59..01b8bb6bb 100644 --- a/lib/pleroma/web/mastodon_api/views/status_view.ex +++ b/lib/pleroma/web/mastodon_api/views/status_view.ex @@ -473,23 +473,10 @@ defmodule Pleroma.Web.MastodonAPI.StatusView do end end - def render_content(%{data: %{"type" => object_type}} = object) - when object_type in ["Video", "Event", "Audio"] do - with name when not is_nil(name) and name != "" <- object.data["name"] do - "<p><a href=\"#{object.data["id"]}\">#{name}</a></p>#{object.data["content"]}" - else - _ -> object.data["content"] || "" - end - end + def render_content(%{data: %{"name" => name}} = object) when not is_nil(name) and name != "" do + url = object.data["url"] || object.data["id"] - def render_content(%{data: %{"type" => object_type}} = object) - when object_type in ["Article", "Page"] do - with summary when not is_nil(summary) and summary != "" <- object.data["name"], - url when is_bitstring(url) <- object.data["url"] do - "<p><a href=\"#{url}\">#{summary}</a></p>#{object.data["content"]}" - else - _ -> object.data["content"] || "" - end + "<p><a href=\"#{url}\">#{name}</a></p>#{object.data["content"]}" end def render_content(object), do: object.data["content"] || "" diff --git a/lib/pleroma/web/media_proxy/media_proxy.ex b/lib/pleroma/web/media_proxy/media_proxy.ex index dfbfcea6b..e18dd8224 100644 --- a/lib/pleroma/web/media_proxy/media_proxy.ex +++ b/lib/pleroma/web/media_proxy/media_proxy.ex @@ -9,28 +9,31 @@ defmodule Pleroma.Web.MediaProxy do alias Pleroma.Web.MediaProxy.Invalidation @base64_opts [padding: false] + @cache_table :banned_urls_cache + + def cache_table, do: @cache_table @spec in_banned_urls(String.t()) :: boolean() - def in_banned_urls(url), do: elem(Cachex.exists?(:banned_urls_cache, url(url)), 1) + def in_banned_urls(url), do: elem(Cachex.exists?(@cache_table, url(url)), 1) def remove_from_banned_urls(urls) when is_list(urls) do - Cachex.execute!(:banned_urls_cache, fn cache -> + Cachex.execute!(@cache_table, fn cache -> Enum.each(Invalidation.prepare_urls(urls), &Cachex.del(cache, &1)) end) end def remove_from_banned_urls(url) when is_binary(url) do - Cachex.del(:banned_urls_cache, url(url)) + Cachex.del(@cache_table, url(url)) end def put_in_banned_urls(urls) when is_list(urls) do - Cachex.execute!(:banned_urls_cache, fn cache -> + Cachex.execute!(@cache_table, fn cache -> Enum.each(Invalidation.prepare_urls(urls), &Cachex.put(cache, &1, true)) end) end def put_in_banned_urls(url) when is_binary(url) do - Cachex.put(:banned_urls_cache, url(url), true) + Cachex.put(@cache_table, url(url), true) end def url(url) when is_nil(url) or url == "", do: nil diff --git a/lib/pleroma/web/preload/timelines.ex b/lib/pleroma/web/preload/timelines.ex index 57de04051..b279a865d 100644 --- a/lib/pleroma/web/preload/timelines.ex +++ b/lib/pleroma/web/preload/timelines.ex @@ -16,7 +16,7 @@ defmodule Pleroma.Web.Preload.Providers.Timelines do end def build_public_tag(acc, params) do - if Pleroma.Config.get([:restrict_unauthenticated, :timelines, :federated], true) do + if Pleroma.Config.restrict_unauthenticated_access?(:timelines, :federated) do acc else Map.put(acc, @public_url, public_timeline(params)) diff --git a/lib/pleroma/web/web_finger/web_finger.ex b/lib/pleroma/web/web_finger/web_finger.ex index 71ccf251a..c4051e63e 100644 --- a/lib/pleroma/web/web_finger/web_finger.ex +++ b/lib/pleroma/web/web_finger/web_finger.ex @@ -149,6 +149,18 @@ defmodule Pleroma.Web.WebFinger do end end + defp get_address_from_domain(domain, encoded_account) when is_binary(domain) do + case find_lrdd_template(domain) do + {:ok, template} -> + String.replace(template, "{uri}", encoded_account) + + _ -> + "https://#{domain}/.well-known/webfinger?resource=#{encoded_account}" + end + end + + defp get_address_from_domain(_, _), do: nil + @spec finger(String.t()) :: {:ok, map()} | {:error, any()} def finger(account) do account = String.trim_leading(account, "@") @@ -163,16 +175,8 @@ defmodule Pleroma.Web.WebFinger do encoded_account = URI.encode("acct:#{account}") - address = - case find_lrdd_template(domain) do - {:ok, template} -> - String.replace(template, "{uri}", encoded_account) - - _ -> - "https://#{domain}/.well-known/webfinger?resource=#{encoded_account}" - end - - with response <- + with address when is_binary(address) <- get_address_from_domain(domain, encoded_account), + response <- HTTP.get( address, [{"accept", "application/xrd+xml,application/jrd+json"}] diff --git a/priv/repo/migrations/20200817120935_add_invisible_index_to_users.exs b/priv/repo/migrations/20200817120935_add_invisible_index_to_users.exs new file mode 100644 index 000000000..2417d366e --- /dev/null +++ b/priv/repo/migrations/20200817120935_add_invisible_index_to_users.exs @@ -0,0 +1,7 @@ +defmodule Pleroma.Repo.Migrations.AddInvisibleIndexToUsers do + use Ecto.Migration + + def change do + create(index(:users, [:invisible])) + end +end diff --git a/test/emails/mailer_test.exs b/test/emails/mailer_test.exs index 3da45056b..9e232d2a0 100644 --- a/test/emails/mailer_test.exs +++ b/test/emails/mailer_test.exs @@ -14,10 +14,10 @@ defmodule Pleroma.Emails.MailerTest do subject: "Pleroma test email", to: [{"Test User", "user1@example.com"}] } - setup do: clear_config([Pleroma.Emails.Mailer, :enabled]) + setup do: clear_config([Pleroma.Emails.Mailer, :enabled], true) test "not send email when mailer is disabled" do - Pleroma.Config.put([Pleroma.Emails.Mailer, :enabled], false) + clear_config([Pleroma.Emails.Mailer, :enabled], false) Mailer.deliver(@email) :timer.sleep(100) diff --git a/test/fixtures/tesla_mock/dist/test.txt b/test/fixtures/tesla_mock/dist/test.txt new file mode 100644 index 000000000..e9ea42a12 --- /dev/null +++ b/test/fixtures/tesla_mock/dist/test.txt @@ -0,0 +1 @@ +this is a text file diff --git a/test/fixtures/tesla_mock/frontend.zip b/test/fixtures/tesla_mock/frontend.zip Binary files differnew file mode 100644 index 000000000..114d576a3 --- /dev/null +++ b/test/fixtures/tesla_mock/frontend.zip diff --git a/test/fixtures/tesla_mock/frontend_dist.zip b/test/fixtures/tesla_mock/frontend_dist.zip Binary files differnew file mode 100644 index 000000000..20d7952a4 --- /dev/null +++ b/test/fixtures/tesla_mock/frontend_dist.zip diff --git a/test/fixtures/tesla_mock/funkwhale_create_audio.json b/test/fixtures/tesla_mock/funkwhale_create_audio.json new file mode 100644 index 000000000..fe6059cbf --- /dev/null +++ b/test/fixtures/tesla_mock/funkwhale_create_audio.json @@ -0,0 +1,58 @@ +{ + "@context": [ + "https://www.w3.org/ns/activitystreams", + "https://w3id.org/security/v1", + "https://funkwhale.audio/ns", + { + "manuallyApprovesFollowers": "as:manuallyApprovesFollowers", + "Hashtag": "as:Hashtag" + } + ], + "type": "Create", + "id": "https://channels.tests.funkwhale.audio/federation/music/uploads/42342395-0208-4fee-a38d-259a6dae0871/activity", + "actor": "https://channels.tests.funkwhale.audio/federation/actors/compositions", + "object": { + "id": "https://channels.tests.funkwhale.audio/federation/music/uploads/42342395-0208-4fee-a38d-259a6dae0871", + "type": "Audio", + "name": "Compositions - Test Audio for Pleroma", + "attributedTo": "https://channels.tests.funkwhale.audio/federation/actors/compositions", + "published": "2020-03-11T10:01:52.714918+00:00", + "to": "https://www.w3.org/ns/activitystreams#Public", + "url": [ + { + "type": "Link", + "mimeType": "audio/ogg", + "href": "https://channels.tests.funkwhale.audio/api/v1/listen/3901e5d8-0445-49d5-9711-e096cf32e515/?upload=42342395-0208-4fee-a38d-259a6dae0871&download=false" + }, + { + "type": "Link", + "mimeType": "text/html", + "href": "https://channels.tests.funkwhale.audio/library/tracks/74" + } + ], + "content": "<p>This is a test Audio for Pleroma.</p>", + "mediaType": "text/html", + "tag": [ + { + "type": "Hashtag", + "name": "#funkwhale" + }, + { + "type": "Hashtag", + "name": "#test" + }, + { + "type": "Hashtag", + "name": "#tests" + } + ], + "summary": "#funkwhale #test #tests", + "@context": [ + "https://www.w3.org/ns/activitystreams", + "https://w3id.org/security/v1", + { + "manuallyApprovesFollowers": "as:manuallyApprovesFollowers" + } + ] + } +} diff --git a/test/plugs/frontend_static_test.exs b/test/plugs/frontend_static_test.exs index d11d91d78..6f4923048 100644 --- a/test/plugs/frontend_static_test.exs +++ b/test/plugs/frontend_static_test.exs @@ -3,6 +3,7 @@ # SPDX-License-Identifier: AGPL-3.0-only defmodule Pleroma.Web.FrontendStaticPlugTest do + alias Pleroma.Plugs.FrontendStatic use Pleroma.Web.ConnCase @dir "test/tmp/instance_static" @@ -14,6 +15,18 @@ defmodule Pleroma.Web.FrontendStaticPlugTest do setup do: clear_config([:instance, :static_dir], @dir) + test "init will give a static plug config + the frontend type" do + opts = + [ + at: "/admin", + frontend_type: :admin + ] + |> FrontendStatic.init() + + assert opts[:at] == ["admin"] + assert opts[:frontend_type] == :admin + end + test "overrides existing static files", %{conn: conn} do name = "pelmora" ref = "uguu" @@ -27,4 +40,18 @@ defmodule Pleroma.Web.FrontendStaticPlugTest do index = get(conn, "/") assert html_response(index, 200) == "from frontend plug" end + + test "overrides existing static files for the `pleroma/admin` path", %{conn: conn} do + name = "pelmora" + ref = "uguu" + + clear_config([:frontends, :admin], %{"name" => name, "ref" => ref}) + path = "#{@dir}/frontends/#{name}/#{ref}" + + File.mkdir_p!(path) + File.write!("#{path}/index.html", "from frontend plug") + + index = get(conn, "/pleroma/admin/") + assert html_response(index, 200) == "from frontend plug" + end end diff --git a/test/tasks/digest_test.exs b/test/tasks/digest_test.exs index eefbc8936..0b444c86d 100644 --- a/test/tasks/digest_test.exs +++ b/test/tasks/digest_test.exs @@ -17,6 +17,8 @@ defmodule Mix.Tasks.Pleroma.DigestTest do :ok end + setup do: clear_config([Pleroma.Emails.Mailer, :enabled], true) + describe "pleroma.digest test" do test "Sends digest to the given user" do user1 = insert(:user) diff --git a/test/tasks/email_test.exs b/test/tasks/email_test.exs index 944c07064..c3af7ef68 100644 --- a/test/tasks/email_test.exs +++ b/test/tasks/email_test.exs @@ -16,6 +16,8 @@ defmodule Mix.Tasks.Pleroma.EmailTest do :ok end + setup do: clear_config([Pleroma.Emails.Mailer, :enabled], true) + describe "pleroma.email test" do test "Sends test email with no given address" do mail_to = Config.get([:instance, :email]) diff --git a/test/tasks/frontend_test.exs b/test/tasks/frontend_test.exs new file mode 100644 index 000000000..0ca2b9a28 --- /dev/null +++ b/test/tasks/frontend_test.exs @@ -0,0 +1,78 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.FrontendTest do + use Pleroma.DataCase + alias Mix.Tasks.Pleroma.Frontend + + import ExUnit.CaptureIO, only: [capture_io: 1] + + @dir "test/frontend_static_test" + + setup do + File.mkdir_p!(@dir) + clear_config([:instance, :static_dir], @dir) + + on_exit(fn -> + File.rm_rf(@dir) + end) + end + + test "it downloads and unzips a known frontend" do + clear_config([:frontends, :available], %{ + "pleroma" => %{ + "ref" => "fantasy", + "name" => "pleroma", + "build_url" => "http://gensokyo.2hu/builds/${ref}" + } + }) + + Tesla.Mock.mock(fn %{url: "http://gensokyo.2hu/builds/fantasy"} -> + %Tesla.Env{status: 200, body: File.read!("test/fixtures/tesla_mock/frontend_dist.zip")} + end) + + capture_io(fn -> + Frontend.run(["install", "pleroma"]) + end) + + assert File.exists?(Path.join([@dir, "frontends", "pleroma", "fantasy", "test.txt"])) + end + + test "it also works given a file" do + clear_config([:frontends, :available], %{ + "pleroma" => %{ + "ref" => "fantasy", + "name" => "pleroma", + "build_dir" => "" + } + }) + + capture_io(fn -> + Frontend.run(["install", "pleroma", "--file", "test/fixtures/tesla_mock/frontend.zip"]) + end) + + assert File.exists?(Path.join([@dir, "frontends", "pleroma", "fantasy", "test.txt"])) + end + + test "it downloads and unzips unknown frontends" do + Tesla.Mock.mock(fn %{url: "http://gensokyo.2hu/madeup.zip"} -> + %Tesla.Env{status: 200, body: File.read!("test/fixtures/tesla_mock/frontend.zip")} + end) + + capture_io(fn -> + Frontend.run([ + "install", + "unknown", + "--ref", + "baka", + "--build-url", + "http://gensokyo.2hu/madeup.zip", + "--build-dir", + "" + ]) + end) + + assert File.exists?(Path.join([@dir, "frontends", "unknown", "baka", "test.txt"])) + end +end diff --git a/test/tasks/relay_test.exs b/test/tasks/relay_test.exs index 79ab72002..e5225b64c 100644 --- a/test/tasks/relay_test.exs +++ b/test/tasks/relay_test.exs @@ -42,7 +42,11 @@ defmodule Mix.Tasks.Pleroma.RelayTest do assert activity.data["object"] == target_user.ap_id :ok = Mix.Tasks.Pleroma.Relay.run(["list"]) - assert_receive {:mix_shell, :info, ["mastodon.example.org (no Accept received)"]} + + assert_receive {:mix_shell, :info, + [ + "http://mastodon.example.org/users/admin - no Accept received (relay didn't follow back)" + ]} end end @@ -95,8 +99,8 @@ defmodule Mix.Tasks.Pleroma.RelayTest do :ok = Mix.Tasks.Pleroma.Relay.run(["list"]) - assert_receive {:mix_shell, :info, ["mstdn.io"]} - assert_receive {:mix_shell, :info, ["mastodon.example.org"]} + assert_receive {:mix_shell, :info, ["https://mstdn.io/users/mayuutann"]} + assert_receive {:mix_shell, :info, ["http://mastodon.example.org/users/admin"]} end end end diff --git a/test/user_test.exs b/test/user_test.exs index b47405895..3cf248659 100644 --- a/test/user_test.exs +++ b/test/user_test.exs @@ -1417,7 +1417,6 @@ defmodule Pleroma.UserTest do test "delete/1 when approval is pending deletes the user" do user = insert(:user, approval_pending: true) - {:ok, user: user} {:ok, job} = User.delete(user) {:ok, _} = ObanHelpers.perform(job) @@ -1426,6 +1425,85 @@ defmodule Pleroma.UserTest do refute User.get_by_id(user.id) end + test "delete/1 purges a user when they wouldn't be fully deleted" do + user = + insert(:user, %{ + bio: "eyy lmao", + name: "qqqqqqq", + password_hash: "pdfk2$1b3n159001", + keys: "RSA begin buplic key", + public_key: "--PRIVATE KEYE--", + avatar: %{"a" => "b"}, + tags: ["qqqqq"], + banner: %{"a" => "b"}, + background: %{"a" => "b"}, + note_count: 9, + follower_count: 9, + following_count: 9001, + locked: true, + confirmation_pending: true, + password_reset_pending: true, + approval_pending: true, + registration_reason: "ahhhhh", + confirmation_token: "qqqq", + domain_blocks: ["lain.com"], + deactivated: true, + ap_enabled: true, + is_moderator: true, + is_admin: true, + mastofe_settings: %{"a" => "b"}, + mascot: %{"a" => "b"}, + emoji: %{"a" => "b"}, + pleroma_settings_store: %{"q" => "x"}, + fields: [%{"gg" => "qq"}], + raw_fields: [%{"gg" => "qq"}], + discoverable: true, + also_known_as: ["https://lol.olo/users/loll"] + }) + + {:ok, job} = User.delete(user) + {:ok, _} = ObanHelpers.perform(job) + user = User.get_by_id(user.id) + + assert %User{ + bio: nil, + raw_bio: nil, + email: nil, + name: nil, + password_hash: nil, + keys: nil, + public_key: nil, + avatar: %{}, + tags: [], + last_refreshed_at: nil, + last_digest_emailed_at: nil, + banner: %{}, + background: %{}, + note_count: 0, + follower_count: 0, + following_count: 0, + locked: false, + confirmation_pending: false, + password_reset_pending: false, + approval_pending: false, + registration_reason: nil, + confirmation_token: nil, + domain_blocks: [], + deactivated: true, + ap_enabled: false, + is_moderator: false, + is_admin: false, + mastofe_settings: nil, + mascot: nil, + emoji: %{}, + pleroma_settings_store: %{}, + fields: [], + raw_fields: [], + discoverable: false, + also_known_as: [] + } = user + end + test "get_public_key_for_ap_id fetches a user that's not in the db" do assert {:ok, _key} = User.get_public_key_for_ap_id("http://mastodon.example.org/users/admin") end diff --git a/test/web/activity_pub/activity_pub_controller_test.exs b/test/web/activity_pub/activity_pub_controller_test.exs index ed900d8f8..57988dc1e 100644 --- a/test/web/activity_pub/activity_pub_controller_test.exs +++ b/test/web/activity_pub/activity_pub_controller_test.exs @@ -533,7 +533,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubControllerTest do end) :ok = Mix.Tasks.Pleroma.Relay.run(["list"]) - assert_receive {:mix_shell, :info, ["relay.mastodon.host"]} + assert_receive {:mix_shell, :info, ["https://relay.mastodon.host/actor"]} end @tag capture_log: true diff --git a/test/web/activity_pub/activity_pub_test.exs b/test/web/activity_pub/activity_pub_test.exs index d6eab7337..03f968aaf 100644 --- a/test/web/activity_pub/activity_pub_test.exs +++ b/test/web/activity_pub/activity_pub_test.exs @@ -990,13 +990,39 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubTest do end describe "uploading files" do - test "copies the file to the configured folder" do - file = %Plug.Upload{ + setup do + test_file = %Plug.Upload{ content_type: "image/jpg", path: Path.absname("test/fixtures/image.jpg"), filename: "an_image.jpg" } + %{test_file: test_file} + end + + test "sets a description if given", %{test_file: file} do + {:ok, %Object{} = object} = ActivityPub.upload(file, description: "a cool file") + assert object.data["name"] == "a cool file" + end + + test "it sets the default description depending on the configuration", %{test_file: file} do + clear_config([Pleroma.Upload, :default_description]) + + Pleroma.Config.put([Pleroma.Upload, :default_description], nil) + {:ok, %Object{} = object} = ActivityPub.upload(file) + assert object.data["name"] == "" + + Pleroma.Config.put([Pleroma.Upload, :default_description], :filename) + {:ok, %Object{} = object} = ActivityPub.upload(file) + assert object.data["name"] == "an_image.jpg" + + Pleroma.Config.put([Pleroma.Upload, :default_description], "unnamed attachment") + {:ok, %Object{} = object} = ActivityPub.upload(file) + assert object.data["name"] == "unnamed attachment" + end + + test "copies the file to the configured folder", %{test_file: file} do + clear_config([Pleroma.Upload, :default_description], :filename) {:ok, %Object{} = object} = ActivityPub.upload(file) assert object.data["name"] == "an_image.jpg" end diff --git a/test/web/activity_pub/transmogrifier/audio_handling_test.exs b/test/web/activity_pub/transmogrifier/audio_handling_test.exs new file mode 100644 index 000000000..0636d00c5 --- /dev/null +++ b/test/web/activity_pub/transmogrifier/audio_handling_test.exs @@ -0,0 +1,83 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.ActivityPub.Transmogrifier.AudioHandlingTest do + use Oban.Testing, repo: Pleroma.Repo + use Pleroma.DataCase + + alias Pleroma.Activity + alias Pleroma.Object + alias Pleroma.Web.ActivityPub.Transmogrifier + + import Pleroma.Factory + + test "it works for incoming listens" do + _user = insert(:user, ap_id: "http://mastodon.example.org/users/admin") + + data = %{ + "@context" => "https://www.w3.org/ns/activitystreams", + "to" => ["https://www.w3.org/ns/activitystreams#Public"], + "cc" => [], + "type" => "Listen", + "id" => "http://mastodon.example.org/users/admin/listens/1234/activity", + "actor" => "http://mastodon.example.org/users/admin", + "object" => %{ + "type" => "Audio", + "id" => "http://mastodon.example.org/users/admin/listens/1234", + "attributedTo" => "http://mastodon.example.org/users/admin", + "title" => "lain radio episode 1", + "artist" => "lain", + "album" => "lain radio", + "length" => 180_000 + } + } + + {:ok, %Activity{local: false} = activity} = Transmogrifier.handle_incoming(data) + + object = Object.normalize(activity) + + assert object.data["title"] == "lain radio episode 1" + assert object.data["artist"] == "lain" + assert object.data["album"] == "lain radio" + assert object.data["length"] == 180_000 + end + + test "Funkwhale Audio object" do + Tesla.Mock.mock(fn + %{url: "https://channels.tests.funkwhale.audio/federation/actors/compositions"} -> + %Tesla.Env{ + status: 200, + body: File.read!("test/fixtures/tesla_mock/funkwhale_channel.json") + } + end) + + data = File.read!("test/fixtures/tesla_mock/funkwhale_create_audio.json") |> Poison.decode!() + + {:ok, %Activity{local: false} = activity} = Transmogrifier.handle_incoming(data) + + assert object = Object.normalize(activity, false) + + assert object.data["to"] == ["https://www.w3.org/ns/activitystreams#Public"] + + assert object.data["cc"] == [] + + assert object.data["url"] == "https://channels.tests.funkwhale.audio/library/tracks/74" + + assert object.data["attachment"] == [ + %{ + "mediaType" => "audio/ogg", + "type" => "Link", + "name" => nil, + "url" => [ + %{ + "href" => + "https://channels.tests.funkwhale.audio/api/v1/listen/3901e5d8-0445-49d5-9711-e096cf32e515/?upload=42342395-0208-4fee-a38d-259a6dae0871&download=false", + "mediaType" => "audio/ogg", + "type" => "Link" + } + ] + } + ] + end +end diff --git a/test/web/activity_pub/transmogrifier/event_handling_test.exs b/test/web/activity_pub/transmogrifier/event_handling_test.exs new file mode 100644 index 000000000..7f1ef2cbd --- /dev/null +++ b/test/web/activity_pub/transmogrifier/event_handling_test.exs @@ -0,0 +1,40 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.ActivityPub.Transmogrifier.EventHandlingTest do + use Oban.Testing, repo: Pleroma.Repo + use Pleroma.DataCase + + alias Pleroma.Object.Fetcher + + test "Mobilizon Event object" do + Tesla.Mock.mock(fn + %{url: "https://mobilizon.org/events/252d5816-00a3-4a89-a66f-15bf65c33e39"} -> + %Tesla.Env{ + status: 200, + body: File.read!("test/fixtures/tesla_mock/mobilizon.org-event.json") + } + + %{url: "https://mobilizon.org/@tcit"} -> + %Tesla.Env{ + status: 200, + body: File.read!("test/fixtures/tesla_mock/mobilizon.org-user.json") + } + end) + + assert {:ok, object} = + Fetcher.fetch_object_from_id( + "https://mobilizon.org/events/252d5816-00a3-4a89-a66f-15bf65c33e39" + ) + + assert object.data["to"] == ["https://www.w3.org/ns/activitystreams#Public"] + assert object.data["cc"] == [] + + assert object.data["url"] == + "https://mobilizon.org/events/252d5816-00a3-4a89-a66f-15bf65c33e39" + + assert object.data["published"] == "2019-12-17T11:33:56Z" + assert object.data["name"] == "Mobilizon Launching Party" + end +end diff --git a/test/web/activity_pub/transmogrifier/question_handling_test.exs b/test/web/activity_pub/transmogrifier/question_handling_test.exs index 9fb965d7f..c82361828 100644 --- a/test/web/activity_pub/transmogrifier/question_handling_test.exs +++ b/test/web/activity_pub/transmogrifier/question_handling_test.exs @@ -24,6 +24,8 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier.QuestionHandlingTest do object = Object.normalize(activity, false) + assert object.data["url"] == "https://mastodon.sdf.org/@rinpatch/102070944809637304" + assert object.data["closed"] == "2019-05-11T09:03:36Z" assert object.data["context"] == activity.data["context"] diff --git a/test/web/activity_pub/transmogrifier_test.exs b/test/web/activity_pub/transmogrifier_test.exs index 0dd4e6e47..3fa41b0c7 100644 --- a/test/web/activity_pub/transmogrifier_test.exs +++ b/test/web/activity_pub/transmogrifier_test.exs @@ -225,35 +225,6 @@ defmodule Pleroma.Web.ActivityPub.TransmogrifierTest do assert Enum.at(object.data["tag"], 2) == "moo" end - test "it works for incoming listens" do - data = %{ - "@context" => "https://www.w3.org/ns/activitystreams", - "to" => ["https://www.w3.org/ns/activitystreams#Public"], - "cc" => [], - "type" => "Listen", - "id" => "http://mastodon.example.org/users/admin/listens/1234/activity", - "actor" => "http://mastodon.example.org/users/admin", - "object" => %{ - "type" => "Audio", - "id" => "http://mastodon.example.org/users/admin/listens/1234", - "attributedTo" => "http://mastodon.example.org/users/admin", - "title" => "lain radio episode 1", - "artist" => "lain", - "album" => "lain radio", - "length" => 180_000 - } - } - - {:ok, %Activity{local: false} = activity} = Transmogrifier.handle_incoming(data) - - object = Object.normalize(activity) - - assert object.data["title"] == "lain radio episode 1" - assert object.data["artist"] == "lain" - assert object.data["album"] == "lain radio" - assert object.data["length"] == 180_000 - end - test "it works for incoming notices with contentMap" do data = File.read!("test/fixtures/mastodon-post-activity-contentmap.json") |> Poison.decode!() diff --git a/test/web/admin_api/controllers/admin_api_controller_test.exs b/test/web/admin_api/controllers/admin_api_controller_test.exs index 66d4b1ef3..dbf478edf 100644 --- a/test/web/admin_api/controllers/admin_api_controller_test.exs +++ b/test/web/admin_api/controllers/admin_api_controller_test.exs @@ -155,13 +155,30 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do describe "DELETE /api/pleroma/admin/users" do test "single user", %{admin: admin, conn: conn} do - user = insert(:user) clear_config([:instance, :federating], true) + user = + insert(:user, + avatar: %{"url" => [%{"href" => "https://someurl"}]}, + banner: %{"url" => [%{"href" => "https://somebanner"}]}, + bio: "Hello world!", + name: "A guy" + ) + + # Create some activities to check they got deleted later + follower = insert(:user) + {:ok, _} = CommonAPI.post(user, %{status: "test"}) + {:ok, _, _, _} = CommonAPI.follow(user, follower) + {:ok, _, _, _} = CommonAPI.follow(follower, user) + user = Repo.get(User, user.id) + assert user.note_count == 1 + assert user.follower_count == 1 + assert user.following_count == 1 refute user.deactivated with_mock Pleroma.Web.Federator, - publish: fn _ -> nil end do + publish: fn _ -> nil end, + perform: fn _, _ -> nil end do conn = conn |> put_req_header("accept", "application/json") @@ -181,6 +198,14 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do user = Repo.get(User, user.id) assert user.deactivated + assert user.avatar == %{} + assert user.banner == %{} + assert user.note_count == 0 + assert user.follower_count == 0 + assert user.following_count == 0 + assert user.bio == nil + assert user.name == nil + assert called(Pleroma.Web.Federator.publish(:_)) end end @@ -356,7 +381,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => user.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" } assert expected == json_response(conn, 200) @@ -638,7 +664,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => admin.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" }, %{ "deactivated" => user.deactivated, @@ -652,7 +679,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => user.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" }, %{ "deactivated" => user2.deactivated, @@ -666,7 +694,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => true, "url" => user2.ap_id, - "registration_reason" => "I'm a chill dude" + "registration_reason" => "I'm a chill dude", + "actor_type" => "Person" } ] |> Enum.sort_by(& &1["nickname"]) @@ -741,7 +770,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => user.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" } ] } @@ -769,7 +799,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => user.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" } ] } @@ -797,7 +828,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => user.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" } ] } @@ -825,7 +857,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => user.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" } ] } @@ -853,7 +886,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => user.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" } ] } @@ -881,7 +915,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => user.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" } ] } @@ -904,7 +939,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => user2.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" } ] } @@ -939,7 +975,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => user.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" } ] } @@ -967,7 +1004,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => user.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" }, %{ "deactivated" => admin.deactivated, @@ -981,7 +1019,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => admin.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" }, %{ "deactivated" => false, @@ -995,7 +1034,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => old_admin.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" } ] |> Enum.sort_by(& &1["nickname"]) @@ -1033,7 +1073,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => true, "url" => user.ap_id, - "registration_reason" => "Plz let me in!" + "registration_reason" => "Plz let me in!", + "actor_type" => "Person" } ] |> Enum.sort_by(& &1["nickname"]) @@ -1066,7 +1107,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => admin.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" }, %{ "deactivated" => false, @@ -1080,7 +1122,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => second_admin.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" } ] |> Enum.sort_by(& &1["nickname"]) @@ -1115,7 +1158,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => moderator.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" } ] } @@ -1143,7 +1187,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => user1.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" }, %{ "deactivated" => false, @@ -1157,7 +1202,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => user2.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" } ] |> Enum.sort_by(& &1["nickname"]) @@ -1220,7 +1266,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => user.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" } ] } @@ -1247,7 +1294,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => admin.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" } ] } @@ -1332,7 +1380,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do "confirmation_pending" => false, "approval_pending" => false, "url" => user.ap_id, - "registration_reason" => nil + "registration_reason" => nil, + "actor_type" => "Person" } log_entry = Repo.one(ModerationLog) diff --git a/test/web/admin_api/controllers/media_proxy_cache_controller_test.exs b/test/web/admin_api/controllers/media_proxy_cache_controller_test.exs index 5ab6cb78a..f243d1fb2 100644 --- a/test/web/admin_api/controllers/media_proxy_cache_controller_test.exs +++ b/test/web/admin_api/controllers/media_proxy_cache_controller_test.exs @@ -48,6 +48,9 @@ defmodule Pleroma.Web.AdminAPI.MediaProxyCacheControllerTest do |> get("/api/pleroma/admin/media_proxy_caches?page_size=2") |> json_response_and_validate_schema(200) + assert response["page_size"] == 2 + assert response["count"] == 5 + assert response["urls"] == [ "http://localhost:4001/media/fb1f4d.jpg", "http://localhost:4001/media/a688346.jpg" @@ -63,6 +66,9 @@ defmodule Pleroma.Web.AdminAPI.MediaProxyCacheControllerTest do "http://localhost:4001/media/tb13f47.jpg" ] + assert response["page_size"] == 2 + assert response["count"] == 5 + response = conn |> get("/api/pleroma/admin/media_proxy_caches?page_size=2&page=3") @@ -70,6 +76,30 @@ defmodule Pleroma.Web.AdminAPI.MediaProxyCacheControllerTest do assert response["urls"] == ["http://localhost:4001/media/wb1f46.jpg"] end + + test "search banned MediaProxy URLs", %{conn: conn} do + MediaProxy.put_in_banned_urls([ + "http://localhost:4001/media/a688346.jpg", + "http://localhost:4001/media/ff44b1f4d.jpg" + ]) + + MediaProxy.put_in_banned_urls("http://localhost:4001/media/gb1f44.jpg") + MediaProxy.put_in_banned_urls("http://localhost:4001/media/tb13f47.jpg") + MediaProxy.put_in_banned_urls("http://localhost:4001/media/wb1f46.jpg") + + response = + conn + |> get("/api/pleroma/admin/media_proxy_caches?page_size=2&query=F44") + |> json_response_and_validate_schema(200) + + assert response["urls"] == [ + "http://localhost:4001/media/gb1f44.jpg", + "http://localhost:4001/media/ff44b1f4d.jpg" + ] + + assert response["page_size"] == 2 + assert response["count"] == 2 + end end describe "POST /api/pleroma/admin/media_proxy_caches/delete" do @@ -79,15 +109,13 @@ defmodule Pleroma.Web.AdminAPI.MediaProxyCacheControllerTest do "http://localhost:4001/media/fb1f4d.jpg" ]) - response = - conn - |> put_req_header("content-type", "application/json") - |> post("/api/pleroma/admin/media_proxy_caches/delete", %{ - urls: ["http://localhost:4001/media/a688346.jpg"] - }) - |> json_response_and_validate_schema(200) + conn + |> put_req_header("content-type", "application/json") + |> post("/api/pleroma/admin/media_proxy_caches/delete", %{ + urls: ["http://localhost:4001/media/a688346.jpg"] + }) + |> json_response_and_validate_schema(200) - assert response["urls"] == ["http://localhost:4001/media/a688346.jpg"] refute MediaProxy.in_banned_urls("http://localhost:4001/media/a688346.jpg") assert MediaProxy.in_banned_urls("http://localhost:4001/media/fb1f4d.jpg") end @@ -106,13 +134,10 @@ defmodule Pleroma.Web.AdminAPI.MediaProxyCacheControllerTest do purge: fn _, _ -> {"ok", 0} end ]} ] do - response = - conn - |> put_req_header("content-type", "application/json") - |> post("/api/pleroma/admin/media_proxy_caches/purge", %{urls: urls, ban: false}) - |> json_response_and_validate_schema(200) - - assert response["urls"] == urls + conn + |> put_req_header("content-type", "application/json") + |> post("/api/pleroma/admin/media_proxy_caches/purge", %{urls: urls, ban: false}) + |> json_response_and_validate_schema(200) refute MediaProxy.in_banned_urls("http://example.com/media/a688346.jpg") refute MediaProxy.in_banned_urls("http://example.com/media/fb1f4d.jpg") @@ -126,16 +151,13 @@ defmodule Pleroma.Web.AdminAPI.MediaProxyCacheControllerTest do ] with_mocks [{MediaProxy.Invalidation.Script, [], [purge: fn _, _ -> {"ok", 0} end]}] do - response = - conn - |> put_req_header("content-type", "application/json") - |> post("/api/pleroma/admin/media_proxy_caches/purge", %{ - urls: urls, - ban: true - }) - |> json_response_and_validate_schema(200) - - assert response["urls"] == urls + conn + |> put_req_header("content-type", "application/json") + |> post( + "/api/pleroma/admin/media_proxy_caches/purge", + %{urls: urls, ban: true} + ) + |> json_response_and_validate_schema(200) assert MediaProxy.in_banned_urls("http://example.com/media/a688346.jpg") assert MediaProxy.in_banned_urls("http://example.com/media/fb1f4d.jpg") diff --git a/test/web/admin_api/controllers/relay_controller_test.exs b/test/web/admin_api/controllers/relay_controller_test.exs index 64086adc5..adadf2b5c 100644 --- a/test/web/admin_api/controllers/relay_controller_test.exs +++ b/test/web/admin_api/controllers/relay_controller_test.exs @@ -39,8 +39,10 @@ defmodule Pleroma.Web.AdminAPI.RelayControllerTest do relay_url: "http://mastodon.example.org/users/admin" }) - assert json_response_and_validate_schema(conn, 200) == - "http://mastodon.example.org/users/admin" + assert json_response_and_validate_schema(conn, 200) == %{ + "actor" => "http://mastodon.example.org/users/admin", + "followed_back" => false + } log_entry = Repo.one(ModerationLog) @@ -59,8 +61,13 @@ defmodule Pleroma.Web.AdminAPI.RelayControllerTest do conn = get(conn, "/api/pleroma/admin/relay") - assert json_response_and_validate_schema(conn, 200)["relays"] -- - ["mastodon.example.org", "mstdn.io"] == [] + assert json_response_and_validate_schema(conn, 200)["relays"] == [ + %{ + "actor" => "http://mastodon.example.org/users/admin", + "followed_back" => true + }, + %{"actor" => "https://mstdn.io/users/mayuutann", "followed_back" => true} + ] end test "DELETE /relay", %{conn: conn, admin: admin} do diff --git a/test/web/mastodon_api/controllers/timeline_controller_test.exs b/test/web/mastodon_api/controllers/timeline_controller_test.exs index 50e0d783d..71bac99f7 100644 --- a/test/web/mastodon_api/controllers/timeline_controller_test.exs +++ b/test/web/mastodon_api/controllers/timeline_controller_test.exs @@ -445,6 +445,23 @@ defmodule Pleroma.Web.MastodonAPI.TimelineControllerTest do assert length(json_response(res_conn, 200)) == 2 end + test "with default settings on private instances, returns 403 for unauthenticated users", %{ + conn: conn, + base_uri: base_uri, + error_response: error_response + } do + clear_config([:instance, :public], false) + clear_config([:restrict_unauthenticated, :timelines]) + + for local <- [true, false] do + res_conn = get(conn, "#{base_uri}?local=#{local}") + + assert json_response(res_conn, :unauthorized) == error_response + end + + ensure_authenticated_access(base_uri) + end + test "with `%{local: true, federated: true}`, returns 403 for unauthenticated users", %{ conn: conn, base_uri: base_uri, diff --git a/test/web/mastodon_api/views/status_view_test.exs b/test/web/mastodon_api/views/status_view_test.exs index 8703d5ba7..70d829979 100644 --- a/test/web/mastodon_api/views/status_view_test.exs +++ b/test/web/mastodon_api/views/status_view_test.exs @@ -517,6 +517,12 @@ defmodule Pleroma.Web.MastodonAPI.StatusViewTest do represented = StatusView.render("show.json", %{for: user, activity: activity}) assert represented[:id] == to_string(activity.id) + + assert represented[:url] == + "https://mobilizon.org/events/252d5816-00a3-4a89-a66f-15bf65c33e39" + + assert represented[:content] == + "<p><a href=\"https://mobilizon.org/events/252d5816-00a3-4a89-a66f-15bf65c33e39\">Mobilizon Launching Party</a></p><p>Mobilizon is now federated! 🎉</p><p></p><p>You can view this event from other instances if they are subscribed to mobilizon.org, and soon directly from Mastodon and Pleroma. It is possible that you may see some comments from other instances, including Mastodon ones, just below.</p><p></p><p>With a Mobilizon account on an instance, you may <strong>participate</strong> at events from other instances and <strong>add comments</strong> on events.</p><p></p><p>Of course, it's still <u>a work in progress</u>: if reports made from an instance on events and comments can be federated, you can't block people right now, and moderators actions are rather limited, but this <strong>will definitely get fixed over time</strong> until first stable version next year.</p><p></p><p>Anyway, if you want to come up with some feedback, head over to our forum or - if you feel you have technical skills and are familiar with it - on our Gitlab repository.</p><p></p><p>Also, to people that want to set Mobilizon themselves even though we really don't advise to do that for now, we have a little documentation but it's quite the early days and you'll probably need some help. No worries, you can chat with us on our Forum or though our Matrix channel.</p><p></p><p>Check our website for more informations and follow us on Twitter or Mastodon.</p>" end describe "build_tags/1" do diff --git a/test/web/preload/timeline_test.exs b/test/web/preload/timeline_test.exs index fea95a6a4..3b1f2f1aa 100644 --- a/test/web/preload/timeline_test.exs +++ b/test/web/preload/timeline_test.exs @@ -12,16 +12,8 @@ defmodule Pleroma.Web.Preload.Providers.TimelineTest do @public_url "/api/v1/timelines/public" describe "unauthenticated timeliness when restricted" do - setup do - svd_config = Pleroma.Config.get([:restrict_unauthenticated, :timelines]) - Pleroma.Config.put([:restrict_unauthenticated, :timelines], %{local: true, federated: true}) - - on_exit(fn -> - Pleroma.Config.put([:restrict_unauthenticated, :timelines], svd_config) - end) - - :ok - end + setup do: clear_config([:restrict_unauthenticated, :timelines, :local], true) + setup do: clear_config([:restrict_unauthenticated, :timelines, :federated], true) test "return nothing" do tl_data = Timelines.generate_terms(%{}) @@ -31,20 +23,10 @@ defmodule Pleroma.Web.Preload.Providers.TimelineTest do end describe "unauthenticated timeliness when unrestricted" do - setup do - svd_config = Pleroma.Config.get([:restrict_unauthenticated, :timelines]) + setup do: clear_config([:restrict_unauthenticated, :timelines, :local], false) + setup do: clear_config([:restrict_unauthenticated, :timelines, :federated], false) - Pleroma.Config.put([:restrict_unauthenticated, :timelines], %{ - local: false, - federated: false - }) - - on_exit(fn -> - Pleroma.Config.put([:restrict_unauthenticated, :timelines], svd_config) - end) - - {:ok, user: insert(:user)} - end + setup do: {:ok, user: insert(:user)} test "returns the timeline when not restricted" do assert Timelines.generate_terms(%{}) diff --git a/test/web/twitter_api/util_controller_test.exs b/test/web/twitter_api/util_controller_test.exs index 109c1e637..354d77b56 100644 --- a/test/web/twitter_api/util_controller_test.exs +++ b/test/web/twitter_api/util_controller_test.exs @@ -586,10 +586,16 @@ defmodule Pleroma.Web.TwitterAPI.UtilControllerTest do end end - test "with proper permissions and valid password", %{conn: conn} do + test "with proper permissions and valid password", %{conn: conn, user: user} do conn = post(conn, "/api/pleroma/delete_account", %{"password" => "test"}) - + ObanHelpers.perform_all() assert json_response(conn, 200) == %{"status" => "success"} + + user = User.get_by_id(user.id) + assert user.deactivated == true + assert user.name == nil + assert user.bio == nil + assert user.password_hash == nil end end end diff --git a/test/web/web_finger/web_finger_test.exs b/test/web/web_finger/web_finger_test.exs index f4884e0a2..96fc0bbaa 100644 --- a/test/web/web_finger/web_finger_test.exs +++ b/test/web/web_finger/web_finger_test.exs @@ -40,6 +40,11 @@ defmodule Pleroma.Web.WebFingerTest do end describe "fingering" do + test "returns error for nonsensical input" do + assert {:error, _} = WebFinger.finger("bliblablu") + assert {:error, _} = WebFinger.finger("pleroma.social") + end + test "returns error when fails parse xml or json" do user = "invalid_content@social.heldscal.la" assert {:error, %Jason.DecodeError{}} = WebFinger.finger(user) |