Merge branch 'relay-fix-admin-fe' into 'develop'

Relay fix for admin-fe

See merge request pleroma/pleroma!2902

13 files changed, 149 insertions, 97 deletions

diff --git a/CHANGELOG.md b/CHANGELOG.md
index cdc0cd8ae..1ed8445b5 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -6,19 +6,20 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 ## [unreleased]
 
 ### Changed
+
 - **Breaking:** The default descriptions on uploads are now empty. The old behavior (filename as default) can be configured, see the cheat sheet.
 - **Breaking:** Added the ObjectAgePolicy to the default set of MRFs. This will delist and strip the follower collection of any message received that is older than 7 days. This will stop users from seeing very old messages in the timelines. The messages can still be viewed on the user's page and in conversations. They also still trigger notifications.
 - **Breaking:** Elixir >=1.9 is now required (was >= 1.8)
 - **Breaking:** Configuration: `:auto_linker, :opts` moved to `:pleroma, Pleroma.Formatter`. Old config namespace is deprecated.
+- **Breaking:** Configuration: `:instance, welcome_user_nickname` moved to `:welcome, :direct_message, :sender_nickname`, `:instance, :welcome_message` moved to `:welcome, :direct_message, :message`. Old config namespace is deprecated.
+- **Breaking:** LDAP: Fallback to local database authentication has been removed for security reasons and lack of a mechanism to ensure the passwords are synchronized when LDAP passwords are updated.
+- **Breaking** Changed defaults for `:restrict_unauthenticated` so that when `:instance, :public` is set to `false` then all `:restrict_unauthenticated` items be effectively set to `true`. If you'd like to allow unauthenticated access to specific API endpoints on a private instance, please explicitly set `:restrict_unauthenticated` to non-default value in `config/prod.secret.exs`.
 - In Conversations, return only direct messages as `last_status`
 - Using the `only_media` filter on timelines will now exclude reblog media
 - MFR policy to set global expiration for all local Create activities
 - OGP rich media parser merged with TwitterCard
 - Configuration: `:instance, rewrite_policy` moved to `:mrf, policies`, `:instance, :mrf_transparency` moved to `:mrf, :transparency`, `:instance, :mrf_transparency_exclusions` moved to `:mrf, :transparency_exclusions`. Old config namespace is deprecated.
 - Configuration: `:media_proxy, whitelist` format changed to host with scheme (e.g. `http://example.com` instead of `example.com`). Domain format is deprecated.
-- **Breaking:** Configuration: `:instance, welcome_user_nickname` moved to `:welcome, :direct_message, :sender_nickname`, `:instance, :welcome_message` moved to `:welcome, :direct_message, :message`. Old config namespace is deprecated.
-- **Breaking:** LDAP: Fallback to local database authentication has been removed for security reasons and lack of a mechanism to ensure the passwords are synchronized when LDAP passwords are updated.
-- **Breaking** Changed defaults for `:restrict_unauthenticated` so that when `:instance, :public` is set to `false` then all `:restrict_unauthenticated` items be effectively set to `true`. If you'd like to allow unauthenticated access to specific API endpoints on a private instance, please explicitly set `:restrict_unauthenticated` to non-default value in `config/prod.secret.exs`. 
 
 <details>
   <summary>API Changes</summary>
@@ -26,29 +27,30 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - **Breaking:** Pleroma API: The routes to update avatar, banner and background have been removed.
 - **Breaking:** Image description length is limited now.
 - **Breaking:** Emoji API: changed methods and renamed routes.
+- **Breaking:** Notification Settings API for suppressing notifications has been simplified down to `block_from_strangers`.
+- **Breaking:** Notification Settings API option for hiding push notification contents has been renamed to `hide_notification_contents`.
 - MastodonAPI: Allow removal of avatar, banner and background.
 - Streaming: Repeats of a user's posts will no longer be pushed to the user's stream.
 - Mastodon API: Added `pleroma.metadata.fields_limits` to /api/v1/instance
 - Mastodon API: On deletion, returns the original post text.
 - Mastodon API: Add `pleroma.unread_count` to the Marker entity.
-- **Breaking:** Notification Settings API for suppressing notifications
-  has been simplified down to `block_from_strangers`.
-- **Breaking:** Notification Settings API option for hiding push notification
-  contents has been renamed to `hide_notification_contents`
 - Mastodon API: Added `pleroma.metadata.post_formats` to /api/v1/instance
 - Mastodon API (legacy): Allow query parameters for `/api/v1/domain_blocks`, e.g. `/api/v1/domain_blocks?domain=badposters.zone`
 - Pleroma API: `/api/pleroma/captcha` responses now include `seconds_valid` with an integer value.
+
 </details>
 
 <details>
   <summary>Admin API Changes</summary>
 
+- **Breaking** Changed relay `/api/pleroma/admin/relay` endpoints response format.
 - Status visibility stats: now can return stats per instance.
-
 - Mix task to refresh counter cache (`mix pleroma.refresh_counter_cache`)
+
 </details>
 
 ### Removed
+
 - **Breaking:** removed `with_move` parameter from notifications timeline.
 
 ### Added
@@ -108,6 +110,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Emoji Packs could not be listed when instance was set to `public: false`
 - Fix whole_word always returning false on filter get requests
 - Migrations not working on OTP releases if the database was connected over ssl
+- Fix relay following
 
 ## [Unreleased (patch)]
 
diff --git a/docs/API/admin_api.md b/docs/API/admin_api.md
index 05e63b528..c0ea074f0 100644
--- a/docs/API/admin_api.md
+++ b/docs/API/admin_api.md
@@ -313,31 +313,53 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
   - On failure: `Not found`
   - On success: JSON array of user's latest statuses
 
+## `GET /api/pleroma/admin/relay`
+
+### List Relays
+
+Params: none
+Response:
+
+* On success: JSON array of relays
+
+```json
+[
+  {"actor": "https://example.com/relay", "followed_back": true},
+  {"actor": "https://example2.com/relay", "followed_back": false}
+]
+```
+
 ## `POST /api/pleroma/admin/relay`
 
 ### Follow a Relay
 
-- Params:
-  - `relay_url`
-- Response:
-  - On success: URL of the followed relay
+Params:
+
+* `relay_url`
+
+Response:
+
+* On success: relay json object
+
+```json
+{"actor": "https://example.com/relay", "followed_back": true}
+```
 
 ## `DELETE /api/pleroma/admin/relay`
 
 ### Unfollow a Relay
 
-- Params:
-  - `relay_url`
-- Response:
-  - On success: URL of the unfollowed relay
+Params:
 
-## `GET /api/pleroma/admin/relay`
+* `relay_url`
 
-### List Relays
+Response:
 
-- Params: none
-- Response:
-  - On success: JSON array of relays
+* On success: URL of the unfollowed relay
+
+```json
+{"https://example.com/relay"}
+```
 
 ## `POST /api/pleroma/admin/users/invite_token`
 
diff --git a/lib/mix/tasks/pleroma/relay.ex b/lib/mix/tasks/pleroma/relay.ex
index c3312507e..a6d8d6c1c 100644
--- a/lib/mix/tasks/pleroma/relay.ex
+++ b/lib/mix/tasks/pleroma/relay.ex
@@ -35,10 +35,16 @@ defmodule Mix.Tasks.Pleroma.Relay do
   def run(["list"]) do
     start_pleroma()
 
-    with {:ok, list} <- Relay.list(true) do
-      list |> Enum.each(&shell_info(&1))
+    with {:ok, list} <- Relay.list() do
+      Enum.each(list, &print_relay_url/1)
     else
       {:error, e} -> shell_error("Error while fetching relay subscription list: #{inspect(e)}")
     end
   end
+
+  defp print_relay_url(%{followed_back: false} = relay) do
+    shell_info("#{relay.actor} - no Accept received (relay didn't follow back)")
+  end
+
+  defp print_relay_url(relay), do: shell_info(relay.actor)
 end
diff --git a/lib/pleroma/following_relationship.ex b/lib/pleroma/following_relationship.ex
index 83b366dd4..2039a259d 100644
--- a/lib/pleroma/following_relationship.ex
+++ b/lib/pleroma/following_relationship.ex
@@ -264,4 +264,12 @@ defmodule Pleroma.FollowingRelationship do
       end
     end)
   end
+
+  @spec following_ap_ids(User.t()) :: [String.t()]
+  def following_ap_ids(%User{} = user) do
+    user
+    |> following_query()
+    |> select([r, u], u.ap_id)
+    |> Repo.all()
+  end
 end
diff --git a/lib/pleroma/user.ex b/lib/pleroma/user.ex
index a9820affa..d2ad9516f 100644
--- a/lib/pleroma/user.ex
+++ b/lib/pleroma/user.ex
@@ -247,6 +247,13 @@ defmodule Pleroma.User do
     end
   end
 
+  defdelegate following_count(user), to: FollowingRelationship
+  defdelegate following(user), to: FollowingRelationship
+  defdelegate following?(follower, followed), to: FollowingRelationship
+  defdelegate following_ap_ids(user), to: FollowingRelationship
+  defdelegate get_follow_requests(user), to: FollowingRelationship
+  defdelegate search(query, opts \\ []), to: User.Search
+
   @doc """
   Dumps Flake Id to SQL-compatible format (16-byte UUID).
   E.g. "9pQtDGXuq4p3VlcJEm" -> <<0, 0, 1, 110, 179, 218, 42, 92, 213, 41, 44, 227, 95, 213, 0, 0>>
@@ -372,8 +379,6 @@ defmodule Pleroma.User do
     from(u in query, where: u.deactivated != ^true)
   end
 
-  defdelegate following_count(user), to: FollowingRelationship
-
   defp truncate_fields_param(params) do
     if Map.has_key?(params, :fields) do
       Map.put(params, :fields, Enum.map(params[:fields], &truncate_field/1))
@@ -868,8 +873,6 @@ defmodule Pleroma.User do
     set_cache(follower)
   end
 
-  defdelegate following(user), to: FollowingRelationship
-
   def follow(%User{} = follower, %User{} = followed, state \\ :follow_accept) do
     deny_follow_blocked = Config.get([:user, :deny_follow_blocked])
 
@@ -923,8 +926,6 @@ defmodule Pleroma.User do
     end
   end
 
-  defdelegate following?(follower, followed), to: FollowingRelationship
-
   @doc "Returns follow state as Pleroma.FollowingRelationship.State value"
   def get_follow_state(%User{} = follower, %User{} = following) do
     following_relationship = FollowingRelationship.get(follower, following)
@@ -1189,8 +1190,6 @@ defmodule Pleroma.User do
     |> Repo.all()
   end
 
-  defdelegate get_follow_requests(user), to: FollowingRelationship
-
   def increase_note_count(%User{} = user) do
     User
     |> where(id: ^user.id)
@@ -2163,8 +2162,6 @@ defmodule Pleroma.User do
     |> Repo.all()
   end
 
-  defdelegate search(query, opts \\ []), to: User.Search
-
   defp put_password_hash(
          %Ecto.Changeset{valid?: true, changes: %{password: password}} = changeset
        ) do
diff --git a/lib/pleroma/web/activity_pub/activity_pub.ex b/lib/pleroma/web/activity_pub/activity_pub.ex
index 8c5b7dac2..624a508ae 100644
--- a/lib/pleroma/web/activity_pub/activity_pub.ex
+++ b/lib/pleroma/web/activity_pub/activity_pub.ex
@@ -1344,9 +1344,8 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
   end
 
   def maybe_handle_clashing_nickname(data) do
-    nickname = data[:nickname]
-
-    with %User{} = old_user <- User.get_by_nickname(nickname),
+    with nickname when is_binary(nickname) <- data[:nickname],
+         %User{} = old_user <- User.get_by_nickname(nickname),
          {_, false} <- {:ap_id_comparison, data[:ap_id] == old_user.ap_id} do
       Logger.info(
         "Found an old user for #{nickname}, the old ap id is #{old_user.ap_id}, new one is #{
@@ -1360,7 +1359,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
     else
       {:ap_id_comparison, true} ->
         Logger.info(
-          "Found an old user for #{nickname}, but the ap id #{data[:ap_id]} is the same as the new user. Race condition? Not changing anything."
+          "Found an old user for #{data[:nickname]}, but the ap id #{data[:ap_id]} is the same as the new user. Race condition? Not changing anything."
         )
 
       _ ->
diff --git a/lib/pleroma/web/activity_pub/builder.ex b/lib/pleroma/web/activity_pub/builder.ex
index f2392ce79..9a7b7d9de 100644
--- a/lib/pleroma/web/activity_pub/builder.ex
+++ b/lib/pleroma/web/activity_pub/builder.ex
@@ -215,7 +215,7 @@ defmodule Pleroma.Web.ActivityPub.Builder do
 
     to =
       cond do
-        actor.ap_id == Relay.relay_ap_id() ->
+        actor.ap_id == Relay.ap_id() ->
           [actor.follower_address]
 
         public? ->
diff --git a/lib/pleroma/web/activity_pub/relay.ex b/lib/pleroma/web/activity_pub/relay.ex
index b09764d2b..b65710a94 100644
--- a/lib/pleroma/web/activity_pub/relay.ex
+++ b/lib/pleroma/web/activity_pub/relay.ex
@@ -10,19 +10,13 @@ defmodule Pleroma.Web.ActivityPub.Relay do
   alias Pleroma.Web.CommonAPI
   require Logger
 
-  @relay_nickname "relay"
+  @nickname "relay"
 
-  def get_actor do
-    actor =
-      relay_ap_id()
-      |> User.get_or_create_service_actor_by_ap_id(@relay_nickname)
+  @spec ap_id() :: String.t()
+  def ap_id, do: "#{Pleroma.Web.Endpoint.url()}/#{@nickname}"
 
-    actor
-  end
-
-  def relay_ap_id do
-    "#{Pleroma.Web.Endpoint.url()}/relay"
-  end
+  @spec get_actor() :: User.t() | nil
+  def get_actor, do: User.get_or_create_service_actor_by_ap_id(ap_id(), @nickname)
 
   @spec follow(String.t()) :: {:ok, Activity.t()} | {:error, any()}
   def follow(target_instance) do
@@ -61,34 +55,38 @@ defmodule Pleroma.Web.ActivityPub.Relay do
 
   def publish(_), do: {:error, "Not implemented"}
 
-  @spec list(boolean()) :: {:ok, [String.t()]} | {:error, any()}
-  def list(with_not_accepted \\ false) do
+  @spec list() :: {:ok, [%{actor: String.t(), followed_back: boolean()}]} | {:error, any()}
+  def list do
     with %User{} = user <- get_actor() do
       accepted =
         user
-        |> User.following()
-        |> Enum.map(fn entry -> URI.parse(entry).host end)
-        |> Enum.uniq()
-
-      list =
-        if with_not_accepted do
-          without_accept =
-            user
-            |> Pleroma.Activity.following_requests_for_actor()
-            |> Enum.map(fn a -> URI.parse(a.data["object"]).host <> " (no Accept received)" end)
-            |> Enum.uniq()
+        |> following()
+        |> Enum.map(fn actor -> %{actor: actor, followed_back: true} end)
 
-          accepted ++ without_accept
-        else
-          accepted
-        end
+      without_accept =
+        user
+        |> Pleroma.Activity.following_requests_for_actor()
+        |> Enum.map(fn activity -> %{actor: activity.data["object"], followed_back: false} end)
+        |> Enum.uniq()
 
-      {:ok, list}
+      {:ok, accepted ++ without_accept}
     else
       error -> format_error(error)
     end
   end
 
+  @spec following() :: [String.t()]
+  def following do
+    get_actor()
+    |> following()
+  end
+
+  defp following(user) do
+    user
+    |> User.following_ap_ids()
+    |> Enum.uniq()
+  end
+
   defp format_error({:error, error}), do: format_error(error)
 
   defp format_error(error) do
diff --git a/lib/pleroma/web/admin_api/controllers/relay_controller.ex b/lib/pleroma/web/admin_api/controllers/relay_controller.ex
index cf9f3a14b..95d06dde7 100644
--- a/lib/pleroma/web/admin_api/controllers/relay_controller.ex
+++ b/lib/pleroma/web/admin_api/controllers/relay_controller.ex
@@ -39,7 +39,7 @@ defmodule Pleroma.Web.AdminAPI.RelayController do
         target: target
       })
 
-      json(conn, target)
+      json(conn, %{actor: target, followed_back: target in Relay.following()})
     else
       _ ->
         conn
diff --git a/lib/pleroma/web/api_spec/operations/admin/relay_operation.ex b/lib/pleroma/web/api_spec/operations/admin/relay_operation.ex
index 67ee5eee0..e06b2d164 100644
--- a/lib/pleroma/web/api_spec/operations/admin/relay_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/admin/relay_operation.ex
@@ -27,8 +27,7 @@ defmodule Pleroma.Web.ApiSpec.Admin.RelayOperation do
             properties: %{
               relays: %Schema{
                 type: :array,
-                items: %Schema{type: :string},
-                example: ["lain.com", "mstdn.io"]
+                items: relay()
               }
             }
           })
@@ -43,19 +42,9 @@ defmodule Pleroma.Web.ApiSpec.Admin.RelayOperation do
       operationId: "AdminAPI.RelayController.follow",
       security: [%{"oAuth" => ["write:follows"]}],
       parameters: admin_api_params(),
-      requestBody:
-        request_body("Parameters", %Schema{
-          type: :object,
-          properties: %{
-            relay_url: %Schema{type: :string, format: :uri}
-          }
-        }),
+      requestBody: request_body("Parameters", relay_url()),
       responses: %{
-        200 =>
-          Operation.response("Status", "application/json", %Schema{
-            type: :string,
-            example: "http://mastodon.example.org/users/admin"
-          })
+        200 => Operation.response("Status", "application/json", relay())
       }
     }
   end
@@ -67,13 +56,7 @@ defmodule Pleroma.Web.ApiSpec.Admin.RelayOperation do
       operationId: "AdminAPI.RelayController.unfollow",
       security: [%{"oAuth" => ["write:follows"]}],
       parameters: admin_api_params(),
-      requestBody:
-        request_body("Parameters", %Schema{
-          type: :object,
-          properties: %{
-            relay_url: %Schema{type: :string, format: :uri}
-          }
-        }),
+      requestBody: request_body("Parameters", relay_url()),
       responses: %{
         200 =>
           Operation.response("Status", "application/json", %Schema{
@@ -83,4 +66,29 @@ defmodule Pleroma.Web.ApiSpec.Admin.RelayOperation do
       }
     }
   end
+
+  defp relay do
+    %Schema{
+      type: :object,
+      properties: %{
+        actor: %Schema{
+          type: :string,
+          example: "https://example.com/relay"
+        },
+        followed_back: %Schema{
+          type: :boolean,
+          description: "Is relay followed back by this actor?"
+        }
+      }
+    }
+  end
+
+  defp relay_url do
+    %Schema{
+      type: :object,
+      properties: %{
+        relay_url: %Schema{type: :string, format: :uri}
+      }
+    }
+  end
 end
diff --git a/test/tasks/relay_test.exs b/test/tasks/relay_test.exs
index 79ab72002..e5225b64c 100644
--- a/test/tasks/relay_test.exs
+++ b/test/tasks/relay_test.exs
@@ -42,7 +42,11 @@ defmodule Mix.Tasks.Pleroma.RelayTest do
       assert activity.data["object"] == target_user.ap_id
 
       :ok = Mix.Tasks.Pleroma.Relay.run(["list"])
-      assert_receive {:mix_shell, :info, ["mastodon.example.org (no Accept received)"]}
+
+      assert_receive {:mix_shell, :info,
+                      [
+                        "http://mastodon.example.org/users/admin - no Accept received (relay didn't follow back)"
+                      ]}
     end
   end
 
@@ -95,8 +99,8 @@ defmodule Mix.Tasks.Pleroma.RelayTest do
 
       :ok = Mix.Tasks.Pleroma.Relay.run(["list"])
 
-      assert_receive {:mix_shell, :info, ["mstdn.io"]}
-      assert_receive {:mix_shell, :info, ["mastodon.example.org"]}
+      assert_receive {:mix_shell, :info, ["https://mstdn.io/users/mayuutann"]}
+      assert_receive {:mix_shell, :info, ["http://mastodon.example.org/users/admin"]}
     end
   end
 end
diff --git a/test/web/activity_pub/activity_pub_controller_test.exs b/test/web/activity_pub/activity_pub_controller_test.exs
index ed900d8f8..57988dc1e 100644
--- a/test/web/activity_pub/activity_pub_controller_test.exs
+++ b/test/web/activity_pub/activity_pub_controller_test.exs
@@ -533,7 +533,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubControllerTest do
       end)
 
       :ok = Mix.Tasks.Pleroma.Relay.run(["list"])
-      assert_receive {:mix_shell, :info, ["relay.mastodon.host"]}
+      assert_receive {:mix_shell, :info, ["https://relay.mastodon.host/actor"]}
     end
 
     @tag capture_log: true
diff --git a/test/web/admin_api/controllers/relay_controller_test.exs b/test/web/admin_api/controllers/relay_controller_test.exs
index 64086adc5..adadf2b5c 100644
--- a/test/web/admin_api/controllers/relay_controller_test.exs
+++ b/test/web/admin_api/controllers/relay_controller_test.exs
@@ -39,8 +39,10 @@ defmodule Pleroma.Web.AdminAPI.RelayControllerTest do
           relay_url: "http://mastodon.example.org/users/admin"
         })
 
-      assert json_response_and_validate_schema(conn, 200) ==
-               "http://mastodon.example.org/users/admin"
+      assert json_response_and_validate_schema(conn, 200) == %{
+               "actor" => "http://mastodon.example.org/users/admin",
+               "followed_back" => false
+             }
 
       log_entry = Repo.one(ModerationLog)
 
@@ -59,8 +61,13 @@ defmodule Pleroma.Web.AdminAPI.RelayControllerTest do
 
       conn = get(conn, "/api/pleroma/admin/relay")
 
-      assert json_response_and_validate_schema(conn, 200)["relays"] --
-               ["mastodon.example.org", "mstdn.io"] == []
+      assert json_response_and_validate_schema(conn, 200)["relays"] == [
+               %{
+                 "actor" => "http://mastodon.example.org/users/admin",
+                 "followed_back" => true
+               },
+               %{"actor" => "https://mstdn.io/users/mayuutann", "followed_back" => true}
+             ]
     end
 
     test "DELETE /relay", %{conn: conn, admin: admin} do