Merge branch 'feature/1469-webfinger-expanding' into 'develop'

Feature/1469 webfinger expanding

Closes #1469 and #2517

See merge request pleroma/pleroma!3361

20 files changed, 489 insertions, 69 deletions

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8a4cd1b05..6ea8b1cb6 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -37,6 +37,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Uploadfilter `Pleroma.Upload.Filter.Exiftool.ReadDescription` returns description values to the FE so they can pre fill the image description field
 - Added move account API
 - Enable remote users to interact with posts
+- Possibility to discover users like `user@example.org`, while Pleroma is working on `pleroma.example.org`. Additional configuration required.
 
 ### Fixed
 - Subscription(Bell) Notifications: Don't create from Pipeline Ingested replies
diff --git a/config/config.exs b/config/config.exs
index 4e21ce457..a84b15a37 100644
--- a/config/config.exs
+++ b/config/config.exs
@@ -869,6 +869,8 @@ config :pleroma, ConcurrentLimiter, [
   {Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy, [max_running: 5, max_waiting: 5]}
 ]
 
+config :pleroma, Pleroma.Web.WebFinger, domain: nil, update_nickname_on_user_fetch: true
+
 # Import environment specific config. This must remain at the bottom
 # of this file so it overrides the configuration defined above.
 import_config "#{Mix.env()}.exs"
diff --git a/config/test.exs b/config/test.exs
index d5c25f65e..f5eb6e5c2 100644
--- a/config/test.exs
+++ b/config/test.exs
@@ -129,6 +129,8 @@ config :pleroma, :pipeline,
 
 config :pleroma, :cachex, provider: Pleroma.CachexMock
 
+config :pleroma, Pleroma.Web.WebFinger, update_nickname_on_user_fetch: false
+
 config :pleroma, :side_effects,
   ap_streamer: Pleroma.Web.ActivityPub.ActivityPubMock,
   logger: Pleroma.LoggerMock
diff --git a/docs/configuration/how_to_serve_another_domain_for_webfinger.md b/docs/configuration/how_to_serve_another_domain_for_webfinger.md
new file mode 100644
index 000000000..4e70f444c
--- /dev/null
+++ b/docs/configuration/how_to_serve_another_domain_for_webfinger.md
@@ -0,0 +1,62 @@
+# How to use a different domain name for Pleroma and the users it serves
+
+Pleroma users are primarily identified by a `user@example.org` handle, and you might want this identifier to be the same as your email or jabber account, for instance.
+However, in this case, you are almost certainly serving some web content on `https://example.org` already, and you might want to use another domain (say `pleroma.example.org`) for Pleroma itself.
+
+Pleroma supports that, but it might be tricky to set up, and any error might prevent you from federating with other instances.
+
+*If you are already running Pleroma on `example.org`, it is no longer possible to move it to `pleroma.example.org`.*
+
+## Account identifiers
+
+It is important to understand that for federation purposes, a user in Pleroma has two unique identifiers associated:
+
+- A webfinger `acct:` URI, used for discovery and as a verifiable global name for the user across Pleroma instances. In our example, our account's acct: URI is `acct:user@example.org`
+- An author/actor URI, used in every other aspect of federation. This is the way in which users are identified in ActivityPub, the underlying protocol used for federation with other Pleroma instances.
+In our case, it is `https://pleroma.example.org/users/user`.
+
+Both account identifiers are unique and required for Pleroma. An important risk if you set up your Pleroma instance incorrectly is to create two users (with different acct: URIs) with conflicting author/actor URIs.
+
+## WebFinger
+
+As said earlier, each Pleroma user has an `acct`: URI, which is used for discovery and authentication. When you add @user@example.org, a webfinger query is performed. This is done in two steps:
+
+1. Querying `https://example.org/.well-known/host-meta` (where the domain of the URL matches the domain part of the `acct`: URI) to get information on how to perform the query.
+This file will indeed contain a URL template of the form `https://example.org/.well-known/webfinger?resource={uri}` that will be used in the second step.
+2. Fill the returned template with the `acct`: URI to be queried and perform the query: `https://example.org/.well-known/webfinger?resource=acct:user@example.org`
+
+## Configuring your Pleroma instance
+
+**_DO NOT ATTEMPT TO CONFIGURE YOUR INSTANCE THIS WAY IF YOU DID NOT UNDERSTAND THE ABOVE_**
+
+### Configuring Pleroma
+
+Pleroma has a two configuration settings to enable using different domains for your users and Pleroma itself. `host` in `Pleroma.Web.Endpoint` and `domain` in `Pleroma.Web.WebFinger`. When the latter is not set, it defaults to the value of `host`.
+
+*Be extra careful when configuring your Pleroma instance, as changing `host` may cause remote instances to register different accounts with the same author/actor URI, which will result in federation issues!*
+
+```elixir
+config :pleroma, Pleroma.Web.Endpoint,
+  url: [host: "pleroma.example.org"]
+
+config :pleroma, Pleroma.Web.WebFinger, domain: "example.org"
+```
+
+- `domain` - is the domain for which your Pleroma instance has authority, it's the domain used in `acct:` URI. In our example, `domain` would be set to `example.org.
+- `host` - is the domain used for any URL generated for your instance, including the author/actor URL's. In our case, that would be `pleroma.example.org.
+
+### Configuring WebFinger domain
+
+Now, you have Pleroma running at `https://pleroma.example.org` as well as a website at `https://example.org`. If you recall how webfinger queries work, the first step is to query `https://example.org/.well-known/host-meta`, which will contain an URL template.
+
+Therefore, the easiest way to configure `example.org` is to redirect `/.well-known/host-meta` to `pleroma.example.org`.
+
+With nginx, it would be as simple as adding:
+
+```nginx
+location = /.well-known/host-meta {
+       return 301 https://pleroma.example.org$request_uri;
+}
+```
+
+in example.org's server block.
diff --git a/lib/pleroma/http.ex b/lib/pleroma/http.ex
index 2e82ceff2..d41061538 100644
--- a/lib/pleroma/http.ex
+++ b/lib/pleroma/http.ex
@@ -106,5 +106,12 @@ defmodule Pleroma.HTTP do
     [Tesla.Middleware.FollowRedirects, Pleroma.Tesla.Middleware.ConnectionPool]
   end
 
-  defp adapter_middlewares(_), do: []
+  defp adapter_middlewares(_) do
+    if Pleroma.Config.get(:env) == :test do
+      # Emulate redirects in test env, which are handled by adapters in other environments
+      [Tesla.Middleware.FollowRedirects]
+    else
+      []
+    end
+  end
 end
diff --git a/lib/pleroma/web/activity_pub/activity_pub.ex b/lib/pleroma/web/activity_pub/activity_pub.ex
index a5d7036d9..5099caef7 100644
--- a/lib/pleroma/web/activity_pub/activity_pub.ex
+++ b/lib/pleroma/web/activity_pub/activity_pub.ex
@@ -1482,7 +1482,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
   defp normalize_image(urls) when is_list(urls), do: urls |> List.first() |> normalize_image()
   defp normalize_image(_), do: nil
 
-  defp object_to_user_data(data) do
+  defp object_to_user_data(data, additional) do
     fields =
       data
       |> Map.get("attachment", [])
@@ -1514,15 +1514,11 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
     public_key =
       if is_map(data["publicKey"]) && is_binary(data["publicKey"]["publicKeyPem"]) do
         data["publicKey"]["publicKeyPem"]
-      else
-        nil
       end
 
     shared_inbox =
       if is_map(data["endpoints"]) && is_binary(data["endpoints"]["sharedInbox"]) do
         data["endpoints"]["sharedInbox"]
-      else
-        nil
       end
 
     birthday =
@@ -1531,13 +1527,15 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
           {:ok, date} -> date
           {:error, _} -> nil
         end
-      else
-        nil
       end
 
     show_birthday = !!birthday
 
-    user_data = %{
+    # if WebFinger request was already done, we probably have acct, otherwise
+    # we request WebFinger here
+    nickname = additional[:nickname_from_acct] || generate_nickname(data)
+
+    %{
       ap_id: data["id"],
       uri: get_actor_url(data["url"]),
       ap_enabled: true,
@@ -1559,23 +1557,29 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
       inbox: data["inbox"],
       shared_inbox: shared_inbox,
       accepts_chat_messages: accepts_chat_messages,
-      pinned_objects: pinned_objects,
       birthday: birthday,
-      show_birthday: show_birthday
+      show_birthday: show_birthday,
+      pinned_objects: pinned_objects,
+      nickname: nickname
     }
+  end
 
-    # nickname can be nil because of virtual actors
-    if data["preferredUsername"] do
-      Map.put(
-        user_data,
-        :nickname,
-        "#{data["preferredUsername"]}@#{URI.parse(data["id"]).host}"
-      )
+  defp generate_nickname(%{"preferredUsername" => username} = data) when is_binary(username) do
+    generated = "#{username}@#{URI.parse(data["id"]).host}"
+
+    if Config.get([WebFinger, :update_nickname_on_user_fetch]) do
+      case WebFinger.finger(generated) do
+        {:ok, %{"subject" => "acct:" <> acct}} -> acct
+        _ -> generated
+      end
     else
-      Map.put(user_data, :nickname, nil)
+      generated
     end
   end
 
+  # nickname can be nil because of virtual actors
+  defp generate_nickname(_), do: nil
+
   def fetch_follow_information_for_user(user) do
     with {:ok, following_data} <-
            Fetcher.fetch_and_contain_remote_object_from_id(user.following_address),
@@ -1647,17 +1651,17 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
 
   defp collection_private(_data), do: {:ok, true}
 
-  def user_data_from_user_object(data) do
+  def user_data_from_user_object(data, additional \\ []) do
     with {:ok, data} <- MRF.filter(data) do
-      {:ok, object_to_user_data(data)}
+      {:ok, object_to_user_data(data, additional)}
     else
       e -> {:error, e}
     end
   end
 
-  def fetch_and_prepare_user_from_ap_id(ap_id) do
+  def fetch_and_prepare_user_from_ap_id(ap_id, additional \\ []) do
     with {:ok, data} <- Fetcher.fetch_and_contain_remote_object_from_id(ap_id),
-         {:ok, data} <- user_data_from_user_object(data) do
+         {:ok, data} <- user_data_from_user_object(data, additional) do
       {:ok, maybe_update_follow_information(data)}
     else
       # If this has been deleted, only log a debug and not an error
@@ -1735,13 +1739,13 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
     end
   end
 
-  def make_user_from_ap_id(ap_id) do
+  def make_user_from_ap_id(ap_id, additional \\ []) do
     user = User.get_cached_by_ap_id(ap_id)
 
     if user && !User.ap_enabled?(user) do
       Transmogrifier.upgrade_user_from_ap_id(ap_id)
     else
-      with {:ok, data} <- fetch_and_prepare_user_from_ap_id(ap_id) do
+      with {:ok, data} <- fetch_and_prepare_user_from_ap_id(ap_id, additional) do
         {:ok, _pid} = Task.start(fn -> pinned_fetch_task(data) end)
 
         if user do
@@ -1761,8 +1765,9 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
   end
 
   def make_user_from_nickname(nickname) do
-    with {:ok, %{"ap_id" => ap_id}} when not is_nil(ap_id) <- WebFinger.finger(nickname) do
-      make_user_from_ap_id(ap_id)
+    with {:ok, %{"ap_id" => ap_id, "subject" => "acct:" <> acct}} when not is_nil(ap_id) <-
+           WebFinger.finger(nickname) do
+      make_user_from_ap_id(ap_id, nickname_from_acct: acct)
     else
       _e -> {:error, "No AP id in WebFinger"}
     end
diff --git a/lib/pleroma/web/web_finger.ex b/lib/pleroma/web/web_finger.ex
index 77ff40f46..f95dc2458 100644
--- a/lib/pleroma/web/web_finger.ex
+++ b/lib/pleroma/web/web_finger.ex
@@ -32,7 +32,13 @@ defmodule Pleroma.Web.WebFinger do
 
   def webfinger(resource, fmt) when fmt in ["XML", "JSON"] do
     host = Pleroma.Web.Endpoint.host()
-    regex = ~r/(acct:)?(?<username>[a-z0-9A-Z_\.-]+)@#{host}/
+
+    regex =
+      if webfinger_domain = Pleroma.Config.get([__MODULE__, :domain]) do
+        ~r/(acct:)?(?<username>[a-z0-9A-Z_\.-]+)@(#{host}|#{webfinger_domain})/
+      else
+        ~r/(acct:)?(?<username>[a-z0-9A-Z_\.-]+)@#{host}/
+      end
 
     with %{"username" => username} <- Regex.named_captures(regex, resource),
          %User{} = user <- User.get_cached_by_nickname(username) do
@@ -64,7 +70,7 @@ defmodule Pleroma.Web.WebFinger do
 
   def represent_user(user, "JSON") do
     %{
-      "subject" => "acct:#{user.nickname}@#{Pleroma.Web.Endpoint.host()}",
+      "subject" => "acct:#{user.nickname}@#{domain()}",
       "aliases" => gather_aliases(user),
       "links" => gather_links(user)
     }
@@ -84,12 +90,16 @@ defmodule Pleroma.Web.WebFinger do
       :XRD,
       %{xmlns: "http://docs.oasis-open.org/ns/xri/xrd-1.0"},
       [
-        {:Subject, "acct:#{user.nickname}@#{Pleroma.Web.Endpoint.host()}"}
+        {:Subject, "acct:#{user.nickname}@#{domain()}"}
       ] ++ aliases ++ links
     }
     |> XmlBuilder.to_doc()
   end
 
+  defp domain do
+    Pleroma.Config.get([__MODULE__, :domain]) || Pleroma.Web.Endpoint.host()
+  end
+
   defp webfinger_from_xml(body) do
     with {:ok, doc} <- XML.parse_document(body) do
       subject = XML.string_from_xpath("//Subject", doc)
@@ -146,17 +156,15 @@ defmodule Pleroma.Web.WebFinger do
   end
 
   def find_lrdd_template(domain) do
-    with {:ok, %{status: status, body: body}} when status in 200..299 <-
-           HTTP.get("http://#{domain}/.well-known/host-meta") do
+    # WebFinger is restricted to HTTPS - https://tools.ietf.org/html/rfc7033#section-9.1
+    meta_url = "https://#{domain}/.well-known/host-meta"
+
+    with {:ok, %{status: status, body: body}} when status in 200..299 <- HTTP.get(meta_url) do
       get_template_from_xml(body)
     else
-      _ ->
-        with {:ok, %{body: body, status: status}} when status in 200..299 <-
-               HTTP.get("https://#{domain}/.well-known/host-meta") do
-          get_template_from_xml(body)
-        else
-          e -> {:error, "Can't find LRDD template: #{inspect(e)}"}
-        end
+      error ->
+        Logger.warn("Can't find LRDD template in #{inspect(meta_url)}: #{inspect(error)}")
+        {:error, :lrdd_not_found}
     end
   end
 
@@ -170,7 +178,7 @@ defmodule Pleroma.Web.WebFinger do
     end
   end
 
-  defp get_address_from_domain(_, _), do: nil
+  defp get_address_from_domain(_, _), do: {:error, :webfinger_no_domain}
 
   @spec finger(String.t()) :: {:ok, map()} | {:error, any()}
   def finger(account) do
@@ -187,13 +195,11 @@ defmodule Pleroma.Web.WebFinger do
     encoded_account = URI.encode("acct:#{account}")
 
     with address when is_binary(address) <- get_address_from_domain(domain, encoded_account),
-         response <-
+         {:ok, %{status: status, body: body, headers: headers}} when status in 200..299 <-
            HTTP.get(
              address,
              [{"accept", "application/xrd+xml,application/jrd+json"}]
-           ),
-         {:ok, %{status: status, body: body, headers: headers}} when status in 200..299 <-
-           response do
+           ) do
       case List.keyfind(headers, "content-type", 0) do
         {_, content_type} ->
           case Plug.Conn.Utils.media_type(content_type) do
@@ -211,10 +217,9 @@ defmodule Pleroma.Web.WebFinger do
           {:error, {:content_type, nil}}
       end
     else
-      e ->
-        Logger.debug(fn -> "Couldn't finger #{account}" end)
-        Logger.debug(fn -> inspect(e) end)
-        {:error, e}
+      error ->
+        Logger.debug("Couldn't finger #{account}: #{inspect(error)}")
+        error
     end
   end
 end
diff --git a/test/fixtures/tesla_mock/framatube.org_host_meta b/test/fixtures/tesla_mock/framatube.org_host_meta
index 91516ff6d..02e25bd64 100644
--- a/test/fixtures/tesla_mock/framatube.org_host_meta
+++ b/test/fixtures/tesla_mock/framatube.org_host_meta
@@ -1,2 +1,2 @@
 <?xml version="1.0" encoding="UTF-8"?>
-<XRD xmlns="http://docs.oasis-open.org/ns/xri/xrd-1.0"><hm:Host xmlns:hm="http://host-meta.net/xrd/1.0">framatube.org</hm:Host><Link rel="lrdd" template="http://framatube.org/main/xrd?uri={uri}"><Title>Resource Descriptor</Title></Link></XRD>
+<XRD xmlns="http://docs.oasis-open.org/ns/xri/xrd-1.0"><hm:Host xmlns:hm="http://host-meta.net/xrd/1.0">framatube.org</hm:Host><Link rel="lrdd" template="https://framatube.org/main/xrd?uri={uri}"><Title>Resource Descriptor</Title></Link></XRD>
diff --git a/test/fixtures/tesla_mock/status.alpicola.com_host_meta b/test/fixtures/tesla_mock/status.alpicola.com_host_meta
index 6948c30ea..78155f644 100644
--- a/test/fixtures/tesla_mock/status.alpicola.com_host_meta
+++ b/test/fixtures/tesla_mock/status.alpicola.com_host_meta
@@ -1,2 +1,2 @@
 <?xml version="1.0" encoding="UTF-8"?>
-<XRD xmlns="http://docs.oasis-open.org/ns/xri/xrd-1.0"><hm:Host xmlns:hm="http://host-meta.net/xrd/1.0">status.alpicola.com</hm:Host><Link rel="lrdd" template="http://status.alpicola.com/main/xrd?uri={uri}"><Title>Resource Descriptor</Title></Link></XRD>
\ No newline at end of file
+<XRD xmlns="http://docs.oasis-open.org/ns/xri/xrd-1.0"><hm:Host xmlns:hm="http://host-meta.net/xrd/1.0">status.alpicola.com</hm:Host><Link rel="lrdd" template="https://status.alpicola.com/main/xrd?uri={uri}"><Title>Resource Descriptor</Title></Link></XRD>
diff --git a/test/fixtures/webfinger/masto-host-meta.xml b/test/fixtures/webfinger/masto-host-meta.xml
new file mode 100644
index 000000000..f432a27c3
--- /dev/null
+++ b/test/fixtures/webfinger/masto-host-meta.xml
@@ -0,0 +1,4 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<XRD xmlns="http://docs.oasis-open.org/ns/xri/xrd-1.0">
+  <Link rel="lrdd" template="https://{{domain}}/.well-known/webfinger?resource={uri}"/>
+</XRD>
diff --git a/test/fixtures/webfinger/masto-user.json b/test/fixtures/webfinger/masto-user.json
new file mode 100644
index 000000000..1702de011
--- /dev/null
+++ b/test/fixtures/webfinger/masto-user.json
@@ -0,0 +1,92 @@
+{
+  "@context": [
+    "https://www.w3.org/ns/activitystreams",
+    "https://w3id.org/security/v1",
+    {
+      "manuallyApprovesFollowers": "as:manuallyApprovesFollowers",
+      "toot": "http://joinmastodon.org/ns#",
+      "featured": {
+        "@id": "toot:featured",
+        "@type": "@id"
+      },
+      "featuredTags": {
+        "@id": "toot:featuredTags",
+        "@type": "@id"
+      },
+      "alsoKnownAs": {
+        "@id": "as:alsoKnownAs",
+        "@type": "@id"
+      },
+      "movedTo": {
+        "@id": "as:movedTo",
+        "@type": "@id"
+      },
+      "schema": "http://schema.org#",
+      "PropertyValue": "schema:PropertyValue",
+      "value": "schema:value",
+      "IdentityProof": "toot:IdentityProof",
+      "discoverable": "toot:discoverable",
+      "Device": "toot:Device",
+      "Ed25519Signature": "toot:Ed25519Signature",
+      "Ed25519Key": "toot:Ed25519Key",
+      "Curve25519Key": "toot:Curve25519Key",
+      "EncryptedMessage": "toot:EncryptedMessage",
+      "publicKeyBase64": "toot:publicKeyBase64",
+      "deviceId": "toot:deviceId",
+      "claim": {
+        "@type": "@id",
+        "@id": "toot:claim"
+      },
+      "fingerprintKey": {
+        "@type": "@id",
+        "@id": "toot:fingerprintKey"
+      },
+      "identityKey": {
+        "@type": "@id",
+        "@id": "toot:identityKey"
+      },
+      "devices": {
+        "@type": "@id",
+        "@id": "toot:devices"
+      },
+      "messageFranking": "toot:messageFranking",
+      "messageType": "toot:messageType",
+      "cipherText": "toot:cipherText",
+      "suspended": "toot:suspended",
+      "focalPoint": {
+        "@container": "@list",
+        "@id": "toot:focalPoint"
+      }
+    }
+  ],
+  "id": "https://{{domain}}/users/{{nickname}}",
+  "type": "Person",
+  "following": "https://{{domain}}/users/{{nickname}}/following",
+  "followers": "https://{{domain}}/users/{{nickname}}/followers",
+  "inbox": "https://{{domain}}/users/{{nickname}}/inbox",
+  "outbox": "https://{{domain}}/users/{{nickname}}/outbox",
+  "featured": "https://{{domain}}/users/{{nickname}}/collections/featured",
+  "featuredTags": "https://{{domain}}/users/{{nickname}}/collections/tags",
+  "preferredUsername": "{{nickname}}",
+  "name": "Name Name",
+  "summary": "<p>Summary</p>",
+  "url": "https://{{domain}}/@{{nickname}}",
+  "manuallyApprovesFollowers": false,
+  "discoverable": false,
+  "devices": "https://{{domain}}/users/{{nickname}}/collections/devices",
+  "publicKey": {
+    "id": "https://{{domain}}/users/{{nickname}}#main-key",
+    "owner": "https://{{domain}}/users/{{nickname}}",
+    "publicKeyPem": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvwDujxmxoYHs64MyVB3L\nG5ZyBxV3ufaMRBFu42bkcTpISq1WwZ+3Zb6CI8zOO+nM+Q2llrVRYjZa4ZFnOLvM\nTq/Kf+Zf5wy2aCRer88gX+MsJOAtItSi412y0a/rKOuFaDYLOLeTkRvmGLgZWbsr\nZJOp+YWb3zQ5qsIOInkc5BwI172tMsGeFtsnbNApPV4lrmtTGaJ8RiM8MR7XANBO\nfOHggSt1+eAIKGIsCmINEMzs1mG9D75xKtC/sM8GfbvBclQcBstGkHAEj1VHPW0c\nh6Bok5/QQppicyb8UA1PAA9bznSFtKlYE4xCH8rlCDSDTBRtdnBWHKcj619Ujz4Q\nawIDAQAB\n-----END PUBLIC KEY-----\n"
+  },
+  "tag": [],
+  "attachment": [],
+  "endpoints": {
+    "sharedInbox": "https://{{domain}}/inbox"
+  },
+  "icon": {
+    "type": "Image",
+    "mediaType": "image/jpeg",
+    "url": "https://s3.wasabisys.com/merp/accounts/avatars/000/000/001/original/6fdd3eee632af247.jpg"
+  }
+}
diff --git a/test/fixtures/webfinger/masto-webfinger.json b/test/fixtures/webfinger/masto-webfinger.json
new file mode 100644
index 000000000..561be3fff
--- /dev/null
+++ b/test/fixtures/webfinger/masto-webfinger.json
@@ -0,0 +1,23 @@
+{
+  "subject": "acct:{{nickname}}@{{domain}}",
+  "aliases": [
+    "https://{{subdomain}}/@{{nickname}}",
+    "https://{{subdomain}}/users/{{nickname}}"
+  ],
+  "links": [
+    {
+      "rel": "http://webfinger.net/rel/profile-page",
+      "type": "text/html",
+      "href": "https://{{subdomain}}/@{{nickname}}"
+    },
+    {
+      "rel": "self",
+      "type": "application/activity+json",
+      "href": "https://{{subdomain}}/users/{{nickname}}"
+    },
+    {
+      "rel": "http://ostatus.org/schema/1.0/subscribe",
+      "template": "https://{{subdomain}}/authorize_interaction?uri={uri}"
+    }
+  ]
+}
diff --git a/test/fixtures/webfinger/pleroma-host-meta.xml b/test/fixtures/webfinger/pleroma-host-meta.xml
new file mode 100644
index 000000000..88c274a1a
--- /dev/null
+++ b/test/fixtures/webfinger/pleroma-host-meta.xml
@@ -0,0 +1 @@
+<?xml version="1.0" encoding="UTF-8"?><XRD xmlns="http://docs.oasis-open.org/ns/xri/xrd-1.0"><Link rel="lrdd" template="https://{{domain}}/.well-known/webfinger?resource={uri}" type="application/xrd+xml" /></XRD>
diff --git a/test/fixtures/webfinger/pleroma-user.json b/test/fixtures/webfinger/pleroma-user.json
new file mode 100644
index 000000000..b822db46c
--- /dev/null
+++ b/test/fixtures/webfinger/pleroma-user.json
@@ -0,0 +1,58 @@
+{
+  "@context": [
+    "https://www.w3.org/ns/activitystreams",
+    "https://{{domain}}/schemas/litepub-0.1.jsonld",
+    {
+      "@language": "und"
+    }
+  ],
+  "alsoKnownAs": [],
+  "attachment": [],
+  "capabilities": {
+    "acceptsChatMessages": true
+  },
+  "discoverable": true,
+  "endpoints": {
+    "oauthAuthorizationEndpoint": "https://{{domain}}/oauth/authorize",
+    "oauthRegistrationEndpoint": "https://{{domain}}/api/v1/apps",
+    "oauthTokenEndpoint": "https://{{domain}}/oauth/token",
+    "sharedInbox": "https://{{domain}}/inbox",
+    "uploadMedia": "https://{{domain}}/api/ap/upload_media"
+  },
+  "followers": "https://{{domain}}/users/{{nickname}}/followers",
+  "following": "https://{{domain}}/users/{{nickname}}/following",
+  "icon": {
+    "type": "Image",
+    "url": "https://{{domain}}/media/a932a27f158b63c3a97e3a57d5384f714a82249274c6fc66c9eca581b4fd8af2.jpg"
+  },
+  "id": "https://{{domain}}/users/{{nickname}}",
+  "image": {
+    "type": "Image",
+    "url": "https://{{domain}}/media/db15f476d0ad14488db4762b7800479e6ef67b1824f8b9ea5c1fa05b7525c5b7.jpg"
+  },
+  "inbox": "https://{{domain}}/users/{{nickname}}/inbox",
+  "manuallyApprovesFollowers": false,
+  "name": "{{nickname}} :verified:",
+  "outbox": "https://{{domain}}/users/{{nickname}}/outbox",
+  "preferredUsername": "{{nickname}}",
+  "publicKey": {
+    "id": "https://{{domain}}/users/{{nickname}}#main-key",
+    "owner": "https://{{domain}}/users/{{nickname}}",
+    "publicKeyPem": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAu4XOAopC4nRIxNlHlt60\n//nCicuedu5wvLGIoQ+KUM2u7/PhLrrTDEqr1A7yQL95S0X8ryYtALgFLI5A54ww\nqjMIbIGAs44lEmDLMEd+XI+XxREE8wdsFpb4QQzWug0DTyqlMouTU25k0tfKh1rF\n4PMJ3uBSjDTAGgFvLNyFWTiVVgChbTNgGOmrEBucRl4NmKzQ69/FIUwENV88oQSU\n3bWvQTEH9rWH1rCLpkmQwdRiWfnhFX/4EUqXukfgoskvenKR8ff3nYhElDqFoE0e\nqUnIW1OZceyl8JewVLcL6m0/wdKeosTsfrcWc8DKfnRYQcBGNoBEq9GrOHDU0q2v\nyQIDAQAB\n-----END PUBLIC KEY-----\n\n"
+  },
+  "summary": "Pleroma BE dev",
+  "tag": [
+    {
+      "icon": {
+        "type": "Image",
+        "url": "https://{{domain}}/emoji/mine/6143373a807b1ae7.png"
+      },
+      "id": "https://{{domain}}/emoji/mine/6143373a807b1ae7.png",
+      "name": ":verified:",
+      "type": "Emoji",
+      "updated": "1970-01-01T00:00:00Z"
+    }
+  ],
+  "type": "Person",
+  "url": "https://{{domain}}/users/{{nickname}}"
+}
diff --git a/test/fixtures/webfinger/pleroma-webfinger.json b/test/fixtures/webfinger/pleroma-webfinger.json
new file mode 100644
index 000000000..8f075eaaf
--- /dev/null
+++ b/test/fixtures/webfinger/pleroma-webfinger.json
@@ -0,0 +1,27 @@
+{
+  "aliases": [
+    "https://{{subdomain}}/users/{{nickname}}"
+  ],
+  "links": [
+    {
+      "href": "https://{{subdomain}}/users/{{nickname}}",
+      "rel": "http://webfinger.net/rel/profile-page",
+      "type": "text/html"
+    },
+    {
+      "href": "https://{{subdomain}}/users/{{nickname}}",
+      "rel": "self",
+      "type": "application/activity+json"
+    },
+    {
+      "href": "https://{{subdomain}}/users/{{nickname}}",
+      "rel": "self",
+      "type": "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams\""
+    },
+    {
+      "rel": "http://ostatus.org/schema/1.0/subscribe",
+      "template": "https://{{subdomain}}/ostatus_subscribe?acct={uri}"
+    }
+  ],
+  "subject": "acct:{{nickname}}@{{domain}}"
+}
diff --git a/test/pleroma/user_test.exs b/test/pleroma/user_test.exs
index 092ad82f5..303598fad 100644
--- a/test/pleroma/user_test.exs
+++ b/test/pleroma/user_test.exs
@@ -859,6 +859,116 @@ defmodule Pleroma.UserTest do
     end
   end
 
+  describe "get_or_fetch/1 remote users with tld, while BE is runned on subdomain" do
+    setup do: clear_config([Pleroma.Web.WebFinger, :update_nickname_on_user_fetch], true)
+
+    test "for mastodon" do
+      Tesla.Mock.mock(fn
+        %{url: "https://example.com/.well-known/host-meta"} ->
+          %Tesla.Env{
+            status: 302,
+            headers: [{"location", "https://sub.example.com/.well-known/host-meta"}]
+          }
+
+        %{url: "https://sub.example.com/.well-known/host-meta"} ->
+          %Tesla.Env{
+            status: 200,
+            body:
+              "test/fixtures/webfinger/masto-host-meta.xml"
+              |> File.read!()
+              |> String.replace("{{domain}}", "sub.example.com")
+          }
+
+        %{url: "https://sub.example.com/.well-known/webfinger?resource=acct:a@example.com"} ->
+          %Tesla.Env{
+            status: 200,
+            body:
+              "test/fixtures/webfinger/masto-webfinger.json"
+              |> File.read!()
+              |> String.replace("{{nickname}}", "a")
+              |> String.replace("{{domain}}", "example.com")
+              |> String.replace("{{subdomain}}", "sub.example.com"),
+            headers: [{"content-type", "application/jrd+json"}]
+          }
+
+        %{url: "https://sub.example.com/users/a"} ->
+          %Tesla.Env{
+            status: 200,
+            body:
+              "test/fixtures/webfinger/masto-user.json"
+              |> File.read!()
+              |> String.replace("{{nickname}}", "a")
+              |> String.replace("{{domain}}", "sub.example.com"),
+            headers: [{"content-type", "application/activity+json"}]
+          }
+
+        %{url: "https://sub.example.com/users/a/collections/featured"} ->
+          %Tesla.Env{
+            status: 200,
+            body:
+              File.read!("test/fixtures/users_mock/masto_featured.json")
+              |> String.replace("{{domain}}", "sub.example.com")
+              |> String.replace("{{nickname}}", "a"),
+            headers: [{"content-type", "application/activity+json"}]
+          }
+      end)
+
+      ap_id = "a@example.com"
+      {:ok, fetched_user} = User.get_or_fetch(ap_id)
+
+      assert fetched_user.ap_id == "https://sub.example.com/users/a"
+      assert fetched_user.nickname == "a@example.com"
+    end
+
+    test "for pleroma" do
+      Tesla.Mock.mock(fn
+        %{url: "https://example.com/.well-known/host-meta"} ->
+          %Tesla.Env{
+            status: 302,
+            headers: [{"location", "https://sub.example.com/.well-known/host-meta"}]
+          }
+
+        %{url: "https://sub.example.com/.well-known/host-meta"} ->
+          %Tesla.Env{
+            status: 200,
+            body:
+              "test/fixtures/webfinger/pleroma-host-meta.xml"
+              |> File.read!()
+              |> String.replace("{{domain}}", "sub.example.com")
+          }
+
+        %{url: "https://sub.example.com/.well-known/webfinger?resource=acct:a@example.com"} ->
+          %Tesla.Env{
+            status: 200,
+            body:
+              "test/fixtures/webfinger/pleroma-webfinger.json"
+              |> File.read!()
+              |> String.replace("{{nickname}}", "a")
+              |> String.replace("{{domain}}", "example.com")
+              |> String.replace("{{subdomain}}", "sub.example.com"),
+            headers: [{"content-type", "application/jrd+json"}]
+          }
+
+        %{url: "https://sub.example.com/users/a"} ->
+          %Tesla.Env{
+            status: 200,
+            body:
+              "test/fixtures/webfinger/pleroma-user.json"
+              |> File.read!()
+              |> String.replace("{{nickname}}", "a")
+              |> String.replace("{{domain}}", "sub.example.com"),
+            headers: [{"content-type", "application/activity+json"}]
+          }
+      end)
+
+      ap_id = "a@example.com"
+      {:ok, fetched_user} = User.get_or_fetch(ap_id)
+
+      assert fetched_user.ap_id == "https://sub.example.com/users/a"
+      assert fetched_user.nickname == "a@example.com"
+    end
+  end
+
   describe "fetching a user from nickname or trying to build one" do
     test "gets an existing user" do
       user = insert(:user)
diff --git a/test/pleroma/web/twitter_api/remote_follow_controller_test.exs b/test/pleroma/web/twitter_api/remote_follow_controller_test.exs
index 2b57a42a4..1194e0afe 100644
--- a/test/pleroma/web/twitter_api/remote_follow_controller_test.exs
+++ b/test/pleroma/web/twitter_api/remote_follow_controller_test.exs
@@ -3,7 +3,7 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.Web.TwitterAPI.RemoteFollowControllerTest do
-  use Pleroma.Web.ConnCase
+  use Pleroma.Web.ConnCase, async: true
 
   alias Pleroma.MFA
   alias Pleroma.MFA.TOTP
diff --git a/test/pleroma/web/web_finger/web_finger_controller_test.exs b/test/pleroma/web/web_finger/web_finger_controller_test.exs
index b5be28e67..5e3ac26f9 100644
--- a/test/pleroma/web/web_finger/web_finger_controller_test.exs
+++ b/test/pleroma/web/web_finger/web_finger_controller_test.exs
@@ -48,6 +48,35 @@ defmodule Pleroma.Web.WebFinger.WebFingerControllerTest do
            ]
   end
 
+  test "reach user on tld, while pleroma is runned on subdomain" do
+    Pleroma.Web.Endpoint.config_change(
+      [{Pleroma.Web.Endpoint, url: [host: "sub.example.com"]}],
+      []
+    )
+
+    clear_config([Pleroma.Web.Endpoint, :url, :host], "sub.example.com")
+
+    clear_config([Pleroma.Web.WebFinger, :domain], "example.com")
+
+    user = insert(:user, ap_id: "https://sub.example.com/users/bobby", nickname: "bobby")
+
+    response =
+      build_conn()
+      |> put_req_header("accept", "application/jrd+json")
+      |> get("/.well-known/webfinger?resource=acct:#{user.nickname}@example.com")
+      |> json_response(200)
+
+    assert response["subject"] == "acct:#{user.nickname}@example.com"
+    assert response["aliases"] == ["https://sub.example.com/users/#{user.nickname}"]
+
+    on_exit(fn ->
+      Pleroma.Web.Endpoint.config_change(
+        [{Pleroma.Web.Endpoint, url: [host: "localhost"]}],
+        []
+      )
+    end)
+  end
+
   test "it returns 404 when user isn't found (JSON)" do
     result =
       build_conn()
diff --git a/test/pleroma/web/web_finger_test.exs b/test/pleroma/web/web_finger_test.exs
index 1cc6ae675..fafef54fe 100644
--- a/test/pleroma/web/web_finger_test.exs
+++ b/test/pleroma/web/web_finger_test.exs
@@ -47,7 +47,7 @@ defmodule Pleroma.Web.WebFingerTest do
 
     test "returns error when there is no content-type header" do
       Tesla.Mock.mock(fn
-        %{url: "http://social.heldscal.la/.well-known/host-meta"} ->
+        %{url: "https://social.heldscal.la/.well-known/host-meta"} ->
           {:ok,
            %Tesla.Env{
              status: 200,
@@ -120,7 +120,7 @@ defmodule Pleroma.Web.WebFingerTest do
     test "it gets the xrd endpoint for statusnet" do
       {:ok, template} = WebFinger.find_lrdd_template("status.alpicola.com")
 
-      assert template == "http://status.alpicola.com/main/xrd?uri={uri}"
+      assert template == "https://status.alpicola.com/main/xrd?uri={uri}"
     end
 
     test "it works with idna domains as nickname" do
@@ -147,7 +147,7 @@ defmodule Pleroma.Web.WebFingerTest do
              headers: [{"content-type", "application/jrd+json"}]
            }}
 
-        %{url: "http://mastodon.social/.well-known/host-meta"} ->
+        %{url: "https://mastodon.social/.well-known/host-meta"} ->
           {:ok,
            %Tesla.Env{
              status: 200,
@@ -170,7 +170,7 @@ defmodule Pleroma.Web.WebFingerTest do
              headers: [{"content-type", "application/xrd+xml"}]
            }}
 
-        %{url: "http://pawoo.net/.well-known/host-meta"} ->
+        %{url: "https://pawoo.net/.well-known/host-meta"} ->
           {:ok,
            %Tesla.Env{
              status: 200,
diff --git a/test/support/http_request_mock.ex b/test/support/http_request_mock.ex
index 7f6065579..b0cf613ac 100644
--- a/test/support/http_request_mock.ex
+++ b/test/support/http_request_mock.ex
@@ -424,14 +424,6 @@ defmodule HttpRequestMock do
     {:error, :nxdomain}
   end
 
-  def get("http://osada.macgirvin.com/.well-known/host-meta", _, _, _) do
-    {:ok,
-     %Tesla.Env{
-       status: 404,
-       body: ""
-     }}
-  end
-
   def get("https://osada.macgirvin.com/.well-known/host-meta", _, _, _) do
     {:ok,
      %Tesla.Env{
@@ -765,7 +757,7 @@ defmodule HttpRequestMock do
     {:ok, %Tesla.Env{status: 406, body: ""}}
   end
 
-  def get("http://squeet.me/.well-known/host-meta", _, _, _) do
+  def get("https://squeet.me/.well-known/host-meta", _, _, _) do
     {:ok,
      %Tesla.Env{status: 200, body: File.read!("test/fixtures/tesla_mock/squeet.me_host_meta")}}
   end
@@ -806,7 +798,7 @@ defmodule HttpRequestMock do
     {:ok, %Tesla.Env{status: 200, body: "", headers: [{"content-type", "application/jrd+json"}]}}
   end
 
-  def get("http://framatube.org/.well-known/host-meta", _, _, _) do
+  def get("https://framatube.org/.well-known/host-meta", _, _, _) do
     {:ok,
      %Tesla.Env{
        status: 200,
@@ -815,7 +807,7 @@ defmodule HttpRequestMock do
   end
 
   def get(
-        "http://framatube.org/main/xrd?uri=acct:framasoft@framatube.org",
+        "https://framatube.org/main/xrd?uri=acct:framasoft@framatube.org",
         _,
         _,
         [{"accept", "application/xrd+xml,application/jrd+json"}]
@@ -850,7 +842,7 @@ defmodule HttpRequestMock do
      }}
   end
 
-  def get("http://status.alpicola.com/.well-known/host-meta", _, _, _) do
+  def get("https://status.alpicola.com/.well-known/host-meta", _, _, _) do
     {:ok,
      %Tesla.Env{
        status: 200,
@@ -858,7 +850,7 @@ defmodule HttpRequestMock do
      }}
   end
 
-  def get("http://macgirvin.com/.well-known/host-meta", _, _, _) do
+  def get("https://macgirvin.com/.well-known/host-meta", _, _, _) do
     {:ok,
      %Tesla.Env{
        status: 200,
@@ -866,7 +858,7 @@ defmodule HttpRequestMock do
      }}
   end
 
-  def get("http://gerzilla.de/.well-known/host-meta", _, _, _) do
+  def get("https://gerzilla.de/.well-known/host-meta", _, _, _) do
     {:ok,
      %Tesla.Env{
        status: 200,