Merge pull request #3419 from ClearlyClaire/glitch-soc/merge-4.4

Merge upstream changes up to 1e5ac0f491 into stable-4.4

.rubocop_todo.yml

@@ -21,11 +21,11 @@ Metrics/BlockNesting:
 
 # Configuration parameters: AllowedMethods, AllowedPatterns.
 Metrics/CyclomaticComplexity:
-  Max: 25
+  Enabled: false
 
 # Configuration parameters: AllowedMethods, AllowedPatterns.
 Metrics/PerceivedComplexity:
-  Max: 27
+  Enabled: false
 
 # This cop supports safe autocorrection (--autocorrect).
 # Configuration parameters: AllowedVars, DefaultToNil.

CHANGELOG.md

@@ -2,6 +2,56 @@
 
 All notable changes to this project will be documented in this file.
 
+## [4.4.14] - 2026-02-24
+
+### Security
+
+- Reject unconfirmed FASPs (#37926 by @oneiros, [GHSA-qgmm-vr4c-ggjg](https://github.com/mastodon/mastodon/security/advisories/GHSA-qgmm-vr4c-ggjg))
+- Re-use custom socket class for FASP requests (#37925 by @oneiros, [GHSA-46w6-g98f-wxqm](https://github.com/mastodon/mastodon/security/advisories/GHSA-46w6-g98f-wxqm))
+
+### Added
+
+- Add `--suspended-only` option to `tootctl emoji purge` (#37828 and #37861 by @ClearlyClaire and @mjankowski)
+
+### Fixed
+
+- Fix custom emojis not being purged on domain suspension (#37808 by @ClearlyClaire)
+- Fix processing of object updates with duplicate hashtags (#37756 by @ClearlyClaire)
+
+## [4.4.13] - 2026-02-03
+
+### Security
+
+- Fix ActivityPub collection caching logic for pinned posts and featured tags not checking blocked accounts ([GHSA-ccpr-m53r-mfwr](https://github.com/mastodon/mastodon/security/advisories/GHSA-ccpr-m53r-mfwr))
+
+### Fixed
+
+- Fix relationship cache not being cleared when handling account migrations (#37664 by @ClearlyClaire)
+- Fix error when encountering invalid tag in updated object (#37635 by @ClearlyClaire)
+- Fix recycled connections not being immediately closed (#37335 and #37674 by @ClearlyClaire and @shleeable)
+
+## [4.4.12] - 2026-01-20
+
+### Security
+
+- Fix missing limits on various federated properties [GHSA-gg8q-rcg7-p79g](https://github.com/mastodon/mastodon/security/advisories/GHSA-gg8q-rcg7-p79g)
+- Fix remote user suspension bypass [GHSA-5h2f-wg8j-xqwp](https://github.com/mastodon/mastodon/security/advisories/GHSA-5h2f-wg8j-xqwp)
+- Fix missing length limits on some user-provided fields [GHSA-6x3w-9g92-gvf3](https://github.com/mastodon/mastodon/security/advisories/GHSA-6x3w-9g92-gvf3)
+- Fix missing access check for push notification settings update [GHSA-f3q8-7vw3-69v4](https://github.com/mastodon/mastodon/security/advisories/GHSA-f3q8-7vw3-69v4)
+
+### Changed
+
+- Skip tombstone creation on deleting from 404 (#37533 by @ClearlyClaire)
+
+### Fixed
+
+- Fix potential duplicate handling of quote accept/reject/delete (#37537 by @ClearlyClaire)
+- Fix `FeedManager#filter_from_home` error when handling a reblog of a deleted status (#37486 by @ClearlyClaire)
+- Fix needlessly complicated SQL query in status batch removal (#37469 by @ClearlyClaire)
+- Fix `Vary` parsing in cache control enforcement (#37426 by @MegaManSec)
+- Fix thread-unsafe ActivityPub activity dispatch (#37423 by @MegaManSec)
+- Fix SignatureParser accepting duplicate parameters in HTTP Signature header (#37375 by @shleeable)
+
 ## [4.4.11] - 2026-01-07
 
 ### Security

FEDERATION.md

@@ -48,3 +48,22 @@ Mastodon requires all `POST` requests to be signed, and MAY require `GET` reques
 ### Additional documentation
 
 - [Mastodon documentation](https://docs.joinmastodon.org/)
+
+## Size limits
+
+Mastodon imposes a few hard limits on federated content.
+These limits are intended to be very generous and way above what the Mastodon user experience is optimized for, so as to accomodate future changes and unusual or unforeseen usage patterns, while still providing some limits for performance reasons.
+The following table attempts to summary those limits.
+
+| Limited property                                              | Size limit | Consequence of exceeding the limit |
+| ------------------------------------------------------------- | ---------- | ---------------------------------- |
+| Serialized JSON-LD                                            | 1MB        | **Activity is rejected/dropped**   |
+| Profile fields (actor `PropertyValue` attachments) name/value | 2047       | Field name/value is truncated      |
+| Number of profile fields (actor `PropertyValue` attachments)  | 50         | Fields list is truncated           |
+| Poll options (number of `anyOf`/`oneOf` in a `Question`)      | 500        | Items list is truncated            |
+| Account username (actor `preferredUsername`) length           | 2048       | **Actor will be rejected**         |
+| Account display name (actor `name`) length                    | 2048       | Display name will be truncated     |
+| Account note (actor `summary`) length                         | 20kB       | Account note will be truncated     |
+| Account `attributionDomains`                                  | 256        | List will be truncated             |
+| Account aliases (actor `alsoKnownAs`)                         | 256        | List will be truncated             |
+| Custom emoji shortcode (`Emoji` `name`)                       | 2048       | Emoji will be rejected             |

SECURITY.md

@@ -17,5 +17,4 @@ A "vulnerability in Mastodon" is a vulnerability in the code distributed through
 | ------- | ---------------- |
 | 4.4.x   | Yes              |
 | 4.3.x   | Until 2026-05-06 |
-| 4.2.x   | Until 2026-01-08 |
-| < 4.2   | No               |
+| < 4.3   | No               |

app/controllers/activitypub/collections_controller.rb

@@ -4,17 +4,31 @@ class ActivityPub::CollectionsController < ActivityPub::BaseController
   vary_by -> { 'Signature' if authorized_fetch_mode? }
 
   before_action :require_account_signature!, if: :authorized_fetch_mode?
+  before_action :check_authorization
   before_action :set_items
   before_action :set_size
   before_action :set_type
 
   def show
     expires_in 3.minutes, public: public_fetch_mode?
-    render_with_cache json: collection_presenter, content_type: 'application/activity+json', serializer: ActivityPub::CollectionSerializer, adapter: ActivityPub::Adapter
+
+    if @unauthorized
+      render json: collection_presenter, content_type: 'application/activity+json', serializer: ActivityPub::CollectionSerializer, adapter: ActivityPub::Adapter
+    else
+      render_with_cache json: collection_presenter, content_type: 'application/activity+json', serializer: ActivityPub::CollectionSerializer, adapter: ActivityPub::Adapter
+    end
   end
 
   private
 
+  def check_authorization
+    # Because in public fetch mode we cache the response, there would be no
+    # benefit from performing the check below, since a blocked account or domain
+    # would likely be served the cache from the reverse proxy anyway
+
+    @unauthorized = authorized_fetch_mode? && !signed_request_account.nil? && (@account.blocking?(signed_request_account) || (!signed_request_account.domain.nil? && @account.domain_blocking?(signed_request_account.domain)))
+  end
+
   def set_items
     case params[:id]
     when 'featured'
@@ -57,11 +71,7 @@ class ActivityPub::CollectionsController < ActivityPub::BaseController
   end
 
   def for_signed_account
-    # Because in public fetch mode we cache the response, there would be no
-    # benefit from performing the check below, since a blocked account or domain
-    # would likely be served the cache from the reverse proxy anyway
-
-    if authorized_fetch_mode? && !signed_request_account.nil? && (@account.blocking?(signed_request_account) || (!signed_request_account.domain.nil? && @account.domain_blocking?(signed_request_account.domain)))
+    if @unauthorized
       []
     else
       yield

app/controllers/activitypub/inboxes_controller.rb

@@ -3,6 +3,7 @@
 class ActivityPub::InboxesController < ActivityPub::BaseController
   include JsonLdHelper
 
+  before_action :skip_large_payload
   before_action :skip_unknown_actor_activity
   before_action :require_actor_signature!
   skip_before_action :authenticate_user!
@@ -16,6 +17,10 @@ class ActivityPub::InboxesController < ActivityPub::BaseController
 
   private
 
+  def skip_large_payload
+    head 413 if request.content_length > ActivityPub::Activity::MAX_JSON_SIZE
+  end
+
   def skip_unknown_actor_activity
     head 202 if unknown_affected_account?
   end

app/controllers/api/fasp/base_controller.rb

@@ -47,7 +47,7 @@ class Api::Fasp::BaseController < ApplicationController
     provider = nil
 
     Linzer.verify!(request.rack_request, no_older_than: 5.minutes) do |keyid|
-      provider = Fasp::Provider.find(keyid)
+      provider = Fasp::Provider.confirmed.find(keyid)
       Linzer.new_ed25519_public_key(provider.provider_public_key_pem, keyid)
     end
 

app/controllers/api/web/push_subscriptions_controller.rb

@@ -62,7 +62,7 @@ class Api::Web::PushSubscriptionsController < Api::Web::BaseController
   end
 
   def set_push_subscription
-    @push_subscription = ::Web::PushSubscription.find(params[:id])
+    @push_subscription = ::Web::PushSubscription.where(user_id: active_session.user_id).find(params[:id])
   end
 
   def subscription_params

app/controllers/concerns/cache_concern.rb

@@ -19,7 +19,7 @@ module CacheConcern
   # from being used as cache keys, while allowing to `Vary` on them (to not serve
   # anonymous cached data to authenticated requests when authentication matters)
   def enforce_cache_control!
-    vary = response.headers['Vary']&.split&.map { |x| x.strip.downcase }
+    vary = response.headers['Vary'].to_s.split(',').map { |x| x.strip.downcase }.reject(&:empty?)
     return unless vary.present? && %w(cookie authorization signature).any? { |header| vary.include?(header) && request.headers[header].present? }
 
     response.cache_control.replace(private: true, no_store: true)

app/lib/activitypub/activity.rb

@@ -5,6 +5,7 @@ class ActivityPub::Activity
   include Redisable
   include Lockable
 
+  MAX_JSON_SIZE = 1.megabyte
   SUPPORTED_TYPES = %w(Note Question).freeze
   CONVERTED_TYPES = %w(Image Audio Video Article Page Event).freeze
 
@@ -21,14 +22,13 @@ class ActivityPub::Activity
 
   class << self
     def factory(json, account, **)
-      @json = json
-      klass&.new(json, account, **)
+      klass_for(json)&.new(json, account, **)
     end
 
     private
 
-    def klass
-      case @json['type']
+    def klass_for(json)
+      case json['type']
       when 'Create'
         ActivityPub::Activity::Create
       when 'Announce'

app/lib/activitypub/activity/delete.rb

@@ -56,7 +56,7 @@ class ActivityPub::Activity::Delete < ActivityPub::Activity
   end
 
   def revoke_quote
-    @quote = Quote.find_by(approval_uri: object_uri, quoted_account: @account)
+    @quote = Quote.find_by(approval_uri: object_uri, quoted_account: @account, state: [:pending, :accepted])
     return if @quote.nil?
 
     ActivityPub::Forwarder.new(@account, @json, @quote.status).forward! if @quote.status.present?

app/lib/activitypub/activity/update.rb

@@ -32,7 +32,8 @@ class ActivityPub::Activity::Update < ActivityPub::Activity
     @status = Status.find_by(uri: object_uri, account_id: @account.id)
 
     # Ignore updates for old unknown objects, since those are updates we are not interested in
-    return if @status.nil? && object_too_old?
+    # Also ignore unknown objects from suspended users for the same reasons
+    return if @status.nil? && (@account.suspended? || object_too_old?)
 
     # We may be getting `Create` and `Update` out of order
     @status ||= ActivityPub::Activity::Create.new(@json, @account, **@options).perform

app/lib/activitypub/parser/poll_parser.rb

@@ -3,6 +3,10 @@
 class ActivityPub::Parser::PollParser
   include JsonLdHelper
 
+  # Limit the number of items for performance purposes.
+  # We truncate rather than error out to avoid missing the post entirely.
+  MAX_ITEMS = 500
+
   def initialize(json)
     @json = json
   end
@@ -48,6 +52,6 @@ class ActivityPub::Parser::PollParser
   private
 
   def items
-    @json['anyOf'] || @json['oneOf']
+    (@json['anyOf'] || @json['oneOf'])&.take(MAX_ITEMS)
   end
 end

app/lib/connection_pool/shared_connection_pool.rb

@@ -41,12 +41,17 @@ class ConnectionPool::SharedConnectionPool < ConnectionPool
       # ConnectionPool 2.4+ calls `checkin(force: true)` after fork.
       # When this happens, we should remove all connections from Thread.current
 
-      ::Thread.current.keys.each do |name| # rubocop:disable Style/HashEachMethods
-        next unless name.to_s.start_with?("#{@key}-")
+      connection_keys = ::Thread.current.keys.select { |key| key.to_s.start_with?("#{@key}-") && !key.to_s.start_with?("#{@key_count}-") }
+      count_keys = ::Thread.current.keys.select { |key| key.to_s.start_with?("#{@key_count}-") }
 
-        @available.push(::Thread.current[name])
-        ::Thread.current[name] = nil
+      connection_keys.each do |key|
+        @available.push(::Thread.current[key])
+        ::Thread.current[key] = nil
       end
+      count_keys.each do |key|
+        ::Thread.current[key] = nil
+      end
+
     elsif ::Thread.current[key(preferred_tag)]
       if ::Thread.current[key_count(preferred_tag)] == 1
         @available.push(::Thread.current[key(preferred_tag)])

app/lib/connection_pool/shared_timed_stack.rb

@@ -70,6 +70,7 @@ class ConnectionPool::SharedTimedStack
     if @created == @max && !@queue.empty?
       throw_away_connection = @queue.pop
       @tagged_queue[throw_away_connection.site].delete(throw_away_connection)
+      throw_away_connection.close
       @create_block.call(preferred_tag)
     elsif @created != @max
       connection = @create_block.call(preferred_tag)

app/lib/fasp/request.rb

@@ -29,7 +29,7 @@ class Fasp::Request
     response = HTTP
                .headers(headers)
                .use(http_signature: { key:, covered_components: COVERED_COMPONENTS })
-               .send(verb, url, body:)
+               .send(verb, url, body:, socket_class: ::Request::Socket)
 
     validate!(response)
 

app/lib/feed_manager.rb

@@ -500,6 +500,7 @@ class FeedManager
     return :filter    if status.reply? && (status.in_reply_to_id.nil? || status.in_reply_to_account_id.nil?)
     return :skip_home if timeline_type != :list && crutches[:exclusive_list_users][status.account_id].present?
     return :filter    if crutches[:languages][status.account_id].present? && status.language.present? && !crutches[:languages][status.account_id].include?(status.language)
+    return :filter    if status.reblog? && status.reblog.blank?
 
     check_for_blocks = crutches[:active_mentions][status.id] || []
     check_for_blocks.push(status.account_id)

app/lib/request.rb

@@ -349,5 +349,5 @@ class Request
     end
   end
 
-  private_constant :ClientLimit, :Socket, :ProxySocket
+  private_constant :ClientLimit
 end

app/lib/signature_parser.rb

@@ -25,9 +25,13 @@ class SignatureParser
 
     # Use `skip` instead of `scan` as we only care about the subgroups
     while scanner.skip(PARAM_RE)
+      key = scanner[:key]
+      # Detect a duplicate key
+      raise Mastodon::SignatureVerificationError, 'Error parsing signature with duplicate keys' if params.key?(key)
+
       # This is not actually correct with regards to quoted pairs, but it's consistent
       # with our previous implementation, and good enough in practice.
-      params[scanner[:key]] = scanner[:value] || scanner[:quoted_value][1...-1]
+      params[key] = scanner[:value] || scanner[:quoted_value][1...-1]
 
       scanner.skip(/\s*/)
       return params if scanner.eos?

app/models/account.rb

@@ -78,6 +78,13 @@ class Account < ApplicationRecord
   DISPLAY_NAME_LENGTH_LIMIT = (ENV['MAX_DISPLAY_NAME_CHARS'] || 30).to_i
   NOTE_LENGTH_LIMIT = (ENV['MAX_BIO_CHARS'] || 500).to_i
 
+  # Hard limits for federated content
+  USERNAME_LENGTH_HARD_LIMIT = 2048
+  DISPLAY_NAME_LENGTH_HARD_LIMIT = 2048
+  NOTE_LENGTH_HARD_LIMIT = 20.kilobytes
+  ATTRIBUTION_DOMAINS_HARD_LIMIT = 256
+  ALSO_KNOWN_AS_HARD_LIMIT = 256
+
   AUTOMATED_ACTOR_TYPES = %w(Application Service).freeze
 
   include Attachmentable # Load prior to Avatar & Header concerns
@@ -109,7 +116,7 @@ class Account < ApplicationRecord
   validates_with UniqueUsernameValidator, if: -> { will_save_change_to_username? }
 
   # Remote user validations, also applies to internal actors
-  validates :username, format: { with: USERNAME_ONLY_RE }, if: -> { (remote? || actor_type_application?) && will_save_change_to_username? }
+  validates :username, format: { with: USERNAME_ONLY_RE }, length: { maximum: USERNAME_LENGTH_HARD_LIMIT }, if: -> { (remote? || actor_type_application?) && will_save_change_to_username? }
 
   # Remote user validations
   validates :uri, presence: true, unless: :local?, on: :create

app/models/custom_emoji.rb

@@ -27,6 +27,8 @@ class CustomEmoji < ApplicationRecord
   LOCAL_LIMIT = (ENV['MAX_EMOJI_SIZE'] || 256.kilobytes).to_i
   LIMIT       = [LOCAL_LIMIT, (ENV['MAX_REMOTE_EMOJI_SIZE'] || 256.kilobytes).to_i].max
   MINIMUM_SHORTCODE_SIZE = 2
+  MAX_SHORTCODE_SIZE = 128
+  MAX_FEDERATED_SHORTCODE_SIZE = 2048
 
   SHORTCODE_RE_FRAGMENT = '[a-zA-Z0-9_]{2,}'
 
@@ -48,7 +50,8 @@ class CustomEmoji < ApplicationRecord
   validates_attachment :image, content_type: { content_type: IMAGE_MIME_TYPES }, presence: true
   validates_attachment_size :image, less_than: LIMIT, unless: :local?
   validates_attachment_size :image, less_than: LOCAL_LIMIT, if: :local?
-  validates :shortcode, uniqueness: { scope: :domain }, format: { with: SHORTCODE_ONLY_RE }, length: { minimum: MINIMUM_SHORTCODE_SIZE }
+  validates :shortcode, uniqueness: { scope: :domain }, format: { with: SHORTCODE_ONLY_RE }, length: { minimum: MINIMUM_SHORTCODE_SIZE, maximum: MAX_FEDERATED_SHORTCODE_SIZE }
+  validates :shortcode, length: { maximum: MAX_SHORTCODE_SIZE }, if: :local?
 
   scope :local, -> { where(domain: nil) }
   scope :remote, -> { where.not(domain: nil) }

app/models/custom_filter.rb

@@ -30,6 +30,8 @@ class CustomFilter < ApplicationRecord
 
   EXPIRATION_DURATIONS = [30.minutes, 1.hour, 6.hours, 12.hours, 1.day, 1.week].freeze
 
+  TITLE_LENGTH_LIMIT = 256
+
   include Expireable
   include Redisable
 
@@ -41,6 +43,7 @@ class CustomFilter < ApplicationRecord
   accepts_nested_attributes_for :keywords, reject_if: :all_blank, allow_destroy: true
 
   validates :title, :context, presence: true
+  validates :title, length: { maximum: TITLE_LENGTH_LIMIT }
   validate :context_must_be_valid
 
   normalizes :context, with: ->(context) { context.map(&:strip).filter_map(&:presence) }

app/models/custom_filter_keyword.rb

@@ -17,7 +17,9 @@ class CustomFilterKeyword < ApplicationRecord
 
   belongs_to :custom_filter
 
-  validates :keyword, presence: true
+  KEYWORD_LENGTH_LIMIT = 512
+
+  validates :keyword, presence: true, length: { maximum: KEYWORD_LENGTH_LIMIT }
 
   alias_attribute :phrase, :keyword
 

app/models/fasp/provider.rb

@@ -34,6 +34,7 @@ class Fasp::Provider < ApplicationRecord
   before_create :create_keypair
   after_commit :update_remote_capabilities
 
+  scope :confirmed, -> { where(confirmed: true) }
   scope :with_capability, lambda { |capability_name|
     where('fasp_providers.capabilities @> ?::jsonb', "[{\"id\": \"#{capability_name}\", \"enabled\": true}]")
   }

app/models/list.rb

@@ -17,6 +17,7 @@ class List < ApplicationRecord
   include Paginable
 
   PER_ACCOUNT_LIMIT = 50
+  TITLE_LENGTH_LIMIT = 256
 
   enum :replies_policy, { list: 0, followed: 1, none: 2 }, prefix: :show, validate: true
 
@@ -26,7 +27,7 @@ class List < ApplicationRecord
   has_many :accounts, through: :list_accounts
   has_many :active_accounts, -> { merge(ListAccount.active) }, through: :list_accounts, source: :account
 
-  validates :title, presence: true
+  validates :title, presence: true, length: { maximum: TITLE_LENGTH_LIMIT }
 
   validate :validate_account_lists_limit, on: :create
 

app/models/quote.rb

@@ -42,9 +42,9 @@ class Quote < ApplicationRecord
 
   def reject!
     if accepted?
-      update!(state: :revoked)
+      update!(state: :revoked, approval_uri: nil)
     elsif !revoked?
-      update!(state: :rejected)
+      update!(state: :rejected, approval_uri: nil)
     end
   end
 

app/services/activitypub/fetch_remote_status_service.rb

@@ -92,7 +92,6 @@ class ActivityPub::FetchRemoteStatusService < BaseService
       existing_status = Status.remote.find_by(uri: uri)
       if existing_status&.distributable?
         Rails.logger.debug { "FetchRemoteStatusService - Got 404 for orphaned status with URI #{uri}, deleting" }
-        Tombstone.find_or_create_by(uri: uri, account: existing_status.account)
         RemoveStatusService.new.call(existing_status, redraft: false)
       end
     end

app/services/activitypub/process_account_service.rb

@@ -6,6 +6,7 @@ class ActivityPub::ProcessAccountService < BaseService
   include Redisable
   include Lockable
 
+  MAX_PROFILE_FIELDS = 50
   SUBDOMAINS_RATELIMIT = 10
   DISCOVERIES_PER_REQUEST = 400
 
@@ -122,15 +123,15 @@ class ActivityPub::ProcessAccountService < BaseService
 
   def set_immediate_attributes!
     @account.featured_collection_url = valid_collection_uri(@json['featured'])
-    @account.display_name            = @json['name'] || ''
-    @account.note                    = @json['summary'] || ''
+    @account.display_name            = (@json['name'] || '')[0...(Account::DISPLAY_NAME_LENGTH_HARD_LIMIT)]
+    @account.note                    = (@json['summary'] || '')[0...(Account::NOTE_LENGTH_HARD_LIMIT)]
     @account.locked                  = @json['manuallyApprovesFollowers'] || false
     @account.fields                  = property_values || {}
-    @account.also_known_as           = as_array(@json['alsoKnownAs'] || []).map { |item| value_or_id(item) }
+    @account.also_known_as           = as_array(@json['alsoKnownAs'] || []).take(Account::ALSO_KNOWN_AS_HARD_LIMIT).map { |item| value_or_id(item) }
     @account.discoverable            = @json['discoverable'] || false
     @account.indexable               = @json['indexable'] || false
     @account.memorial                = @json['memorial'] || false
-    @account.attribution_domains     = as_array(@json['attributionDomains'] || []).map { |item| value_or_id(item) }
+    @account.attribution_domains     = as_array(@json['attributionDomains'] || []).take(Account::ATTRIBUTION_DOMAINS_HARD_LIMIT).map { |item| value_or_id(item) }
   end
 
   def set_fetchable_key!
@@ -251,7 +252,10 @@ class ActivityPub::ProcessAccountService < BaseService
   def property_values
     return unless @json['attachment'].is_a?(Array)
 
-    as_array(@json['attachment']).select { |attachment| attachment['type'] == 'PropertyValue' }.map { |attachment| attachment.slice('name', 'value') }
+    as_array(@json['attachment'])
+      .select { |attachment| attachment['type'] == 'PropertyValue' }
+      .take(MAX_PROFILE_FIELDS)
+      .map { |attachment| attachment.slice('name', 'value') }
   end
 
   def mismatching_origin?(url)

app/services/activitypub/process_status_update_service.rb

@@ -205,7 +205,11 @@ class ActivityPub::ProcessStatusUpdateService < BaseService
 
   def update_tags!
     previous_tags = @status.tags.to_a
-    current_tags = @status.tags = Tag.find_or_create_by_names(@raw_tags)
+    current_tags = @status.tags = @raw_tags.flat_map do |tag|
+      Tag.find_or_create_by_names([tag]).filter(&:valid?)
+    rescue ActiveRecord::RecordInvalid
+      []
+    end.uniq
 
     return unless @status.distributable?
 

app/services/batched_remove_status_service.rb

@@ -34,7 +34,7 @@ class BatchedRemoveStatusService < BaseService
     # transaction lock the database, but we use the delete method instead
     # of destroy to avoid all callbacks. We rely on foreign keys to
     # cascade the delete faster without loading the associations.
-    statuses_and_reblogs.each_slice(50) { |slice| Status.where(id: slice.map(&:id)).delete_all }
+    statuses_and_reblogs.each_slice(50) { |slice| Status.unscoped.where(id: slice.pluck(:id)).delete_all }
 
     # Since we skipped all callbacks, we also need to manually
     # deindex the statuses

app/services/block_domain_service.rb

@@ -29,7 +29,12 @@ class BlockDomainService < BaseService
       suspend_accounts!
     end
 
-    DomainClearMediaWorker.perform_async(domain_block.id) if domain_block.reject_media?
+    if domain_block.suspend?
+      # Account images and attachments are already handled by `suspend_accounts!`
+      PurgeCustomEmojiWorker.perform_async(blocked_domain)
+    elsif domain_block.reject_media?
+      DomainClearMediaWorker.perform_async(domain_block.id)
+    end
   end
 
   def silence_accounts!

app/services/fan_out_on_write_service.rb

@@ -14,6 +14,8 @@ class FanOutOnWriteService < BaseService
     @account   = status.account
     @options   = options
 
+    return if @status.proper.account.suspended?
+
     check_race_condition!
     warm_payload_cache!
 

app/workers/move_worker.rb

@@ -64,6 +64,16 @@ class MoveWorker
                    .in_batches do |follows|
       ListAccount.where(follow: follows).in_batches.update_all(account_id: @target_account.id)
       num_moved += follows.update_all(target_account_id: @target_account.id)
+
+      # Clear any relationship cache, since callbacks are not called
+      Rails.cache.delete_multi(follows.flat_map do |follow|
+        [
+          ['relationship', follow.account_id, follow.target_account_id],
+          ['relationship', follow.target_account_id, follow.account_id],
+          ['relationship', follow.account_id, @target_account.id],
+          ['relationship', @target_account.id, follow.account_id],
+        ]
+      end)
     end
 
     num_moved

app/workers/purge_custom_emoji_worker.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+class PurgeCustomEmojiWorker
+  include Sidekiq::IterableJob
+
+  def build_enumerator(domain, cursor:)
+    return if domain.blank?
+
+    active_record_batches_enumerator(CustomEmoji.by_domain_and_subdomains(domain), cursor:)
+  end
+
+  def each_iteration(custom_emojis, _domain)
+    AttachmentBatch.new(CustomEmoji, custom_emojis).delete
+  end
+end

docker-compose.yml

@@ -59,7 +59,7 @@ services:
   web:
     # You can uncomment the following line if you want to not use the prebuilt image, for example if you have local code changes
     # build: .
-    image: ghcr.io/glitch-soc/mastodon:v4.4.11
+    image: ghcr.io/glitch-soc/mastodon:v4.4.14
     restart: always
     env_file: .env.production
     command: bundle exec puma -C config/puma.rb
@@ -83,7 +83,7 @@ services:
     # build:
     #   dockerfile: ./streaming/Dockerfile
     #   context: .
-    image: ghcr.io/glitch-soc/mastodon-streaming:v4.4.11
+    image: ghcr.io/glitch-soc/mastodon-streaming:v4.4.14
     restart: always
     env_file: .env.production
     command: node ./streaming/index.js
@@ -102,7 +102,7 @@ services:
   sidekiq:
     # You can uncomment the following line if you want to not use the prebuilt image, for example if you have local code changes
     # build: .
-    image: ghcr.io/glitch-soc/mastodon:v4.4.11
+    image: ghcr.io/glitch-soc/mastodon:v4.4.14
     restart: always
     env_file: .env.production
     command: bundle exec sidekiq

lib/mastodon/cli/emoji.rb

@@ -109,15 +109,27 @@ module Mastodon::CLI
     end
 
     option :remote_only, type: :boolean
+    option :suspended_only, type: :boolean
     desc 'purge', 'Remove all custom emoji'
     long_desc <<-LONG_DESC
       Removes all custom emoji.
 
       With the --remote-only option, only remote emoji will be deleted.
+
+      With the --suspended-only option, only emoji from suspended servers will be deleted.
     LONG_DESC
     def purge
-      scope = options[:remote_only] ? CustomEmoji.remote : CustomEmoji
-      scope.in_batches.destroy_all
+      if options[:suspended_only]
+        DomainBlock.where(severity: :suspend).find_each do |domain_block|
+          CustomEmoji.by_domain_and_subdomains(domain_block.domain).find_in_batches do |custom_emojis|
+            AttachmentBatch.new(CustomEmoji, custom_emojis).delete
+          end
+        end
+      else
+        scope = options[:remote_only] ? CustomEmoji.remote : CustomEmoji
+        scope.in_batches.destroy_all
+      end
+
       say('OK', :green)
     end
 

lib/mastodon/version.rb

@@ -13,7 +13,7 @@ module Mastodon
     end
 
     def patch
-      11
+      14
     end
 
     def default_prerelease

spec/lib/fasp/request_spec.rb

@@ -55,6 +55,24 @@ RSpec.describe Fasp::Request do
         end
       end
     end
+
+    context 'when the provider host name resolves to a private address' do
+      around do |example|
+        WebMock.disable!
+        example.run
+        WebMock.enable!
+      end
+
+      it 'raises Mastodon::ValidationError' do
+        resolver = instance_double(Resolv::DNS)
+
+        allow(resolver).to receive(:getaddresses).with('reqprov.example.com').and_return(%w(0.0.0.0 2001:db8::face))
+        allow(resolver).to receive(:timeouts=).and_return(nil)
+        allow(Resolv::DNS).to receive(:open).and_yield(resolver)
+
+        expect { subject.send(method, '/test_path') }.to raise_error(Mastodon::ValidationError)
+      end
+    end
   end
 
   describe '#get' do

spec/lib/mastodon/cli/emoji_spec.rb

@@ -23,6 +23,36 @@ RSpec.describe Mastodon::CLI::Emoji do
           .to output_results('OK')
       end
     end
+
+    context 'with --suspended-only and existing custom emoji on blocked servers' do
+      let(:blocked_domain) { 'evil.com' }
+      let(:blocked_subdomain) { 'subdomain.evil.org' }
+      let(:blocked_domain_without_emoji) { 'blocked.com' }
+      let(:silenced_domain) { 'silenced.com' }
+
+      let(:options) { { suspended_only: true } }
+
+      before do
+        Fabricate(:custom_emoji)
+        Fabricate(:custom_emoji, domain: blocked_domain)
+        Fabricate(:custom_emoji, domain: blocked_subdomain)
+        Fabricate(:custom_emoji, domain: silenced_domain)
+
+        Fabricate(:domain_block, severity: :suspend, domain: blocked_domain)
+        Fabricate(:domain_block, severity: :suspend, domain: 'evil.org')
+        Fabricate(:domain_block, severity: :suspend, domain: blocked_domain_without_emoji)
+        Fabricate(:domain_block, severity: :silence, domain: silenced_domain)
+      end
+
+      it 'reports a successful purge' do
+        expect { subject }
+          .to output_results('OK')
+          .and change { CustomEmoji.by_domain_and_subdomains(blocked_domain).count }.to(0)
+          .and change { CustomEmoji.by_domain_and_subdomains('evil.org').count }.to(0)
+          .and not_change { CustomEmoji.by_domain_and_subdomains(silenced_domain).count }
+          .and(not_change { CustomEmoji.local.count })
+      end
+    end
   end
 
   describe '#import' do

spec/requests/api/fasp/data_sharing/v0/backfill_requests_spec.rb

@@ -6,34 +6,33 @@ RSpec.describe 'Api::Fasp::DataSharing::V0::BackfillRequests', feature: :fasp do
   include ProviderRequestHelper
 
   describe 'POST /api/fasp/data_sharing/v0/backfill_requests' do
-    let(:provider) { Fabricate(:fasp_provider) }
+    subject do
+      post api_fasp_data_sharing_v0_backfill_requests_path, headers:, params:, as: :json
+    end
+
+    let(:provider) { Fabricate(:confirmed_fasp) }
+    let(:params) { { category: 'content', maxCount: 10 } }
+    let(:headers) do
+      request_authentication_headers(provider,
+                                     url: api_fasp_data_sharing_v0_backfill_requests_url,
+                                     method: :post,
+                                     body: params)
+    end
+
+    it_behaves_like 'forbidden for unconfirmed provider'
 
     context 'with valid parameters' do
       it 'creates a new backfill request' do
-        params = { category: 'content', maxCount: 10 }
-        headers = request_authentication_headers(provider,
-                                                 url: api_fasp_data_sharing_v0_backfill_requests_url,
-                                                 method: :post,
-                                                 body: params)
-
-        expect do
-          post api_fasp_data_sharing_v0_backfill_requests_path, headers:, params:, as: :json
-        end.to change(Fasp::BackfillRequest, :count).by(1)
+        expect { subject }.to change(Fasp::BackfillRequest, :count).by(1)
         expect(response).to have_http_status(201)
       end
     end
 
     context 'with invalid parameters' do
-      it 'does not create a backfill request' do
-        params = { category: 'unknown', maxCount: 10 }
-        headers = request_authentication_headers(provider,
-                                                 url: api_fasp_data_sharing_v0_backfill_requests_url,
-                                                 method: :post,
-                                                 body: params)
+      let(:params) { { category: 'unknown', maxCount: 10 } }
 
-        expect do
-          post api_fasp_data_sharing_v0_backfill_requests_path, headers:, params:, as: :json
-        end.to_not change(Fasp::BackfillRequest, :count)
+      it 'does not create a backfill request' do
+        expect { subject }.to_not change(Fasp::BackfillRequest, :count)
         expect(response).to have_http_status(422)
       end
     end

spec/requests/api/fasp/data_sharing/v0/continuations_spec.rb

@@ -6,15 +6,22 @@ RSpec.describe 'Api::Fasp::DataSharing::V0::Continuations', feature: :fasp do
   include ProviderRequestHelper
 
   describe 'POST /api/fasp/data_sharing/v0/backfill_requests/:id/continuations' do
-    let(:backfill_request) { Fabricate(:fasp_backfill_request) }
-    let(:provider) { backfill_request.fasp_provider }
+    subject do
+      post api_fasp_data_sharing_v0_backfill_request_continuation_path(backfill_request), headers:, as: :json
+    end
+
+    let(:provider) { Fabricate(:confirmed_fasp) }
+    let(:backfill_request) { Fabricate(:fasp_backfill_request, fasp_provider: provider) }
+    let(:headers) do
+      request_authentication_headers(provider,
+                                     url: api_fasp_data_sharing_v0_backfill_request_continuation_url(backfill_request),
+                                     method: :post)
+    end
+
+    it_behaves_like 'forbidden for unconfirmed provider'
 
     it 'queues a job to continue the given backfill request' do
-      headers = request_authentication_headers(provider,
-                                               url: api_fasp_data_sharing_v0_backfill_request_continuation_url(backfill_request),
-                                               method: :post)
-
-      post api_fasp_data_sharing_v0_backfill_request_continuation_path(backfill_request), headers:, as: :json
+      subject
       expect(response).to have_http_status(204)
       expect(Fasp::BackfillWorker).to have_enqueued_sidekiq_job(backfill_request.id)
     end

spec/requests/api/fasp/data_sharing/v0/event_subscriptions_spec.rb

@@ -6,51 +6,57 @@ RSpec.describe 'Api::Fasp::DataSharing::V0::EventSubscriptions', feature: :fasp
   include ProviderRequestHelper
 
   describe 'POST /api/fasp/data_sharing/v0/event_subscriptions' do
-    let(:provider) { Fabricate(:fasp_provider) }
+    subject do
+      post api_fasp_data_sharing_v0_event_subscriptions_path, headers:, params:, as: :json
+    end
+
+    let(:provider) { Fabricate(:confirmed_fasp) }
+    let(:params) { { category: 'content', subscriptionType: 'lifecycle', maxBatchSize: 10 } }
+    let(:headers) do
+      request_authentication_headers(provider,
+                                     url: api_fasp_data_sharing_v0_event_subscriptions_url,
+                                     method: :post,
+                                     body: params)
+    end
+
+    it_behaves_like 'forbidden for unconfirmed provider'
 
     context 'with valid parameters' do
       it 'creates a new subscription' do
-        params = { category: 'content', subscriptionType: 'lifecycle', maxBatchSize: 10 }
-        headers = request_authentication_headers(provider,
-                                                 url: api_fasp_data_sharing_v0_event_subscriptions_url,
-                                                 method: :post,
-                                                 body: params)
-
         expect do
-          post api_fasp_data_sharing_v0_event_subscriptions_path, headers:, params:, as: :json
+          subject
         end.to change(Fasp::Subscription, :count).by(1)
         expect(response).to have_http_status(201)
       end
     end
 
     context 'with invalid parameters' do
-      it 'does not create a subscription' do
-        params = { category: 'unknown' }
-        headers = request_authentication_headers(provider,
-                                                 url: api_fasp_data_sharing_v0_event_subscriptions_url,
-                                                 method: :post,
-                                                 body: params)
+      let(:params) { { category: 'unknown' } }
 
-        expect do
-          post api_fasp_data_sharing_v0_event_subscriptions_path, headers:, params:, as: :json
-        end.to_not change(Fasp::Subscription, :count)
+      it 'does not create a subscription' do
+        expect { subject }.to_not change(Fasp::Subscription, :count)
         expect(response).to have_http_status(422)
       end
     end
   end
 
   describe 'DELETE /api/fasp/data_sharing/v0/event_subscriptions/:id' do
-    let(:subscription) { Fabricate(:fasp_subscription) }
-    let(:provider) { subscription.fasp_provider }
+    subject do
+      delete api_fasp_data_sharing_v0_event_subscription_path(subscription), headers:, as: :json
+    end
+
+    let(:provider) { Fabricate(:confirmed_fasp) }
+    let!(:subscription) { Fabricate(:fasp_subscription, fasp_provider: provider) }
+    let(:headers) do
+      request_authentication_headers(provider,
+                                     url: api_fasp_data_sharing_v0_event_subscription_url(subscription),
+                                     method: :delete)
+    end
+
+    it_behaves_like 'forbidden for unconfirmed provider'
 
     it 'deletes the subscription' do
-      headers = request_authentication_headers(provider,
-                                               url: api_fasp_data_sharing_v0_event_subscription_url(subscription),
-                                               method: :delete)
-
-      expect do
-        delete api_fasp_data_sharing_v0_event_subscription_path(subscription), headers:, as: :json
-      end.to change(Fasp::Subscription, :count).by(-1)
+      expect { subject }.to change(Fasp::Subscription, :count).by(-1)
       expect(response).to have_http_status(204)
     end
   end

spec/requests/api/fasp/debug/v0/callback/responses_spec.rb

@@ -6,18 +6,23 @@ RSpec.describe 'Api::Fasp::Debug::V0::Callback::Responses', feature: :fasp do
   include ProviderRequestHelper
 
   describe 'POST /api/fasp/debug/v0/callback/responses' do
-    let(:provider) { Fabricate(:debug_fasp) }
+    subject do
+      post api_fasp_debug_v0_callback_responses_path, headers:, params: payload, as: :json
+    end
+
+    let(:provider) { Fabricate(:confirmed_fasp) }
+    let(:payload) { { test: 'call' } }
+    let(:headers) do
+      request_authentication_headers(provider,
+                                     url: api_fasp_debug_v0_callback_responses_url,
+                                     method: :post,
+                                     body: payload)
+    end
+
+    it_behaves_like 'forbidden for unconfirmed provider'
 
     it 'create a record of the callback' do
-      payload = { test: 'call' }
-      headers = request_authentication_headers(provider,
-                                               url: api_fasp_debug_v0_callback_responses_url,
-                                               method: :post,
-                                               body: payload)
-
-      expect do
-        post api_fasp_debug_v0_callback_responses_path, headers:, params: payload, as: :json
-      end.to change(Fasp::DebugCallback, :count).by(1)
+      expect { subject }.to change(Fasp::DebugCallback, :count).by(1)
       expect(response).to have_http_status(201)
 
       debug_callback = Fasp::DebugCallback.last

spec/requests/api/web/push_subscriptions_spec.rb

@@ -65,9 +65,10 @@ RSpec.describe 'API Web Push Subscriptions' do
   end
 
   describe 'PUT /api/web/push_subscriptions/:id' do
-    before { sign_in Fabricate :user }
+    before { sign_in user }
 
-    let(:subscription) { Fabricate :web_push_subscription }
+    let(:user) { Fabricate(:user) }
+    let(:subscription) { Fabricate(:web_push_subscription, user: user) }
 
     it 'gracefully handles invalid nested params' do
       put api_web_push_subscription_path(subscription), params: { data: 'invalid' }

spec/services/activitypub/process_status_update_service_spec.rb

@@ -258,6 +258,9 @@ RSpec.describe ActivityPub::ProcessStatusUpdateService do
           tag: [
             { type: 'Hashtag', name: 'foo' },
             { type: 'Hashtag', name: 'bar' },
+            { type: 'Hashtag', name: '#2024' },
+            { type: 'Hashtag', name: 'Foo Bar' },
+            { type: 'Hashtag', name: 'FooBar' },
           ],
         }
       end
@@ -269,7 +272,7 @@ RSpec.describe ActivityPub::ProcessStatusUpdateService do
 
       it 'updates tags and featured tags' do
         expect { subject.call(status, json, json) }
-          .to change { status.tags.reload.pluck(:name) }.from(contain_exactly('test', 'foo')).to(contain_exactly('foo', 'bar'))
+          .to change { status.tags.reload.pluck(:name) }.from(contain_exactly('test', 'foo')).to(contain_exactly('foo', 'bar', 'foobar'))
           .and change { status.account.featured_tags.find_by(name: 'test').statuses_count }.by(-1)
           .and change { status.account.featured_tags.find_by(name: 'bar').statuses_count }.by(1)
           .and change { status.account.featured_tags.find_by(name: 'bar').last_status_at }.from(nil).to(be_present)

spec/support/examples/fasp/api.rb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+RSpec.shared_examples 'forbidden for unconfirmed provider' do
+  context 'when the requesting provider is unconfirmed' do
+    let(:provider) { Fabricate(:fasp_provider) }
+
+    it 'returns http unauthorized' do
+      subject
+
+      expect(response).to have_http_status(401)
+    end
+  end
+end

spec/workers/fasp/announce_account_lifecycle_event_worker_spec.rb

@@ -6,10 +6,11 @@ RSpec.describe Fasp::AnnounceAccountLifecycleEventWorker do
   include ProviderRequestHelper
 
   let(:account_uri) { 'https://masto.example.com/accounts/1' }
+  let(:provider) { Fabricate(:confirmed_fasp) }
   let(:subscription) do
-    Fabricate(:fasp_subscription, category: 'account')
+    Fabricate(:fasp_subscription, fasp_provider: provider, category: 'account')
   end
-  let(:provider) { subscription.fasp_provider }
+
   let!(:stubbed_request) do
     stub_provider_request(provider,
                           method: :post,

spec/workers/fasp/announce_content_lifecycle_event_worker_spec.rb

@@ -6,10 +6,11 @@ RSpec.describe Fasp::AnnounceContentLifecycleEventWorker do
   include ProviderRequestHelper
 
   let(:status_uri) { 'https://masto.example.com/status/1' }
+  let(:provider) { Fabricate(:confirmed_fasp) }
   let(:subscription) do
-    Fabricate(:fasp_subscription)
+    Fabricate(:fasp_subscription, fasp_provider: provider)
   end
-  let(:provider) { subscription.fasp_provider }
+
   let!(:stubbed_request) do
     stub_provider_request(provider,
                           method: :post,

spec/workers/fasp/announce_trend_worker_spec.rb

@@ -6,14 +6,15 @@ RSpec.describe Fasp::AnnounceTrendWorker do
   include ProviderRequestHelper
 
   let(:status) { Fabricate(:status) }
+  let(:provider) { Fabricate(:confirmed_fasp) }
   let(:subscription) do
     Fabricate(:fasp_subscription,
+              fasp_provider: provider,
               category: 'content',
               subscription_type: 'trends',
               threshold_timeframe: 15,
               threshold_likes: 2)
   end
-  let(:provider) { subscription.fasp_provider }
   let!(:stubbed_request) do
     stub_provider_request(provider,
                           method: :post,

spec/workers/fasp/backfill_worker_spec.rb

@@ -5,8 +5,10 @@ require 'rails_helper'
 RSpec.describe Fasp::BackfillWorker do
   include ProviderRequestHelper
 
-  let(:backfill_request) { Fabricate(:fasp_backfill_request) }
-  let(:provider) { backfill_request.fasp_provider }
+  subject { described_class.new.perform(backfill_request.id) }
+
+  let(:provider) { Fabricate(:confirmed_fasp) }
+  let(:backfill_request) { Fabricate(:fasp_backfill_request, fasp_provider: provider) }
   let(:status) { Fabricate(:status) }
   let!(:stubbed_request) do
     stub_provider_request(provider,

spec/workers/purge_custom_emoji_worker_spec.rb

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+require 'rails_helper'
+
+RSpec.describe PurgeCustomEmojiWorker do
+  let(:worker) { described_class.new }
+
+  let(:domain) { 'evil' }
+
+  before do
+    Fabricate(:custom_emoji)
+    Fabricate(:custom_emoji, domain: 'example.com')
+    Fabricate.times(5, :custom_emoji, domain: domain)
+  end
+
+  describe '#perform' do
+    context 'when domain is nil' do
+      it 'does not delete emojis' do
+        expect { worker.perform(nil) }
+          .to_not(change(CustomEmoji, :count))
+      end
+    end
+
+    context 'when passing a domain' do
+      it 'deletes emojis from this domain only' do
+        expect { worker.perform(domain) }
+          .to change { CustomEmoji.where(domain: domain).count }.to(0)
+          .and not_change { CustomEmoji.local.count }
+          .and(not_change { CustomEmoji.where(domain: 'example.com').count })
+      end
+    end
+  end
+end