PR above got merged, moving back to the backlog, we'll continue working on this once we get closer to an opensearch version that has this feature

In T347282#10859528, @gmodena wrote:

In this case, would you suggest making the HTTP call synchronous ? IIRC we tried this early on, but the interation between python, beam and Flink did lead to very high latencies even for very low throughput streams. I'd need to revisit.

In T394791#10858155, @daniel wrote:

It's unclear to me why this implements PageDeleteHook - that hook runs *before* deletion, the deletion may not even happen. Since the handler method does the same thing as the one for PageDeleteCompleteHook, it seems redundant. Perhaps it is the result of a misundersatnding.

From the code:

		// We use this to pick up redirects so we can update their targets.
		// Can't re-use PageDeleteComplete because the page info's
		// already gone
		// If we abort or fail deletion it's no big deal because this will
		// end up being a no-op when it executes.

There's the same hack in EventBus but I believe the new event system solves this issue by keeping track of the redirect target with \MediaWiki\Page\Event\PageDeletedEvent::wasRedirect() and \MediaWiki\Page\Event\PageDeletedEvent::getRedirectTargetBefore() and this will no longer be necessary.

Here's my understanding of the possible solutions:

• use the keyed state: will probably have a huge impact on throughput & latency, a new batch will be created per key leading to most batches being rather small (1 event) and always fired by the timer
• use the operator state: probably the most natural solution to keep the current logic, in-flight events will be re-played on restarts, issue is that CheckpointedFunction does not appear to be available with pyflink
• use the AsyncIO operator, should be the preferred approach, this solutions provides delivery guarantees with no extra duplicates, unfortunately not available with pyflink
• use the flink parallelism, instead of batching events we could achieve higher concurrency by simply setting the parallelism of the fetch operator (12 to match the default process_max_workers_default), not the best use of resources but probably acceptable for the expected throughput of the page_change stream

In P76406#307378, @OKarakaya-WMF wrote:

Hi @dcausse ,

Thank you very much for sharing this. Works like a charm!
Looking into the data, we have article topic predictions with scores, and if the article has a link recommendation as a boolean.
This is awesome.
So, if we want to find the add-a-link recommendation scores, we should look in either maria db directly or another index.

In T388538#10851661, @Clement_Goubert wrote:

In T388538#10851630, @dcausse wrote:

pinging @hoo & Wikidata for visibility on the work on mediawiki_job_wikidata-updateQueryServiceLag.timer

Should that job alert to Wikidata rather than Discovery-Search ?

pinging @hoo & Wikidata for visibility on the work on mediawiki_job_wikidata-updateQueryServiceLag.timer

In T388538#10851450, @Clement_Goubert wrote:

It's when trying to run on s8, so wikidata, yes. I could also just remove s8 from the shards the script is running on?

Sure no need to run on s8 indeed.

@Clement_Goubert thanks!

Seems to be working: searching for Яблочный пирог shows Рецепт:Яблочный_пирог in the sidebar.

might be related to T392058

Looks great!
I would find a bit more natural to have serp positioned before the target page

@Keewanlew some clarifications: intitle is searching for words in the titles:

• intitle:s does not find the page named Some title
• intitle:some can find a page named Some title
• intitle:s can find a page named The letter S

PR uploaded to add support for matched_fields: https://github.com/opensearch-project/OpenSearch/pull/18166

Batch id of the enwiki_titlewiki index in eqiad is 1746625724 (Wed May 07 2025 13:48:44) so this means the failure is possibly related to the incident or could just be a coincidence.

I've added a quick cronjob running from stat1009:/home/dcausse/wdqs_reconcile/reconcile.sh running daily at 10:00 UTC and will reconcile all items edited the previous day that have a change in a SomeValue node.
Moving to waiting to not forgot to stop that job once the reload is done.

Ran the script from Erik and found:

hewikisource 3014
fiwiktionary 1151
trwiktionary 2914
zhwiktionary 1754
mgwiktionary 1859
enwiktionary 1556
enwiki 5335305

Re-opening, we seem to have promoted a bad build recently causing T393663. Unfortunately we disabled completion index rebuilds as part of the opensearch migration and the bad index kept serving stale results for quite some time.
Reason for the bad promotion is quite unclear, sole trace I could find is https://logstash.wikimedia.org/app/discover#/doc/logstash-*/logstash-mediawiki-1-7.0.0-1-2025.05.07?id=OJUeq5YBfOjk-Vo1yy77 but this error suggests that the build failed and should not have promoted the index. It's possible the bad index was promoted on the previous run on May 6 but not finding anything about this yet.

Quick heads up that wdqs users are starting to get impacted by this.

This should now be resolved, please see T393663#10803425

This should now be resolved, please see T393663#10803425

This should now be resolved, please see T393663#10803425

Apologies about this, as part of T388610: Migrate production Elastic clusters to Opensearch (CirrusSearch backend infrastructure) we disabled some updates, some transition took longer than we expected. I routed the search traffic to codfw which should have fresh indices. The example query mentioned in the description now returns results.

Very likely due to the opensearch migration

I think it's a reasonable expectation that when you search for the default namespaces you want the default namespaces to be searched on the sister wikis as well.

just a quick heads in case you planned to re-index commons/wikidata as part of this task, please skip these 2 indices til T392058 is fixed.

In T271776#10759021, @Nikki wrote:

Why was it too ambiguous? The idea was to match the existing haslabel, hasdescription and hascaption keywords (https://www.mediawiki.org/wiki/Help:Extension:WikibaseCirrusSearch#haslabel/hascaption) - lemmas are effectively labels for lexemes, so it makes sense for the lemma keywords to be similar to the label keywords.

Does some kind of similar logging/tracking already exist in Query Service? What information does it contain?

Also happens on action=parse

@gmodena thanks for working on this!

@Nikki (or anyone else interested in filtering on lemma spelling variants) while working on this we realized that some clarifications might be needed.
The new search keyword we will add is currently named lemmaspellingvariant, it's not ideal because quite long but I found that haslemma was too ambiguous (please let us know if you have objections/suggestions).
The use of this keyword will be like other keywords and quite independent from the rest of the search query, for instance: aluminium lemmaspellingvariant:en-us will find https://www.wikidata.org/wiki/Lexeme:L18179. From the ticket description I think this is what is expected but if not please let us know. Allowing to match a particular lemma string against its specific language variant will require some thinking on our side and is not entirely trivial.

I think this task should be done as part of T372912 which will involve some refactoring of the way the tags are shipped.
I suspect that the delta you generate could easily be grouped by page_id after they're computed.

Cirrus should now gracefully handle this exception, I took a quick look at EditPage but I'm not quite clear how to fail gracefully there.

Tagging Advanced-Search because we introduced subpageof in CirrusSearch to workaround confusing behaviors of existing keywords like prefix:, see T159321 and T180495. There's possibly something to do on the UI to better guide the user using this field?
Search keywords that can escape the initial namespace filter have been quite confusing as well and we decided to not introduce new ones because there's no way to understand from the CirrusSearch perspective what's the actual intent of the user.

tested wdqs & wcqs deploys and all the expected services got restarted successfully.

already done

we now use a custom elasticsearch sink, I doubt we'll want to go back to the one provided by flink

fetch_articletopic_prediction_thresholds has been removed

I've been playing with grouping top results by cluster, this is at http://localhost:12222/clustered. Could be interesting in the context of diversity search.

Wrote a small demo available on stat1009, you need a tunnel there with ssh -L12222:localhost:12222 stat1009.eqiad.wmnet and then open http://localhost:12222/.

Tagging CirrusSearch, we should probably handle this exception in CirrusSearch to minimize the log spam, we'll probably mark these pages as "broken" in the search index with an artificial template like CirrusSearchBadRevision.

@gmodena thanks!

Additionally if we're confident that we'll always have less than 500k tags/week we might consider unblocking T372912: Migrate image recommendation to use page_weighted_tags_changed stream

@mfossati thanks! Do these two snapshots use the same dumps? if yes we might perhaps wait for a run that uses different dump and see?

I think the problem is the param statement:boost= whose intent is I believe to disable statement boosting but this is setting a float where a map of properties -> weight is expected.
Removing CirrusSearch since it relates to param overrides in MediaSearch extensions/WikibaseMediaInfo/src/Search/MediaSearchProfiles.php:

	foreach ( RequestContext::getMain()->getRequest()->getQueryValues() as $key => $value ) {
		// convert [ 'one:two' => 'three' ] into ['one']['two'] = 'three'
		$flat = array_merge( explode( ':', $key ), [ floatval( $value ) ] );
		$result = array_reduce(
			array_reverse( $flat ),
			static function ( $previous, $key ) {
				return $previous !== null ? [ $key => $previous ] : $key;
			},
			null
		);
		$settings = array_replace_recursive( $settings, $result );
	}

Where I think the target setting must be appropriately checked to make sure that a float is not put in place of an array.

@elukey sorry I missed your initial ping, reading the changelog I don't foresee any problems on the wdqs cookbooks, do you need any help for the version bump?

The items have been reconciled, the root cause is unfortunately not fixed and these issues might happen again if we don't get to an agreement on how to handle missing events or make the event platform more resilient.