Python3.9 test passed but 3.2 test failed, due to

E           FileNotFoundError: [Errno 2] No such file or directory: '/home/runner/.apache_beam/cache/jars/mysql-connector-java-8.0.28.jar.tmp' -> '/home/runner/.apache_beam/cache/jars/mysql-connector-java-8.0.28.jar'

Likely a race condition that two rename run at the same time when tests run in parallel:

os.rename(cached_jar + '.tmp', cached_jar)

R: @liferoad

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment assign set of reviewers

R: @kennknowles also

This appears to break

test_avro_schema_to_beam_schema_with_nullable_atomic_fields (apache_beam.io.avroio_test.TestFastAvro)

E File "apache_beam/coders/coder_impl.py", line 983, in apache_beam.coders.coder_impl.VarInt32CoderImpl.encode_to_stream
E OverflowError: value too large to convert to int32_t [while running 'Map()-ptransform-56']

This appears to break

test_avro_schema_to_beam_schema_with_nullable_atomic_fields (apache_beam.io.avroio_test.TestFastAvro)

E File "apache_beam/coders/coder_impl.py", line 983, in apache_beam.coders.coder_impl.VarInt32CoderImpl.encode_to_stream E OverflowError: value too large to convert to int32_t [while running 'Map()-ptransform-56']

Also breaks these XVR jobs: #31418, #30601, #30602, #30593, #30517

Yeah they all fails on same test. Let me investigate now.

From first glance it fails on encoding, which means the row passing in already have large integers

hmmm test passed locally:

apache_beam/io/avroio_test.py::TestFastAvro::test_avro_schema_to_beam_schema_with_nullable_atomic_fields PASSED [100%]

================================================================================ 1 passed in 13.05s ================================================================================

It turns out that the test already had an integar overflow in Row passed in:

print out the pcollection pass going to pass to SqlTransform:

Row(name='Henry', favorite_number=3, favorite_color='green')
Row(name='Toby', favorite_number=7, favorite_color='brown')
Row(name='Gordon', favorite_number=4, favorite_color='blue')
Row(name='Emily', favorite_number=4294967295, favorite_color='Red')
Row(name='Percy', favorite_number=6, favorite_color='Green')
Row(name='Bruce', favorite_number=None, favorite_color=None)

this happens to both before and after this change (#34354)

more suprisingly, when read back, it converts back to -1:

{'name': 'Thomas', 'favorite_number': 1, 'favorite_color': 'blue'}
{'name': 'Henry', 'favorite_number': 3, 'favorite_color': 'green'}
{'name': 'Toby', 'favorite_number': 7, 'favorite_color': 'brown'}
{'name': 'Gordon', 'favorite_number': 4, 'favorite_color': 'blue'}
{'name': 'Emily', 'favorite_number': -1, 'favorite_color': 'Red'}
{'name': 'Percy', 'favorite_number': 6, 'favorite_color': 'Green'}
{'name': 'Bruce', 'favorite_number': None, 'favorite_color': None}