AMT coverage: g3-graphql-query-processing - #193
Conversation
… and solidity abi meta Adds discriminating tests for the mutation-probe survivors of the g3-graphql-query-processing behaviour list: process_deployer_query guards/len/hex-decode error mapping, AuthoringMeta v1 item-level encode/decode/validate + json fallback + unpack, AuthoringMetaV2 word trimming/utf8 and the fetch_for_contract error-wrap pipeline over httpmock, and SolidityAbiMeta per-kind serialization, exact field order, indexed guards, required-field errors and defaults. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Warning Review limit reached
Next review available in: 5 minutes Limit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?Wait for the limit to reset, then comment An organization admin can change what happens after included review limits in Billing. How do review limits work?CodeRabbit enforces per-developer PR review limits within each organization. For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The destructured metaboard_url binding in test_get_metabytes_by_hash_success shadowed the server url it was meant to be checked against, so the assertion compared a String with its own to_string. Renaming the binding makes the assertion read the error field against the url the fetch was actually given. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Pushed 6abd15f on top: renames the destructured Independent corroboration: a parallel run of the same g3 campaign re-derived this PR's matrix from scratch with its own 84-mutant probe against bba50a7 — pass 1 (pre-existing suite): 15 killed / 68 survived / 0 no-run; pass 2 with equivalent new tests: 83/84 killed with the single survivor being the same equivalent mutant documented here (vacuous SolidityAbiMeta validation, #153). The parallel run's duplicate issues (#207 #208 #209 #212) are closed in favour of #153 #164 #156 #160; its three non-duplicate findings stayed open: #210 (no hash verification of subgraph meta bytes), #211 (panic-freedom of metas[0]/decoded[0] rests on cross-crate invariants), #213 (corrupt subgraph payloads collapse into NoRecordFound). |
A second independently derived mutant set probed against this branch surfaced six killable survivors: description validation removal in AuthoringMetaItem, missing-inputs defaults for the constructor, event and error deserialize arms, nested event tuple component recursion, and the cbor_decode error wrap in fetch_for_contract. One new discriminating test per gap. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Pushed ef51fb8: the corroborating 84-mutant cross-probe described above found six killable survivors on this branch — description-validation removal, constructor/event/error missing-inputs defaults, nested event tuple component recursion, and the cbor_decode error wrap in fetch_for_contract — each now has one discriminating test, and the --only re-probe kills all six (6/6, exit 0; suite 178 green). Details in the Cross-probe addendum appended to the PR body. |
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 participant
AMT (adversarial mutation testing) coverage for the g3-graphql-query-processing group: 30 behaviours across
crates/cli/src/meta/query/mod.rs,crates/cli/src/meta/types/solidity_abi/v2.rs,crates/cli/src/meta/types/authoring/v1.rsandcrates/cli/src/meta/types/authoring/v2.rs.Process: baseline green on unmutated code; the full 81-mutant probe ran against the pre-existing suite FIRST (16 killed, 65 survived); survivors got new discriminating tests (41 tests added, no existing test edited or deleted); the full probe then re-ran against the finished suite. Two mutants remain unkilled because they are provably equivalent —
SolidityAbiMetavalidation is vacuous (zero#[validate]rules), so removing the validate gate or its early return changes nothing observable; filed as #153 rather than papered over.Adversarial findings filed (all labelled
audit+adversarial): #153, #154, #156, #157, #160, #164, #166, #168.Behaviour matrix
Verdict legend:
pre= killed by a pre-existing test in the first probe pass;new= survived the first pass, killed by a test added here;equivalent= survived both passes, no discriminating test exists (see #153).Noneguard -> NoRecordFoundVec::new()test_process_deployer_query_null_bytecode_is_no_record_foundNoneguard -> NoRecordFoundVec::new()test_process_deployer_query_null_parser_is_no_record_foundNoneguard -> NoRecordFoundVec::new()test_process_deployer_query_null_store_is_no_record_foundNoneguard -> NoRecordFoundVec::new()test_process_deployer_query_null_interpreter_is_no_record_foundNoneguard -> NoRecordFoundVec::new()test_process_deployer_query_null_deploy_transaction_is_no_record_found== 1->!is_empty()test_process_deployer_query_two_metas_is_no_record_found== 1-><= 1test_process_deployer_query_zero_metas_is_no_record_found.or(Err(NoRecordFound))?->.unwrap_or_default()test_process_deployer_query_invalid_bytecode_hex_is_no_record_foundtest_process_deployer_query_invalid_parser_hex_is_no_record_foundtest_process_deployer_query_invalid_store_hex_is_no_record_foundtest_process_deployer_query_invalid_interpreter_hex_is_no_record_foundtest_process_deployer_query_invalid_meta_id_hex_is_no_record_foundtest_process_deployer_query_invalid_tx_id_hex_is_no_record_foundtest_process_deployer_query_invalid_constructor_meta_hash_hex_is_no_record_foundtest_process_deployer_query_invalid_constructor_meta_hex_is_no_record_found["abi"]->["ABI"]test_from_artifact_extracts_abi_keyreturn Err(e)droppedOk(t.validate().map(|_| t)?)->Ok(t)Ok(empty)test_try_from_bytes_rejects_invalid_jsonOk(empty)test_try_from_bytes_rejects_invalid_jsonunpack()?-> raw payloadtest_try_from_item_unpacks_content_encodingunpack()?-> raw payloadtest_try_from_item_unpacks_content_encodingtest_alltest_alltest_alltest_serialize_all_item_kinds_roundtriptest_serialize_all_item_kinds_roundtriptest_serialize_all_item_kinds_roundtriptest_alltest_serialize_all_item_kinds_roundtriptest_alltest_serialize_fn_exact_field_order_and_component_skippingtest_alltest_alltest_allif falsetest_deserialize_rejects_indexed_on_fn_iotest_serialize_all_item_kinds_roundtripok_or(...)->unwrap_or(false)test_deserialize_requires_indexed_on_event_inputif falsetest_deserialize_rejects_indexed_on_event_componenttest_serialize_all_item_kinds_roundtripif falsetest_deserialize_rejects_indexed_on_error_inputtest_serialize_all_item_kinds_roundtripok_or->unwrap_or_defaulttest_deserialize_missing_required_fields_error_messagesok_or->unwrap_or(NonPayable)test_deserialize_missing_required_fields_error_messagesok_or->unwrap_or(NonPayable)test_deserialize_missing_required_fields_error_messagesok_or->unwrap_or(NonPayable)test_deserialize_missing_required_fields_error_messagesok_or->unwrap_or(NonPayable)test_deserialize_missing_required_fields_error_messagesok_or->unwrap_or_defaulttest_deserialize_missing_required_fields_error_messagesok_or->unwrap_or(false)test_deserialize_missing_required_fields_error_messagesok_or->unwrap_or_defaulttest_deserialize_missing_required_fields_error_messagesNone => vec![]-> errortest_deserialize_missing_inputs_outputs_default_to_emptyNone => vec![]-> errortest_deserialize_missing_inputs_outputs_default_to_emptytest_item_encode_decode_roundtrip_offset_and_word_bytestest_item_abi_encode_validate_rejects_invalid_wordtest_item_encode_decode_roundtrip_offset_and_word_bytestest_item_abi_decode_validate_rejects_invalid_wordtest_item_abi_encode_validate_rejects_invalid_wordtest_encode_decode_validate,meta::tests::authoring_meta_roundtriptest_encode_decode_validate,meta::tests::authoring_meta_roundtriptest_encode_decode_validatetest_array_validate_rejects_and_annotates_offending_indextest_array_validate_rejects_and_annotates_offending_indextest_try_from_bytes_json_fallbacktest_try_from_bytes_json_fallbackunpack()?-> raw payloadtest_try_from_meta_item_unpacks_content_encodingposition->rpositiontest_abi_decode_valid,test_try_from_validtest_abi_decode_full_32_byte_word_kept_wholefrom_utf8-> lossytest_abi_decode_invalid_utf8_word_is_utf8_errortest_abi_decode_valid,test_try_from_validtest_fetch_for_contract_empty_rpcs_is_no_rpcs_errortest_fetch_for_contract_invalid_rpc_url_is_url_parse_errortest_get_metabytes_by_hash_successtest_fetch_for_contract_rpc_error_on_describe_calltest_fetch_for_contract_abi_decode_error_on_describe_calltest_fetch_for_contract_invalid_metaboard_url_is_url_parse_errortest_fetch_for_contract_empty_metaboard_response_is_subgraph_errortest_fetch_for_contract_success_decodes_first_metatest_fetch_for_contract_success_decodes_first_metatest_try_from_invalid_magicunpack()?-> raw payloadtest_try_from_deflate_encoded_item_unpackstest_try_from_validQA
meta::query::tests::test_process_deployer_query_*(16),meta::types::authoring::v1::tests::{test_item_encode_decode_roundtrip_offset_and_word_bytes, test_item_abi_encode_validate_rejects_invalid_word, test_item_abi_decode_validate_rejects_invalid_word, test_array_validate_rejects_and_annotates_offending_index, test_try_from_bytes_json_fallback, test_try_from_meta_item_unpacks_content_encoding},meta::types::authoring::v2::tests::{test_abi_decode_full_32_byte_word_kept_whole, test_abi_decode_invalid_utf8_word_is_utf8_error, test_fetch_for_contract_*(7), test_try_from_deflate_encoded_item_unpacks},meta::types::solidity_abi::v2::tests::{test_from_artifact_extracts_abi_key, test_try_from_bytes_rejects_invalid_json, test_try_from_item_unpacks_content_encoding, test_serialize_all_item_kinds_roundtrip, test_serialize_fn_exact_field_order_and_component_skipping, test_deserialize_*(6)}— each fails on base under its target mutation (verified by the mutation-probe pass-2 run below: every mutant that survived pass 1 is killed in pass 2 by exactly these tests, per the probe's failing-test attribution).nix run github:rainlanguage/adversarial-mutation-test#mutation-probe(suite:nix develop -c cargo test -p rain-metadata; mutants file kept outside the repo). Pass 1 against the pre-existing suite at bba50a7: 16 KILLED / 65 SURVIVED / 0 NO-RUN / 0 HARNESS-ERROR. Pass 2 after this PR's tests (98235d0): 79 KILLED / 2 SURVIVED / 0 NO-RUN / 0 HARNESS-ERROR. The full line -> mutation -> killing-test mapping is the behaviour matrix above; the 2 pass-2 survivors (S02 drop Validate early-return, S03a drop TryFrom validate gate) are semantically equivalent mutants —SolidityAbiMetaitem types deriveValidatewith zero rules sovalidate()is alwaysOk— filed as SolidityAbiMeta validation is vacuous: Validate derives carry no rules, so the TryFrom validate gate and per-index annotation can never reject #153, no discriminating test can exist.stateMutability, type tags) for serializer assertions; the ABI encoding spec for the byte-offset assertions intest_item_encode_decode_roundtrip_offset_and_word_bytes; the documented error vocabulary (NoRecordFound,NoRpcs,UrlParseError,RpcError,AbiDecodeError,MetaboardSubgraphError,MetaMagicNumberMismatch,Utf8Error) for error-path variants; hand-written JSON-RPC/graphql fixtures served by httpmock, with the erc165 probe sequence taken from the ERC-165 spec via rain-erc's documented check1/check2 flow.🤖 Generated with Claude Code
Cross-probe addendum (6abd15f, ef51fb8)
A second, independently derived 84-mutant set over the same 30 behaviours (different mutation choices, same probe harness and suite command) was run against this branch as corroboration:
test_description_rejects_unprintable_chars. No prior test fed an invalid description: the A106 regex-swap died only because a valid description fails the swapped-in symbol regex, which leaves outright removal of the validator attribute undetected.inputsdefault to[]->test_deserialize_missing_inputs_default_for_constructor_event_error. The prior defaults test covered the function arm only.test_event_component_nested_components_roundtrip. The prior roundtrip nested event components one level deep, so dropping the inner recursion survived.cbor_decodefailure wrapped asMetaErrorinfetch_for_contract->test_fetch_for_contract_invalid_cbor_is_meta_error.SolidityAbiMetavalidation, SolidityAbiMeta validation is vacuous: Validate derives carry no rules, so the TryFrom validate gate and per-index annotation can never reject #153).--onlyre-probe of those six after ef51fb8: all KILLED.metaboard_urlbinding in the pre-existingtest_get_metabytes_by_hash_successso its assertion compares the wrapped error field against the input url rather than itself.