Morning Singularity Digest - 2026-07-14

Context

arXiv:2512.17776v5 Announce Type: replace Abstract: Recent advances in large language models have enabled deep research systems that generate expert-level reports through multi-step reasoning and evidence-based synthesis.

What's new

To address these issues, we propose DEER, a benchmark for evaluating expert-level deep research reports.

Key details

• However, evaluating such reports remains challenging: report quality is multifaceted, making it difficult to determine what to assess and which criteria to use; LLM-based judges may miss errors that require domain expertise to identify; and because deep res...
• To address these issues, we propose DEER, a benchmark for evaluating expert-level deep research reports.
• DEER systematizes evaluation criteria with an expert-developed taxonomy (7 dimensions, 25 subdimensions) operationalized as 101 fine-grained rubric items.
• We also provide task-specific Expert Evaluation Guidance to support LLM-based judging.

Results & evidence

• arXiv:2512.17776v5 Announce Type: replace Abstract: Recent advances in large language models have enabled deep research systems that generate expert-level reports through multi-step reasoning and evidence-based synthesis.
• DEER systematizes evaluation criteria with an expert-developed taxonomy (7 dimensions, 25 subdimensions) operationalized as 101 fine-grained rubric items.
• Computer Science > Computation and Language [Submitted on 19 Dec 2025 (v1), last revised 13 Jul 2026 (this version, v5)] Title:DEER: A Benchmark for Evaluating Deep Research Agents on Expert Report Generation View PDF HTML (experimental)Abstract:Recent adva...

Limitations / unknowns

• However, evaluating such reports remains challenging: report quality is multifaceted, making it difficult to determine what to assess and which criteria to use; LLM-based judges may miss errors that require domain expertise to identify; and because deep res...
• Beyond performance comparisons, DEER makes system strengths and limitations interpretable and provides diagnostic signals for improvement.

Next-step validation checks

• Reproduce one claim with a public baseline and fixed evaluation settings.
• Check robustness on out-of-distribution or long-context cases.
• Track whether independent teams report matching results.

Context

This report presents the CVPR 2026@AdvML Workshop Challenge on adversarial multimodal attacks against autonomous-driving VLAs.

What's new

arXiv:2607.11560v1 Announce Type: cross Abstract: Vision-language agents (VLAs) are increasingly used to interpret complex driving scenes and support safety-critical reasoning.

Key details

• This report presents the CVPR 2026@AdvML Workshop Challenge on adversarial multimodal attacks against autonomous-driving VLAs.
• Built on DriveLM-style multi-view visual question answering, the challenge represents each scene with six synchronized camera images and a structured collection of driving-related question-answer pairs.
• Participants generate adversarial images and suffix-only textual perturbations that induce model responses to deviate from reference answers while preserving image fidelity and limiting textual cost.
• The competition comprises two phases, with Phase II adding a hidden black-box model to assess transferability.

Results & evidence

• arXiv:2607.11560v1 Announce Type: cross Abstract: Vision-language agents (VLAs) are increasingly used to interpret complex driving scenes and support safety-critical reasoning.
• This report presents the CVPR 2026@AdvML Workshop Challenge on adversarial multimodal attacks against autonomous-driving VLAs.
• Computer Science > Computer Vision and Pattern Recognition [Submitted on 13 Jul 2026] Title:Technical Report on the CVPR 2026@AdvML Workshop Challenge View PDF HTML (experimental)Abstract:Vision-language agents (VLAs) are increasingly used to interpret comp...

Limitations / unknowns

• Participants generate adversarial images and suffix-only textual perturbations that induce model responses to deviate from reference answers while preserving image fidelity and limiting textual cost.

Next-step validation checks

• Reproduce one claim with a public baseline and fixed evaluation settings.
• Check robustness on out-of-distribution or long-context cases.
• Track whether independent teams report matching results.

Context

Show HN: Turnitin Report – AI checker and AI detector for student papers

What's new

Show HN: Turnitin Report – AI checker and AI detector for student papers

Key details

• Show HN: Turnitin Report – AI checker and AI detector for student papers

Results & evidence

• No hard numbers surfaced in the source text; treat claims as directional until benchmarks appear.

Limitations / unknowns

• Generalization outside curated tasks is still unclear.

Next-step validation checks

• Reproduce one claim with a public baseline and fixed evaluation settings.
• Check robustness on out-of-distribution or long-context cases.
• Track whether independent teams report matching results.

Context

"Vibe-Trading: Your Personal Trading Agent"

What's new

"Vibe-Trading: Your Personal Trading Agent"

Key details

• "Vibe-Trading: Your Personal Trading Agent"

Results & evidence

• No hard numbers surfaced in the source text; treat claims as directional until benchmarks appear.

Limitations / unknowns

• Generalization outside curated tasks is still unclear.

Next-step validation checks

• Reproduce one claim with a public baseline and fixed evaluation settings.
• Check robustness on out-of-distribution or long-context cases.
• Track whether independent teams report matching results.

Context

The Destructive Command Guard (dcg) is for blocking dangerous git and shell commands from being executed by agents.

What's new

The Destructive Command Guard (dcg) is for blocking dangerous git and shell commands from being executed by agents.

Key details

• The Destructive Command Guard (dcg) is for blocking dangerous git and shell commands from being executed by agents.

Results & evidence

• No hard numbers surfaced in the source text; treat claims as directional until benchmarks appear.

Limitations / unknowns

• Generalization outside curated tasks is still unclear.

Next-step validation checks

• Reproduce one claim with a public baseline and fixed evaluation settings.
• Check robustness on out-of-distribution or long-context cases.
• Track whether independent teams report matching results.

Context

arXiv:2512.17776v5 Announce Type: replace Abstract: Recent advances in large language models have enabled deep research systems that generate expert-level reports through multi-step reasoning and evidence-based synthesis.

What's new

To address these issues, we propose DEER, a benchmark for evaluating expert-level deep research reports.

Key details

• However, evaluating such reports remains challenging: report quality is multifaceted, making it difficult to determine what to assess and which criteria to use; LLM-based judges may miss errors that require domain expertise to identify; and because deep res...
• To address these issues, we propose DEER, a benchmark for evaluating expert-level deep research reports.
• DEER systematizes evaluation criteria with an expert-developed taxonomy (7 dimensions, 25 subdimensions) operationalized as 101 fine-grained rubric items.
• We also provide task-specific Expert Evaluation Guidance to support LLM-based judging.

Results & evidence

• arXiv:2512.17776v5 Announce Type: replace Abstract: Recent advances in large language models have enabled deep research systems that generate expert-level reports through multi-step reasoning and evidence-based synthesis.
• DEER systematizes evaluation criteria with an expert-developed taxonomy (7 dimensions, 25 subdimensions) operationalized as 101 fine-grained rubric items.
• Computer Science > Computation and Language [Submitted on 19 Dec 2025 (v1), last revised 13 Jul 2026 (this version, v5)] Title:DEER: A Benchmark for Evaluating Deep Research Agents on Expert Report Generation View PDF HTML (experimental)Abstract:Recent adva...

Limitations / unknowns

• However, evaluating such reports remains challenging: report quality is multifaceted, making it difficult to determine what to assess and which criteria to use; LLM-based judges may miss errors that require domain expertise to identify; and because deep res...
• Beyond performance comparisons, DEER makes system strengths and limitations interpretable and provides diagnostic signals for improvement.

Next-step validation checks

• Reproduce one claim with a public baseline and fixed evaluation settings.
• Check robustness on out-of-distribution or long-context cases.
• Track whether independent teams report matching results.

Context

This report presents the CVPR 2026@AdvML Workshop Challenge on adversarial multimodal attacks against autonomous-driving VLAs.

What's new

arXiv:2607.11560v1 Announce Type: cross Abstract: Vision-language agents (VLAs) are increasingly used to interpret complex driving scenes and support safety-critical reasoning.

Key details

• This report presents the CVPR 2026@AdvML Workshop Challenge on adversarial multimodal attacks against autonomous-driving VLAs.
• Built on DriveLM-style multi-view visual question answering, the challenge represents each scene with six synchronized camera images and a structured collection of driving-related question-answer pairs.
• Participants generate adversarial images and suffix-only textual perturbations that induce model responses to deviate from reference answers while preserving image fidelity and limiting textual cost.
• The competition comprises two phases, with Phase II adding a hidden black-box model to assess transferability.

Results & evidence

• arXiv:2607.11560v1 Announce Type: cross Abstract: Vision-language agents (VLAs) are increasingly used to interpret complex driving scenes and support safety-critical reasoning.
• This report presents the CVPR 2026@AdvML Workshop Challenge on adversarial multimodal attacks against autonomous-driving VLAs.
• Computer Science > Computer Vision and Pattern Recognition [Submitted on 13 Jul 2026] Title:Technical Report on the CVPR 2026@AdvML Workshop Challenge View PDF HTML (experimental)Abstract:Vision-language agents (VLAs) are increasingly used to interpret comp...

Limitations / unknowns

• Participants generate adversarial images and suffix-only textual perturbations that induce model responses to deviate from reference answers while preserving image fidelity and limiting textual cost.

Next-step validation checks

• Reproduce one claim with a public baseline and fixed evaluation settings.
• Check robustness on out-of-distribution or long-context cases.
• Track whether independent teams report matching results.

Context

arXiv:2509.24148v3 Announce Type: replace-cross Abstract: Test-Driven Development (TDD) is a widely adopted practice that requires developers to create and execute tests alongside implementation.

What's new

We propose TENET, an agentic framework for repository-level code generation under the TDD paradigm.

Key details

• With recent advances in Large Language Models (LLMs), developers can shift from manually writing the code to defining tests as executable specifications and delegating code synthesis to AI agents.
• However, enabling repository-level TDD under developer-written tests is challenging, requiring: (1) specification enhancement: identifying a concise yet representative test subset from large suites with rich task semantics; (2) retrieval augmentation: using...
• We propose TENET, an agentic framework for repository-level code generation under the TDD paradigm.
• TENET includes: (1) a test harness mechanism that selects a concise test suite to maximize diversity of the target usage scenarios; (2) a tailored agent toolset for efficient retrieval and debugging; and (3) a reflection-based refinement workflow that itera...

Results & evidence

• arXiv:2509.24148v3 Announce Type: replace-cross Abstract: Test-Driven Development (TDD) is a widely adopted practice that requires developers to create and execute tests alongside implementation.
• However, enabling repository-level TDD under developer-written tests is challenging, requiring: (1) specification enhancement: identifying a concise yet representative test subset from large suites with rich task semantics; (2) retrieval augmentation: using...
• TENET includes: (1) a test harness mechanism that selects a concise test suite to maximize diversity of the target usage scenarios; (2) a tailored agent toolset for efficient retrieval and debugging; and (3) a reflection-based refinement workflow that itera...

Limitations / unknowns

• However, enabling repository-level TDD under developer-written tests is challenging, requiring: (1) specification enhancement: identifying a concise yet representative test subset from large suites with rich task semantics; (2) retrieval augmentation: using...

Next-step validation checks

• Reproduce one claim with a public baseline and fixed evaluation settings.
• Check robustness on out-of-distribution or long-context cases.
• Track whether independent teams report matching results.

Context

arXiv:2512.17776v5 Announce Type: replace Abstract: Recent advances in large language models have enabled deep research systems that generate expert-level reports through multi-step reasoning and evidence-based synthesis.

What's new

To address these issues, we propose DEER, a benchmark for evaluating expert-level deep research reports.

Key details

• However, evaluating such reports remains challenging: report quality is multifaceted, making it difficult to determine what to assess and which criteria to use; LLM-based judges may miss errors that require domain expertise to identify; and because deep res...
• To address these issues, we propose DEER, a benchmark for evaluating expert-level deep research reports.
• DEER systematizes evaluation criteria with an expert-developed taxonomy (7 dimensions, 25 subdimensions) operationalized as 101 fine-grained rubric items.
• We also provide task-specific Expert Evaluation Guidance to support LLM-based judging.

Results & evidence

• arXiv:2512.17776v5 Announce Type: replace Abstract: Recent advances in large language models have enabled deep research systems that generate expert-level reports through multi-step reasoning and evidence-based synthesis.
• DEER systematizes evaluation criteria with an expert-developed taxonomy (7 dimensions, 25 subdimensions) operationalized as 101 fine-grained rubric items.
• Computer Science > Computation and Language [Submitted on 19 Dec 2025 (v1), last revised 13 Jul 2026 (this version, v5)] Title:DEER: A Benchmark for Evaluating Deep Research Agents on Expert Report Generation View PDF HTML (experimental)Abstract:Recent adva...

Limitations / unknowns

• However, evaluating such reports remains challenging: report quality is multifaceted, making it difficult to determine what to assess and which criteria to use; LLM-based judges may miss errors that require domain expertise to identify; and because deep res...
• Beyond performance comparisons, DEER makes system strengths and limitations interpretable and provides diagnostic signals for improvement.

Next-step validation checks

• Reproduce one claim with a public baseline and fixed evaluation settings.
• Check robustness on out-of-distribution or long-context cases.
• Track whether independent teams report matching results.

Context

This report presents the CVPR 2026@AdvML Workshop Challenge on adversarial multimodal attacks against autonomous-driving VLAs.

What's new

arXiv:2607.11560v1 Announce Type: cross Abstract: Vision-language agents (VLAs) are increasingly used to interpret complex driving scenes and support safety-critical reasoning.

Key details

• This report presents the CVPR 2026@AdvML Workshop Challenge on adversarial multimodal attacks against autonomous-driving VLAs.
• Built on DriveLM-style multi-view visual question answering, the challenge represents each scene with six synchronized camera images and a structured collection of driving-related question-answer pairs.
• Participants generate adversarial images and suffix-only textual perturbations that induce model responses to deviate from reference answers while preserving image fidelity and limiting textual cost.
• The competition comprises two phases, with Phase II adding a hidden black-box model to assess transferability.

Results & evidence

• arXiv:2607.11560v1 Announce Type: cross Abstract: Vision-language agents (VLAs) are increasingly used to interpret complex driving scenes and support safety-critical reasoning.
• This report presents the CVPR 2026@AdvML Workshop Challenge on adversarial multimodal attacks against autonomous-driving VLAs.
• Computer Science > Computer Vision and Pattern Recognition [Submitted on 13 Jul 2026] Title:Technical Report on the CVPR 2026@AdvML Workshop Challenge View PDF HTML (experimental)Abstract:Vision-language agents (VLAs) are increasingly used to interpret comp...

Limitations / unknowns

• Participants generate adversarial images and suffix-only textual perturbations that induce model responses to deviate from reference answers while preserving image fidelity and limiting textual cost.

Next-step validation checks

• Reproduce one claim with a public baseline and fixed evaluation settings.
• Check robustness on out-of-distribution or long-context cases.
• Track whether independent teams report matching results.

Context

arXiv:2509.24148v3 Announce Type: replace-cross Abstract: Test-Driven Development (TDD) is a widely adopted practice that requires developers to create and execute tests alongside implementation.

What's new

We propose TENET, an agentic framework for repository-level code generation under the TDD paradigm.

Key details

• With recent advances in Large Language Models (LLMs), developers can shift from manually writing the code to defining tests as executable specifications and delegating code synthesis to AI agents.
• However, enabling repository-level TDD under developer-written tests is challenging, requiring: (1) specification enhancement: identifying a concise yet representative test subset from large suites with rich task semantics; (2) retrieval augmentation: using...
• We propose TENET, an agentic framework for repository-level code generation under the TDD paradigm.
• TENET includes: (1) a test harness mechanism that selects a concise test suite to maximize diversity of the target usage scenarios; (2) a tailored agent toolset for efficient retrieval and debugging; and (3) a reflection-based refinement workflow that itera...

Results & evidence

• arXiv:2509.24148v3 Announce Type: replace-cross Abstract: Test-Driven Development (TDD) is a widely adopted practice that requires developers to create and execute tests alongside implementation.
• However, enabling repository-level TDD under developer-written tests is challenging, requiring: (1) specification enhancement: identifying a concise yet representative test subset from large suites with rich task semantics; (2) retrieval augmentation: using...
• TENET includes: (1) a test harness mechanism that selects a concise test suite to maximize diversity of the target usage scenarios; (2) a tailored agent toolset for efficient retrieval and debugging; and (3) a reflection-based refinement workflow that itera...

Limitations / unknowns

• However, enabling repository-level TDD under developer-written tests is challenging, requiring: (1) specification enhancement: identifying a concise yet representative test subset from large suites with rich task semantics; (2) retrieval augmentation: using...

Next-step validation checks

• Reproduce one claim with a public baseline and fixed evaluation settings.
• Check robustness on out-of-distribution or long-context cases.
• Track whether independent teams report matching results.

Context

arXiv:2607.09885v1 Announce Type: new Abstract: We present Index-1.9B, a series of open small language models developed at Bilibili.

What's new

arXiv:2607.09885v1 Announce Type: new Abstract: We present Index-1.9B, a series of open small language models developed at Bilibili.

Key details

• The series comprises four models: Index-1.9B-Base, a foundation model with 1.9 billion non-embedding parameters pre-trained on 2.8 trillion predominantly Chinese and English tokens; Index-1.9B-Pure, a control variant trained with an identical recipe but wit...
• Pre-training employs a Warmup-Stable-Decay learning-rate schedule in which the concentration of curated data is raised substantially during the decay phase, together with a Norm-Head output layer that stabilizes training under large learning rates.
• On a suite of standard benchmarks covering examination, reasoning, mathematics, and code, Index-1.9B-Base attains an average score of 64.92, competitive with or exceeding open models of several times its size.
• We further report controlled studies on model depth, learning-rate magnitude and scheduling, the interaction between learning-rate decay and data quality, and the effect of including instruction data during pre-training, and we document an unexplained surge...

Results & evidence

• arXiv:2607.09885v1 Announce Type: new Abstract: We present Index-1.9B, a series of open small language models developed at Bilibili.
• The series comprises four models: Index-1.9B-Base, a foundation model with 1.9 billion non-embedding parameters pre-trained on 2.8 trillion predominantly Chinese and English tokens; Index-1.9B-Pure, a control variant trained with an identical recipe but wit...
• On a suite of standard benchmarks covering examination, reasoning, mathematics, and code, Index-1.9B-Base attains an average score of 64.92, competitive with or exceeding open models of several times its size.

Limitations / unknowns

• Generalization outside curated tasks is still unclear.

Next-step validation checks

• Reproduce one claim with a public baseline and fixed evaluation settings.
• Check robustness on out-of-distribution or long-context cases.
• Track whether independent teams report matching results.

Context

100+ AI Agent & RAG apps you can actually run — clone, customize, ship.

What's new

100+ AI Agent & RAG apps you can actually run — clone, customize, ship.

Key details

• 100+ AI Agent & RAG apps you can actually run — clone, customize, ship.

Results & evidence

• 100+ AI Agent & RAG apps you can actually run — clone, customize, ship.

Limitations / unknowns

• Generalization outside curated tasks is still unclear.

Next-step validation checks

• Reproduce one claim with a public baseline and fixed evaluation settings.
• Check robustness on out-of-distribution or long-context cases.
• Track whether independent teams report matching results.

Context

Anti-AI-slop design skill for Claude Code, Cursor, and Codex.

What's new

Anti-AI-slop design skill for Claude Code, Cursor, and Codex.

Key details

• Anti-AI-slop design skill for Claude Code, Cursor, and Codex.

Results & evidence

• No hard numbers surfaced in the source text; treat claims as directional until benchmarks appear.

Limitations / unknowns

• Generalization outside curated tasks is still unclear.

Next-step validation checks

• Reproduce one claim with a public baseline and fixed evaluation settings.
• Check robustness on out-of-distribution or long-context cases.
• Track whether independent teams report matching results.

Context

we had hundreds of discussions with engineering leaders over the past few months, and everyone's trying to understand where they are in the AI journey.

we collected all this data into a benchmark and built a free grader to let you know where you stan...

What's new

we had hundreds of discussions with engineering leaders over the past few months, and everyone's trying to understand where they are in the AI journey.

we collected all this data into a benchmark and built a free grader to let you know where you stan...

Key details

• we had hundreds of discussions with engineering leaders over the past few months, and everyone's trying to understand where they are in the AI journey.

  we.

Results & evidence

• No hard numbers surfaced in the source text; treat claims as directional until benchmarks appear.

Limitations / unknowns

• Generalization outside curated tasks is still unclear.

Next-step validation checks

• Reproduce one claim with a public baseline and fixed evaluation settings.
• Check robustness on out-of-distribution or long-context cases.
• Track whether independent teams report matching results.

Context

Show HN: I RL-trained an agent that trains models with RL (for –$1.3k)

What's new

Show HN: I RL-trained an agent that trains models with RL (for –$1.3k)

Key details

• Show HN: I RL-trained an agent that trains models with RL (for –$1.3k)

Results & evidence

• No hard numbers surfaced in the source text; treat claims as directional until benchmarks appear.

Limitations / unknowns

• Generalization outside curated tasks is still unclear.

Next-step validation checks

• Reproduce one claim with a public baseline and fixed evaluation settings.
• Check robustness on out-of-distribution or long-context cases.
• Track whether independent teams report matching results.

Context

Mensfeld/code-on-incus: Give each AI agent its own isolated machine

What's new

Mensfeld/code-on-incus: Give each AI agent its own isolated machine

Key details

• Mensfeld/code-on-incus: Give each AI agent its own isolated machine

Results & evidence

• No hard numbers surfaced in the source text; treat claims as directional until benchmarks appear.

Limitations / unknowns

• Generalization outside curated tasks is still unclear.

Next-step validation checks

• Reproduce one claim with a public baseline and fixed evaluation settings.
• Check robustness on out-of-distribution or long-context cases.
• Track whether independent teams report matching results.