One issue that a lot of teams run into is that they want to switch their configs between different agents e.g. - What happened: Hey HN,
One issue that a lot of teams run into is that they want to switch their configs between different agents e.g. - Why it matters: Hey HN,
One issue that a lot of teams run into is that they want to switch their configs between different agents e.g. - What to do: Track for corroboration and benchmark data before adopting. - Score: **Overall 5.9/10 | Signal 8.4 | Novelty 5.1 | Impact 2.6 | Confidence 7.5 | Actionability 3.5** - Evidence badges: [Repo](https://github.com/tilework-tech/nori-skillsets) - Why this made the cut: Signal 8.4, Confidence 7.5, and Impact 2.6 combined to rank this in the top set. - Deep: - Context: Hey HN,
One issue that a lot of teams run into is that they want to switch their configs between different agents e.g. - What's new: Hey HN,
One issue that a lot of teams run into is that they want to switch their configs between different agents e.g. - Key quotes/snippets: - "Hey HN,
One issue that a lot of teams run into is that they want to switch their configs between different agents e.g." - "Claude Code --> Codex or they want to switch out different configs for the same agent e.g." - Limitations / unknowns: - Generalization outside curated tasks is still unclear. - Next-step validation checks: - Reproduce one claim with a public baseline and fixed evaluation settings. - Check robustness on out-of-distribution or long-context cases. ## What Changed Overnight _Read time: ~1 min_ - New: AgentDS Technical Report: Benchmarking the Future of Human-AI Collaboration in Domain-Specific Data Science - New: VulnAgent-R2: Evidence-Calibrated Multi-Agent Auditing for Repository-Level Vulnerability Detection - New: Towards Verifiable Multimodal Deep Research: A Multi-Agent Harness for Interleaved Report Generation - New: HighTide: An Agent-Curated Open-Source VLSI Benchmark Suite - New: CyberGym-E2E: Scalable Real-World Benchmark for AI Agents' End-to-End Cybersecurity Capabilities - New: New Benchmarking Shows Limited Generalization Power of TCR Antigenic Epitope Prediction Models - Removed: EURO-5K: When Does Domain Pretraining Matter? Benchmarking Transformers for EU Reporting Obligation Extraction (fell below rank threshold) - Removed: VulnAgent-R2: Evidence-Calibrated Multi-Agent Auditing for Repository-Level Vulnerability Detection (fell below rank threshold) - Removed: The Agent's First Day: Benchmarking Learning, Exploration, and Scheduling in the Workplace Scenarios (fell below rank threshold) - Removed: AUDITFLOW: Executable Symbolic Environments for Structured Financial Reporting Verification (fell below rank threshold) - - What to do now: - Validate with one small internal benchmark and compare against your current baseline this week. - Track for corroboration and benchmark data before adopting. ## Deep Dives _Read time: ~5 min_ - ### [affaan-m/ECC: The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.](https://github.com/affaan-m/ECC) - Summary: The agent harness performance optimization system. - What happened: The agent harness performance optimization system. - Why it matters: The agent harness performance optimization system. - What to do: Validate with one small internal benchmark and compare against your current baseline this week. - Score: **Overall 8.0/10 | Signal 10.0 | Novelty 6.2 | Impact 8.2 | Confidence 7.0 | Actionability 6.5** - Evidence badges: [Repo](https://github.com/affaan-m/ECC) - Why this made the cut: Signal 10.0, Confidence 7.0, and Impact 8.2 combined to rank this in the top set. - Deep: - Context: | Topic | What You'll Learn | |---|---| | Token Optimization | Model selection, system prompt slimming, background processes | | Memory Persistence | Hooks that save/load context across sessions automatically | | Continuous Learning | Auto-extract patterns... - What's new: Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond. - Key quotes/snippets: - "The agent harness performance optimization system." - "Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond." - Limitations / unknowns: - Generalization outside curated tasks is still unclear. - Next-step validation checks: - Reproduce one claim with a public baseline and fixed evaluation settings. - Check robustness on out-of-distribution or long-context cases. - ### [AgentDS Technical Report: Benchmarking the Future of Human-AI Collaboration in Domain-Specific Data Science](https://arxiv.org/abs/2603.19005) - Summary: arXiv:2603.19005v3 Announce Type: replace-cross Abstract: Data science plays a critical role in transforming complex data into actionable insights across numerous domains. - What happened: We introduce AgentDS, a benchmark and competition designed to evaluate both AI agents and human-AI collaboration performance in domain-specific data science. - Why it matters: arXiv:2603.19005v3 Announce Type: replace-cross Abstract: Data science plays a critical role in transforming complex data into actionable insights across numerous. - What to do: Validate with one small internal benchmark and compare against your current baseline this week. - Score: **Overall 6.8/10 | Signal 9.4 | Novelty 6.2 | Impact 2.0 | Confidence 9.5 | Actionability 6.5** - Evidence badges: [Paper](https://arxiv.org/abs/2603.19005), [Benchmarks](https://agentds.org/) - Why this made the cut: Signal 9.4, Confidence 9.5, and Impact 2.0 combined to rank this in the top set. - Deep: - Context: AgentDS consists of 17 challenges across six industries: commerce, food production, healthcare, insurance, manufacturing, and retail banking. - What's new: We conducted an open competition involving 29 teams and 80 participants, enabling systematic comparison between human-AI collaborative approaches and AI-only baselines. - Key quotes/snippets: - "arXiv:2603.19005v3 Announce Type: replace-cross Abstract: Data science plays a critical role in transforming complex data into actionable insights across numerous domains." - "Recent developments in large language models (LLMs) and artificial intelligence (AI) agents have significantly automated data science workflow." - Limitations / unknowns: - However, it remains unclear to what extent AI agents can match the performance of human experts on domain-specific data science tasks, and in which aspects human expertise continues to provide advantages. - Next-step validation checks: - Reproduce one claim with a public baseline and fixed evaluation settings. - Check robustness on out-of-distribution or long-context cases. - ### [Show HN: Switch skills between agents, locally manage multiple configs](https://github.com/tilework-tech/nori-skillsets) - Summary: Hey HN,
One issue that a lot of teams run into is that they want to switch their configs between different agents e.g. - What happened: Hey HN,
One issue that a lot of teams run into is that they want to switch their configs between different agents e.g. - Why it matters: Hey HN,
One issue that a lot of teams run into is that they want to switch their configs between different agents e.g. - What to do: Track for corroboration and benchmark data before adopting. - Score: **Overall 5.9/10 | Signal 8.4 | Novelty 5.1 | Impact 2.6 | Confidence 7.5 | Actionability 3.5** - Evidence badges: [Repo](https://github.com/tilework-tech/nori-skillsets) - Why this made the cut: Signal 8.4, Confidence 7.5, and Impact 2.6 combined to rank this in the top set. - Deep: - Context: Hey HN,
One issue that a lot of teams run into is that they want to switch their configs between different agents e.g. - What's new: Hey HN,
One issue that a lot of teams run into is that they want to switch their configs between different agents e.g. - Key quotes/snippets: - "Hey HN,
One issue that a lot of teams run into is that they want to switch their configs between different agents e.g." - "Claude Code --> Codex or they want to switch out different configs for the same agent e.g." - Limitations / unknowns: - Generalization outside curated tasks is still unclear. - Next-step validation checks: - Reproduce one claim with a public baseline and fixed evaluation settings. - Check robustness on out-of-distribution or long-context cases. ## Reality Check _Read time: ~1 min_ - affaan-m/ECC: The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond. - Primary source: yes - Demo available: no - Benchmarks/evals: no - Baselines/ablations: no - Third-party corroboration: no - Reproducibility details: yes - What would change my mind: - Independent replication with comparable or better results. - Public benchmark numbers with clear baseline comparisons. - Likely failure mode: Performance may collapse outside curated demos or narrow tasks. - VulnAgent-R2: Evidence-Calibrated Multi-Agent Auditing for Repository-Level Vulnerability Detection - Primary source: yes - Demo available: no - Benchmarks/evals: yes - Baselines/ablations: no - Third-party corroboration: no - Reproducibility details: yes - What would change my mind: - Independent replication with comparable or better results. - Public benchmark numbers with clear baseline comparisons. - Likely failure mode: Performance may collapse outside curated demos or narrow tasks. - Show HN: Switch skills between agents, locally manage multiple configs - Primary source: yes - Demo available: no - Benchmarks/evals: no - Baselines/ablations: no - Third-party corroboration: no - Reproducibility details: yes - What would change my mind: - Independent replication with comparable or better results. - Public benchmark numbers with clear baseline comparisons. - Likely failure mode: Performance may collapse outside curated demos or narrow tasks. - affaan-m/ECC: The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond. - Primary source: yes - Demo available: no - Benchmarks/evals: no - Baselines/ablations: no - Third-party corroboration: no - Reproducibility details: yes - What would change my mind: - Independent replication with comparable or better results. - Public benchmark numbers with clear baseline comparisons. - Likely failure mode: Performance may collapse outside curated demos or narrow tasks. ## Lab Notes _Read time: ~1 min_ - Tool/Repo of the day: MemPalace/mempalace: The best-benchmarked open-source AI memory system. And it's free. (https://github.com/MemPalace/mempalace) - Prompt/Workflow of the day: summarize claim -> evidence -> risk in three passes before acting. - Tiny snippet: `uv run python -m msd.run --scheduled` ## Research Radar _Read time: ~6 min_ - ### [AgentDS Technical Report: Benchmarking the Future of Human-AI Collaboration in Domain-Specific Data Science](https://arxiv.org/abs/2603.19005) - Summary: arXiv:2603.19005v3 Announce Type: replace-cross Abstract: Data science plays a critical role in transforming complex data into actionable insights across numerous domains. - What happened: We introduce AgentDS, a benchmark and competition designed to evaluate both AI agents and human-AI collaboration performance in domain-specific data science. - Why it matters: arXiv:2603.19005v3 Announce Type: replace-cross Abstract: Data science plays a critical role in transforming complex data into actionable insights across numerous. - What to do: Validate with one small internal benchmark and compare against your current baseline this week. - Score: **Overall 6.8/10 | Signal 9.4 | Novelty 6.2 | Impact 2.0 | Confidence 9.5 | Actionability 6.5** - Evidence badges: [Paper](https://arxiv.org/abs/2603.19005), [Benchmarks](https://agentds.org/) - Why this made the cut: Signal 9.4, Confidence 9.5, and Impact 2.0 combined to rank this in the top set. - Deep: - Context: AgentDS consists of 17 challenges across six industries: commerce, food production, healthcare, insurance, manufacturing, and retail banking. - What's new: We conducted an open competition involving 29 teams and 80 participants, enabling systematic comparison between human-AI collaborative approaches and AI-only baselines. - Key quotes/snippets: - "arXiv:2603.19005v3 Announce Type: replace-cross Abstract: Data science plays a critical role in transforming complex data into actionable insights across numerous domains." - "Recent developments in large language models (LLMs) and artificial intelligence (AI) agents have significantly automated data science workflow." - Limitations / unknowns: - However, it remains unclear to what extent AI agents can match the performance of human experts on domain-specific data science tasks, and in which aspects human expertise continues to provide advantages. - Next-step validation checks: - Reproduce one claim with a public baseline and fixed evaluation settings. - Check robustness on out-of-distribution or long-context cases. - ### [VulnAgent-R2: Evidence-Calibrated Multi-Agent Auditing for Repository-Level Vulnerability Detection](https://arxiv.org/abs/2603.13384) - Summary: arXiv:2603.13384v3 Announce Type: replace-cross Abstract: Software vulnerabilities often depend on cross-file data flow, build options, framework conventions, and runtime guards. - What happened: arXiv:2603.13384v3 Announce Type: replace-cross Abstract: Software vulnerabilities often depend on cross-file data flow, build options, framework conventions, and. - Why it matters: Treating vulnerability detection as calibrated evidence accumulation improves detection, localization, auditability, and cost control under the evaluated protocol, while. - What to do: Validate with one small internal benchmark and compare against your current baseline this week. - Score: **Overall 6.4/10 | Signal 9.4 | Novelty 5.1 | Impact 2.0 | Confidence 8.7 | Actionability 6.5** - Evidence badges: Repo, [Paper](https://arxiv.org/abs/2603.13384), [Benchmarks](https://github.com/renweimeng/Vlun-Agent-X.) - Why this made the cut: Signal 9.4, Confidence 8.7, and Impact 2.0 combined to rank this in the top set. - Deep: - Context: The system combines graph triage, bounded context optimization, role-specialized agents, sceptic counter-evidence, selective dynamic verification, and calibrated fusion. - What's new: arXiv:2603.13384v3 Announce Type: replace-cross Abstract: Software vulnerabilities often depend on cross-file data flow, build options, framework conventions, and runtime guards, so isolated function classifiers produce fragile and poorly calibrated warnings. - Key quotes/snippets: - "arXiv:2603.13384v3 Announce Type: replace-cross Abstract: Software vulnerabilities often depend on cross-file data flow, build options, framework conventions, and runtime guards, so." - "Repository-level LLM agents can gather richer evidence, but prior variants under-specify reproducibility, verifier behavior, baseline fairness, and statistical uncertainty." - Limitations / unknowns: - We present VulnAgent-R2, a budget-aware agentic auditing framework with three additional reusable modules: counterfactual evidence reweighting, build-aware verification-plan synthesis, and a cost-risk Pareto scheduler. - Next-step validation checks: - Reproduce one claim with a public baseline and fixed evaluation settings. - Check robustness on out-of-distribution or long-context cases. - ### [Towards Verifiable Multimodal Deep Research: A Multi-Agent Harness for Interleaved Report Generation](https://arxiv.org/abs/2605.29861) - Summary: arXiv:2605.29861v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have advanced autonomous agents from deep search, which retrieves concise factual answers. - What happened: We further introduce PtahEval, an evaluation protocol that augments existing benchmarks with image-level and presentation-level assessments. - Why it matters: arXiv:2605.29861v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have advanced autonomous agents from deep search, which retrieves concise factual. - What to do: Validate with one small internal benchmark and compare against your current baseline this week. - Score: **Overall 6.4/10 | Signal 9.4 | Novelty 5.1 | Impact 2.0 | Confidence 8.7 | Actionability 6.5** - Evidence badges: Repo, [Paper](https://arxiv.org/abs/2605.29861), [Benchmarks](https://github.com/SnowNation101/Ptah) - Why this made the cut: Signal 9.4, Confidence 8.7, and Impact 2.0 combined to rank this in the top set. - Deep: - Context: arXiv:2605.29861v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have advanced autonomous agents from deep search, which retrieves concise factual answers, to deep research, which synthesizes scattered evidence into long-form reports. - What's new: We propose Ptah, a multi-agent harness for interleaved report generation. - Key quotes/snippets: - "arXiv:2605.29861v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have advanced autonomous agents from deep search, which retrieves concise factual answers, to deep." - "However, verifiable multimodal deep research remains challenging due to open-ended synthesis without deterministic ground truth and the need to interleave textual arguments with visual." - Limitations / unknowns: - However, verifiable multimodal deep research remains challenging due to open-ended synthesis without deterministic ground truth and the need to interleave textual arguments with visual evidence. - Next-step validation checks: - Reproduce one claim with a public baseline and fixed evaluation settings. - Check robustness on out-of-distribution or long-context cases. ## Forecast & Watchlist _Read time: ~1 min_ - Watch: agent - Watch: llm - Watch: cs.ai - Watch: cs.lg - Watch: rss - Watch: cs.cl - Watch: python - Watch: benchmark ## Save for Later _Read time: ~7 min_ - ### [paperclipai/paperclip: The open-source app everyone uses to manage agents at work](https://github.com/paperclipai/paperclip) - Summary: The open-source app everyone uses to manage agents at work Quickstart · Docs · GitHub · Discord · Twitter · Website full-tour.webm Open-source orchestration for teams of AI agents. - What happened: The open-source app everyone uses to manage agents at work Quickstart · Docs · GitHub · Discord · Twitter · Website full-tour.webm Open-source orchestration for teams of. - Why it matters: The open-source app everyone uses to manage agents at work Quickstart · Docs · GitHub · Discord · Twitter · Website full-tour.webm Open-source orchestration for teams of. - What to do: Validate with one small internal benchmark and compare against your current baseline this week. - Score: **Overall 7.9/10 | Signal 10.0 | Novelty 6.2 | Impact 7.7 | Confidence 7.0 | Actionability 6.5** - Evidence badges: [Repo](https://github.com/paperclipai/paperclip), Paper - Why this made the cut: Signal 10.0, Confidence 7.0, and Impact 7.7 combined to rank this in the top set. - Deep: - Context: The open-source app everyone uses to manage agents at work Quickstart · Docs · GitHub · Discord · Twitter · Website full-tour.webm Open-source orchestration for teams of AI agents. - What's new: The open-source app everyone uses to manage agents at work Quickstart · Docs · GitHub · Discord · Twitter · Website full-tour.webm Open-source orchestration for teams of AI agents. - Key quotes/snippets: - "The open-source app everyone uses to manage agents at work Quickstart · Docs · GitHub · Discord · Twitter · Website full-tour.webm Open-source orchestration for teams of AI agents." - "If OpenClaw is an employee, Paperclip is the company." - Limitations / unknowns: - When they hit the limit, they stop. - Next-step validation checks: - Reproduce one claim with a public baseline and fixed evaluation settings. - Check robustness on out-of-distribution or long-context cases. - ### [VoltAgent/awesome-design-md: A collection of DESIGN.md files analysis by popular brand design systems. Drop one into your project and let coding agents generate a matching UI.](https://github.com/VoltAgent/awesome-design-md) - Summary: A collection of DESIGN.md files analysis by popular brand design systems. - What happened: DESIGN.md is a new concept introduced by Google Stitch. - Why it matters: A collection of DESIGN.md files analysis by popular brand design systems. - What to do: Validate with one small internal benchmark and compare against your current baseline this week. - Score: **Overall 7.7/10 | Signal 10.0 | Novelty 5.1 | Impact 7.8 | Confidence 7.0 | Actionability 6.5** - Evidence badges: [Repo](https://github.com/VoltAgent/awesome-design-md) - Why this made the cut: Signal 10.0, Confidence 7.0, and Impact 7.8 combined to rank this in the top set. - Deep: - Context: A collection of DESIGN.md files analysis by popular brand design systems. - What's new: DESIGN.md is a new concept introduced by Google Stitch. - Key quotes/snippets: - "A collection of DESIGN.md files analysis by popular brand design systems." - "Drop one into your project and let coding agents generate a matching UI." - Limitations / unknowns: - Generalization outside curated tasks is still unclear. - Next-step validation checks: - Reproduce one claim with a public baseline and fixed evaluation settings. - Check robustness on out-of-distribution or long-context cases. - ### [Beyond Prompt-Based Planning: MCP-Native Graph Planning-based Biomedical Agent System](https://arxiv.org/abs/2606.04494) - Summary: arXiv:2606.04494v1 Announce Type: new Abstract: Biomedical agents promise to automate complex biological workflows, yet current systems face two fundamental bottlenecks. - What happened: We introduce BioManus, an MCP-native biomedical agent built on graph-scaffolded planning over structured biological capabilities. - Why it matters: Experiments on BioAgentBench and LAB-Bench show that BioManus improves execution accuracy, workflow validity, and context efficiency over advanced biomedical agent. - What to do: Track for corroboration and benchmark data before adopting. - Score: **Overall 6.1/10 | Signal 9.4 | Novelty 5.1 | Impact 2.0 | Confidence 7.5 | Actionability 5.2** - Evidence badges: [Paper](https://arxiv.org/abs/2606.04494), Benchmarks - Why this made the cut: Signal 9.4, Confidence 7.5, and Impact 2.0 combined to rank this in the top set. - Deep: - Context: As biomedical software ecosystems grow, this coupling between tool coverage and context size leads to tool confusion, unstable planning, and inefficient execution. - What's new: arXiv:2606.04494v1 Announce Type: new Abstract: Biomedical agents promise to automate complex biological workflows, yet current systems face two fundamental bottlenecks: bioinformatics tools are highly heterogeneous in interfaces and execution environments,... - Key quotes/snippets: - "arXiv:2606.04494v1 Announce Type: new Abstract: Biomedical agents promise to automate complex biological workflows, yet current systems face two fundamental bottlenecks: bioinformatics." - "As biomedical software ecosystems grow, this coupling between tool coverage and context size leads to tool confusion, unstable planning, and inefficient execution." - Limitations / unknowns: - Generalization outside curated tasks is still unclear. - Next-step validation checks: - Reproduce one claim with a public baseline and fixed evaluation settings. - Check robustness on out-of-distribution or long-context cases. - ### [Show HN: A GitOps-style registry for AI agent Workflows, Skills and MCP servers](https://github.com/Friz-zy/ai-capability-registry) - Summary: Show HN: A GitOps-style registry for AI agent Workflows, Skills and MCP servers - What happened: Show HN: A GitOps-style registry for AI agent Workflows, Skills and MCP servers - Why it matters: Could materially affect near-term AI workflows. - What to do: Track for corroboration and benchmark data before adopting. - Score: **Overall 5.9/10 | Signal 8.4 | Novelty 5.1 | Impact 2.6 | Confidence 7.5 | Actionability 3.5** - Evidence badges: [Repo](https://github.com/Friz-zy/ai-capability-registry) - Why this made the cut: Signal 8.4, Confidence 7.5, and Impact 2.6 combined to rank this in the top set. - Deep: - Context: Show HN: A GitOps-style registry for AI agent Workflows, Skills and MCP servers - What's new: Show HN: A GitOps-style registry for AI agent Workflows, Skills and MCP servers - Key quotes/snippets: - "Show HN: A GitOps-style registry for AI agent Workflows, Skills and MCP servers" - Limitations / unknowns: - Generalization outside curated tasks is still unclear. - Next-step validation checks: - Reproduce one claim with a public baseline and fixed evaluation settings. - Check robustness on out-of-distribution or long-context cases. - ### [Attow Nexus – Git for AI agent state](https://github.com/sharb1235-hash/attow-nexus) - Summary: Attow Nexus – Git for AI agent state - What happened: Attow Nexus – Git for AI agent state - Why it matters: Could materially affect near-term AI workflows. - What to do: Track for corroboration and benchmark data before adopting. - Score: **Overall 5.8/10 | Signal 8.4 | Novelty 5.1 | Impact 2.4 | Confidence 7.5 | Actionability 3.5** - Evidence badges: [Repo](https://github.com/sharb1235-hash/attow-nexus) - Why this made the cut: Signal 8.4, Confidence 7.5, and Impact 2.4 combined to rank this in the top set. - Deep: - Context: Attow Nexus – Git for AI agent state - What's new: Attow Nexus – Git for AI agent state - Key quotes/snippets: - "Attow Nexus – Git for AI agent state" - Limitations / unknowns: - Generalization outside curated tasks is still unclear. - Next-step validation checks: - Reproduce one claim with a public baseline and fixed evaluation settings. - Check robustness on out-of-distribution or long-context cases. - ### [Knox – Govern AI agent tool calls before they execute](https://github.com/qoris-ai/knox) - Summary: Knox – Govern AI agent tool calls before they execute - What happened: Knox – Govern AI agent tool calls before they execute - Why it matters: Could materially affect near-term AI workflows. - What to do: Track for corroboration and benchmark data before adopting. - Score: **Overall 5.8/10 | Signal 8.4 | Novelty 5.1 | Impact 2.4 | Confidence 7.5 | Actionability 3.5** - Evidence badges: [Repo](https://github.com/qoris-ai/knox) - Why this made the cut: Signal 8.4, Confidence 7.5, and Impact 2.4 combined to rank this in the top set. - Deep: - Context: Knox – Govern AI agent tool calls before they execute - What's new: Knox – Govern AI agent tool calls before they execute - Key quotes/snippets: - "Knox – Govern AI agent tool calls before they execute" - Limitations / unknowns: - Generalization outside curated tasks is still unclear. - Next-step validation checks: - Reproduce one claim with a public baseline and fixed evaluation settings. - Check robustness on out-of-distribution or long-context cases.