http://127.0.0.1:3001/inn/12 2025-07-15T20:42:49Z <p>Golang</p> http://127.0.0.1:3001/inn/12/3048 2025-07-15T20:42:49Z surdeus http://127.0.0.1:3001/user/1 <article class="message is-info"> <div class="message-header"> <p>Info</p> </div> <div class="message-body"> This post is auto-generated from RSS feed <b>Hacker News</b>. Source: <a href="https://go.dev/blog/fips140">The FIPS 140-3 Go Cryptographic Module</a> </div> </article> <a href="https://news.ycombinator.com/item?id=44575607" rel="noopener noreferrer">Comments</a> http://127.0.0.1:3001/inn/12/3049 2025-07-07T16:56:48Z surdeus http://127.0.0.1:3001/user/1 <article class="message is-info"> <div class="message-header"> <p>Info</p> </div> <div class="message-body"> This post is auto-generated from RSS feed <b>Hacker News</b>. Source: <a href="https://go.dev/blog/generic-interfaces">Generic interfaces</a> </div> </article> <a href="https://news.ycombinator.com/item?id=44492290" rel="noopener noreferrer">Comments</a> http://127.0.0.1:3001/inn/12/3047 2025-06-03T00:00:00Z surdeus http://127.0.0.1:3001/user/1 <article class="message is-info"> <div class="message-header"> <p>Info</p> </div> <div class="message-body"> This post is auto-generated from RSS feed <b>The Go Blog</b>. Source: <a href="https://go.dev/blog/error-syntax">[ On | No ] syntactic support for error handling</a> </div> </article> <div id="blog"><div id="content"> <div id="content"> <div class=""> <h1><a href="https://go.dev/blog/" rel="noopener noreferrer">The Go Blog</a></h1> <h1>[ On | No ] syntactic support for error handling</h1> <p> Robert Griesemer<br> 3 June 2025 </p> <p>One of the oldest and most persistent complaints about Go concerns the verbosity of error handling. We are all intimately (some may say painfully) familiar with this code pattern:</p> <pre><code>x, err := call() if err != nil { // handle err } </code></pre> <p>The test <code>if err != nil</code> can be so pervasive that it drowns out the rest of the code. This typically happens in programs that do a lot of API calls, and where handling errors is rudimentary and they are simply returned. Some programs end up with code that looks like this:</p> <pre><code>func printSum(a, b string) error { x, err := strconv.Atoi(a) if err != nil { return err } y, err := strconv.Atoi(b) if err != nil { return err } fmt.Println("result:", x + y) return nil } </code></pre> <p>Of the ten lines of code in this function body, only four (the calls and the last two lines) appear to do real work. The remaining six lines come across as noise. The verbosity is real, and so it’s no wonder that complaints about error handling have topped our annual user surveys for years. (For a while, the lack of generics surpassed complaints about error handling, but now that Go supports generics, error handling is back on top.)</p> <p>The Go team takes community feedback seriously, and so for many years now we have tried to come up with a solution for this problem, together with input from the Go community.</p> <p>The first explicit attempt by the Go team dates back to 2018, when Russ Cox <a href="https://go.googlesource.com/proposal/+/master/design/go2draft-error-handling-overview.md" rel="noopener noreferrer">formally described the problem</a> as part of what we called the Go 2 effort at that time. He outlined a possible solution based on a <a href="https://go.googlesource.com/proposal/+/master/design/go2draft-error-handling.md" rel="noopener noreferrer">draft design</a> by Marcel van Lohuizen. The design was based on a <code>check</code> and <code>handle</code> mechanism and was fairly comprehensive. The draft includes a detailed analysis of alternative solutions, including comparisons with approaches taken by other languages. If you’re wondering if your particular error handling idea was previously considered, read this document!</p> <pre><code>// printSum implementation using the proposed check/handle mechanism. func printSum(a, b string) error { handle err { return err } x := check strconv.Atoi(a) y := check strconv.Atoi(b) fmt.Println("result:", x + y) return nil } </code></pre> <p>The <code>check</code> and <code>handle</code> approach was deemed too complicated and almost a year later, in 2019, we followed up with the much simplified and by now <a href="https://go.dev/issue/32437#issuecomment-2278932700" rel="noopener noreferrer">infamous</a> <a href="https://go.googlesource.com/proposal/+/master/design/32437-try-builtin.md" rel="noopener noreferrer"><code>try</code> proposal</a>. It was based on the ideas of <code>check</code> and <code>handle</code>, but the <code>check</code> pseudo-keyword became the <code>try</code> built-in function and the <code>handle</code> part was omitted. To explore the impact of the <code>try</code> built-in, we wrote a simple tool (<a href="https://github.com/griesemer/tryhard" rel="noopener noreferrer">tryhard</a>) that rewrites existing error handling code using <code>try</code>. The proposal was argued over intensively, approaching 900 comments on the <a href="https://go.dev/issue/32437" rel="noopener noreferrer">GitHub issue</a>.</p> <pre><code>// printSum implementation using the proposed try mechanism. func printSum(a, b string) error { // use a defer statement to augment errors before returning x := try(strconv.Atoi(a)) y := try(strconv.Atoi(b)) fmt.Println("result:", x + y) return nil } </code></pre> <p>However, <code>try</code> affected control flow by returning from the enclosing function in case of an error, and did so from potentially deeply nested expressions, thus hiding this control flow from view. This made the proposal unpalatable to many, and despite significant investment into this proposal we decided to abandon this effort too. In retrospect it might have been better to introduce a new keyword, something that we could do now since we have fine-grained control over the language version via <code>go.mod</code> files and file-specific directives. Restricting the use of <code>try</code> to assignments and statements might have alleviated some of the other concerns. A <a href="https://go.dev/issue/73376" rel="noopener noreferrer">recent proposal</a> by Jimmy Frasche, which essentially goes back to the original <code>check</code> and <code>handle</code> design and addresses some of that design’s shortcomings, pursues that direction.</p> <p>The repercussions of the <code>try</code> proposal led to much soul searching including a series of blog posts by Russ Cox: <a href="https://research.swtch.com/proposals-intro" rel="noopener noreferrer">“Thinking about the Go Proposal Process”</a>. One conclusion was that we likely diminished our chances for a better outcome by presenting an almost fully baked proposal with little space for community feedback and a “threatening” implementation timeline. Per <a href="https://research.swtch.com/proposals-large" rel="noopener noreferrer">“Go Proposal Process: Large Changes”</a>: “in retrospect, <code>try</code> was a large enough change that the new design we published […] should have been a second draft design, not a proposal with an implementation timeline”. But irrespective of a possible process and communication failure in this case, the user sentiment towards the proposal was very strongly not in favor.</p> <p>We didn’t have a better solution at that time and didn’t pursue syntax changes for error handling for several years. Plenty of people in the community were inspired, though, and we received a steady trickle of error handling proposals, many very similar to each other, some interesting, some incomprehensible, and some infeasible. To keep track of the expanding landscape, another year later, Ian Lance Taylor created an <a href="https://go.dev/issue/40432" rel="noopener noreferrer">umbrella issue</a> which summarizes the current state of proposed changes for improved error handling. A <a href="https://go.dev/wiki/Go2ErrorHandlingFeedback" rel="noopener noreferrer">Go Wiki</a> was created to collect related feedback, discussions, and articles. Independently, other people have started tracking all the many error handling proposals over the years. It’s amazing to see the sheer volume of them all, for instance in Sean K. H. Liao’s blog post on <a href="https://seankhliao.com/blog/12020-11-23-go-error-handling-proposals/" rel="noopener noreferrer">“go error handling proposals”</a>.</p> <p>The complaints about the verbosity of error handling persisted (see <a href="https://go.dev/blog/survey2024-h1-results" rel="noopener noreferrer">Go Developer Survey 2024 H1 Results</a>), and so, after a series of increasingly refined Go team internal proposals, Ian Lance Taylor published <a href="https://go.dev/issue/71203" rel="noopener noreferrer">“reduce error handling boilerplate using <code>?</code>”</a> in 2024. This time the idea was to borrow from a construct implemented in <a href="https://www.rust-lang.org/" rel="noopener noreferrer">Rust</a>, specifically the <a href="https://doc.rust-lang.org/std/result/index.html#the-question-mark-operator-" rel="noopener noreferrer"><code>?</code> operator</a>. The hope was that by leaning on an existing mechanism using an established notation, and taking into account what we had learned over the years, we should be able to finally make some progress. In small informal user studies where programmers were shown Go code using <code>?</code>, the vast majority of participants correctly guessed the meaning of the code, which further convinced us to give it another shot. To be able to see the impact of the change, Ian wrote a tool that converts ordinary Go code into code that uses the proposed new syntax, and we also prototyped the feature in the compiler.</p> <pre><code>// printSum implementation using the proposed "?" statements. func printSum(a, b string) error { x := strconv.Atoi(a) ? y := strconv.Atoi(b) ? fmt.Println("result:", x + y) return nil } </code></pre> <p>Unfortunately, as with the other error handling ideas, this new proposal was also quickly overrun with comments and many suggestions for minor tweaks, often based on individual preferences. Ian closed the proposal and moved the content into a <a href="https://go.dev/issue/71460" rel="noopener noreferrer">discussion</a> to facilitate the conversation and to collect further feedback. A slightly modified version was received <a href="https://github.com/golang/go/discussions/71460#discussioncomment-12060294" rel="noopener noreferrer">a bit more positively</a> but broad support remained elusive.</p> <p>After so many years of trying, with three full-fledged proposals by the Go team and literally <a href="https://go.dev/issues?q=+is%3Aissue+label%3Aerror-handling" rel="noopener noreferrer">hundreds</a> (!) of community proposals, most of them variations on a theme, all of which failed to attract sufficient (let alone overwhelming) support, the question we now face is: how to proceed? Should we proceed at all?</p> <p><em>We think not.</em></p> <p>To be more precise, we should stop trying to solve the <em>syntactic problem</em>, at least for the foreseeable future. The <a href="https://github.com/golang/proposal?tab=readme-ov-file#consensus-and-disagreement" rel="noopener noreferrer">proposal process</a> provides justification for this decision:</p> <blockquote> <p>The goal of the proposal process is to reach general consensus about the outcome in a timely manner. If proposal review cannot identify a general consensus in the discussion of the issue on the issue tracker, the usual result is that the proposal is declined.</p> </blockquote> <p>Furthermore:</p> <blockquote> <p>It can happen that proposal review may not identify a general consensus and yet it is clear that the proposal should not be outright declined. […] If the proposal review group cannot identify a consensus nor a next step for the proposal, the decision about the path forward passes to the Go architects […], who review the discussion and aim to reach a consensus among themselves.</p> </blockquote> <p>None of the error handling proposals reached anything close to a consensus, so they were all declined. Even the most senior members of the Go team at Google do not unanimously agree on the best path forward <em>at this time</em> (perhaps that will change at some point). But without a strong consensus we cannot reasonably move forward.</p> <p>There are valid arguments in favor of the status quo:</p> <ul> <li> <p>If Go had introduced specific syntactic sugar for error handling early on, few would argue over it today. But we are 15 years down the road, the opportunity has passed, and Go has a perfectly fine way to handle errors, even if it may seem verbose at times.</p> </li> <li> <p>Looking from a different angle, let’s assume we came across the perfect solution today. Incorporating it into the language would simply lead from one unhappy group of users (the one that roots for the change) to another (the one that prefers the status quo). We were in a similar situation when we decided to add generics to the language, albeit with an important difference: today nobody is forced to use generics, and good generic libraries are written such that users can mostly ignore the fact that they are generic, thanks to type inference. On the contrary, if a new syntactic construct for error handling gets added to the language, virtually everybody will need to start using it, lest their code become unidiomatic.</p> </li> <li> <p>Not adding extra syntax is in line with one of Go’s design rules: do not provide multiple ways of doing the same thing. There are exceptions to this rule in areas with high “foot traffic”: assignments come to mind. Ironically, the ability to <em>redeclare</em> a variable in <a href="https://go.dev/ref/spec#Short_variable_declarations" rel="noopener noreferrer">short variable declarations</a> (<code>:=</code>) was introduced to address a problem that arose because of error handling: without redeclarations, sequences of error checks require a differently named <code>err</code> variable for each check (or additional separate variable declarations). At that time, a better solution might have been to provide more syntactic support for error handling. Then, the redeclaration rule may not have been needed, and with it gone, so would be various associated <a href="https://go.dev/issue/377" rel="noopener noreferrer">complications</a>.</p> </li> <li> <p>Going back to actual error handling code, verbosity fades into the background if errors are actually <em>handled</em>. Good error handling often requires additional information added to an error. For instance, a recurring comment in user surveys is about the lack of stack traces associated with an error. This could be addressed with support functions that produce and return an augmented error. In this (admittedly contrived) example, the relative amount of boilerplate is much smaller:</p> <pre><code>func printSum(a, b string) error { x, err := strconv.Atoi(a) if err != nil { return fmt.Errorf("invalid integer: %q", a) } y, err := strconv.Atoi(b) if err != nil { return fmt.Errorf("invalid integer: %q", b) } fmt.Println("result:", x + y) return nil } </code></pre> </li> <li> <p>New standard library functionality can help reduce error handling boilerplate as well, very much in the vein of Rob Pike’s 2015 blog post <a href="https://go.dev/blog/errors-are-values" rel="noopener noreferrer">“Errors are values”</a>. For instance, in some cases <a href="https://go.dev/pkg/cmp#Or" rel="noopener noreferrer"><code>cmp.Or</code></a> may be used to deal with a series of errors all at once:</p> <pre><code>func printSum(a, b string) error { x, err1 := strconv.Atoi(a) y, err2 := strconv.Atoi(b) if err := cmp.Or(err1, err2); err != nil { return err } fmt.Println("result:", x+y) return nil } </code></pre> </li> <li> <p>Writing, reading, and debugging code are all quite different activities. Writing repeated error checks can be tedious, but today’s IDEs provide powerful, even LLM-assisted code completion. Writing basic error checks is straightforward for these tools. The verbosity is most obvious when reading code, but tools might help here as well; for instance an IDE with a Go language setting could provide a toggle switch to hide error handling code. Such switches already exist for other code sections such as function bodies.</p> </li> <li> <p>When debugging error handling code, being able to quickly add a <code>println</code> or have a dedicated line or source location for setting a breakpoint in a debugger is helpful. This is easy when there is already a dedicated <code>if</code> statement. But if all the error handling logic is hidden behind a <code>check</code>, <code>try</code>, or <code>?</code>, the code may have to be changed into an ordinary <code>if</code> statement first, which complicates debugging and may even introduce subtle bugs.</p> </li> <li> <p>There are also practical considerations: Coming up with a new syntax idea for error handling is cheap; hence the proliferation of a multitude of proposals from the community. Coming up with a good solution that holds up to scrutiny: not so much. It takes a concerted effort to properly design a language change and to actually implement it. The real cost still comes afterwards: all the code that needs to be changed, the documentation that needs to be updated, the tools that need to be adjusted. Taken all into account, language changes are very expensive, the Go team is relatively small, and there are a lot of other priorities to address. (These latter points may change: priorities can shift, team sizes can go up or down.)</p> </li> <li> <p>On a final note, some of us recently had the opportunity to attend <a href="https://cloud.withgoogle.com/next/25" rel="noopener noreferrer">Google Cloud Next 2025</a>, where the Go team had a booth and where we also hosted a small Go Meetup. Every single Go user we had a chance to ask was adamant that we should not change the language for better error handling. Many mentioned that the lack of specific error handling support in Go is most apparent when coming freshly from another language that has that support. As one becomes more fluent and writes more idiomatic Go code, the issue becomes much less important. This is of course not a sufficiently large set of people to be representative, but it may be a different set of people than we see on GitHub, and their feedback serves as yet another data point.</p> </li> </ul> <p>Of course, there are also valid arguments in favor of change:</p> <ul> <li> <p>Lack of better error handling support remains the top complaint in our user surveys. If the Go team really does take user feedback seriously, we ought to do something about this eventually. (Although there does not seem to be <a href="https://github.com/golang/go/discussions/71460#discussioncomment-11977299" rel="noopener noreferrer">overwhelming support</a> for a language change either.)</p> </li> <li> <p>Perhaps the singular focus on reducing the character count is misguided. A better approach might be to make default error handling highly visible with a keyword while still removing boilerplate (<code>err != nil</code>). Such an approach might make it easier for a reader (a code reviewer!) to see that an error is handled, without “looking twice”, resulting in improved code quality and safety. This would bring us back to the beginnings of <code>check</code> and <code>handle</code>.</p> </li> <li> <p>We don’t really know how much the issue is the straightforward syntactic verbosity of error checking, versus the verbosity of good error handling: constructing errors that are a useful part of an API and meaningful to developers and end-users alike. This is something we’d like to study in greater depth.</p> </li> </ul> <p>Still, no attempt to address error handling so far has gained sufficient traction. If we are honestly taking stock of where we are, we can only admit that we neither have a shared understanding of the problem, nor do we all agree that there is a problem in the first place. With this in mind, we are making the following pragmatic decision:</p> <p><em>For the foreseeable future, the Go team will stop pursuing syntactic language changes for error handling. We will also close all open and incoming proposals that concern themselves primarily with the syntax of error handling, without further investigation.</em></p> <p>The community has put tremendous effort into exploring, discussing, and debating these issues. While this may not have resulted in any changes to error handling syntax, these efforts have resulted in many other improvements to the Go language and our processes. Maybe, at some point in the future, a clearer picture will emerge on error handling. Until then, we look forward to focusing this incredible passion on new opportunities to make Go better for everyone.</p> <p>Thank you!</p> </div> <div class=""> <p> <b>Next article: </b><a href="https://go.dev/blog/generic-interfaces" rel="noopener noreferrer">Generic interfaces</a><br> <b>Previous article: </b><a href="https://go.dev/blog/tob-crypto-audit" rel="noopener noreferrer">Go Cryptography Security Audit</a><br> <b><a href="https://go.dev/blog/all" rel="noopener noreferrer">Blog Index</a></b> </p></div> </div> </div> </div> http://127.0.0.1:3001/inn/12/3046 2025-05-19T00:00:00Z surdeus http://127.0.0.1:3001/user/1 <article class="message is-info"> <div class="message-header"> <p>Info</p> </div> <div class="message-body"> This post is auto-generated from RSS feed <b>The Go Blog</b>. Source: <a href="https://go.dev/blog/tob-crypto-audit">Go Cryptography Security Audit</a> </div> </article> <div id="blog"><div id="content"> <div id="content"> <div class=""> <h1><a href="https://go.dev/blog/" rel="noopener noreferrer">The Go Blog</a></h1> <h1>Go Cryptography Security Audit</h1> <p> Roland Shoemaker and Filippo Valsorda<br> 19 May 2025 </p> <p>Go ships with a full suite of cryptography packages in the standard library to help developers build secure applications. Google recently contracted the independent security firm <a href="https://www.trailofbits.com/" rel="noopener noreferrer">Trail of Bits</a> to complete an audit of the core set of packages that are also validated as part of the <a href="https://go.dev/doc/go1.24#fips140" rel="noopener noreferrer">new native FIPS 140-3 module</a>. The audit produced a single low-severity finding, in the legacy and unsupported <a href="https://go.dev/doc/security/fips140#goboringcrypto" rel="noopener noreferrer">Go+BoringCrypto integration</a>, and a handful of informational findings. The full text of the audit report can be found <a href="https://github.com/trailofbits/publications/blob/d47e8fafa7e3323e5620d228f2f3f3bf58ed5978/reviews/2025-03-google-gocryptographiclibraries-securityreview.pdf" rel="noopener noreferrer">here</a>.</p> <p>The scope of the audit included our implementations of key exchange (ECDH and post-quantum ML-KEM), digital signature (ECDSA, RSA, and Ed25519), encryption (AES-GCM, AES-CBC, and AES-CTR), hashing (SHA-1, SHA-2, and SHA-3), key derivation (HKDF and PBKDF2), and authentication (HMAC), as well as the cryptographic random number generator. Low-level big integer and elliptic curve implementations, with their delicate assembly cores, were included. Higher level protocols like TLS and X.509 were not in scope. Three Trail of Bits engineers worked on the audit for a month.</p> <p>We are proud of the security track record of the Go cryptography packages, and of the outcome of this audit, which is just one of many ways we gain assurance of the packages’ correctness. First, we aggressively limit their complexity, guided by the <a href="https://go.dev/design/cryptography-principles" rel="noopener noreferrer">Cryptography Principles</a> which for example prioritize security over performance. Further, we <a href="https://www.youtube.com/watch?v=lkEH3V3PkS0" rel="noopener noreferrer">thoroughly test them</a> with an array of different techniques. We make a point of leveraging safe APIs even for internal packages, and naturally we can rely on the Go language properties to avoid memory management issues. Finally, we focus on readability to make maintenance easier and code review and audits more effective.</p> <h2>One low-severity finding</h2> <p>The only potentially exploitable issue, TOB-GOCL-3, has <em>low severity</em>, meaning it had minor impact and was difficult to trigger. This issue has been fixed in Go 1.24.</p> <p>Crucially, TOB-GOCL-3 (<a href="#cgo-memory-management" rel="noopener noreferrer">discussed further below</a>) concerns memory management in the <a href="https://go.dev/doc/security/fips140#goboringcrypto" rel="noopener noreferrer">legacy Go+BoringCrypto GOEXPERIMENT</a>, which is not enabled by default and unsupported for use outside of Google.</p> <h2>Five informational findings</h2> <p>The remaining findings are <em>informational</em>, meaning they do not pose an immediate risk but are relevant to security best practices. We addressed these in the current Go 1.25 development tree.</p> <p>Findings TOB-GOCL-1, TOB-GOCL-2, and TOB-GOCL-6 concern possible timing side-channels in various cryptographic operations. Of these three findings, only TOB-GOCL-2 affects operations that were expected to be constant time due to operating on secret values, but it only affects Power ISA targets (GOARCH ppc64 and ppc64le). TOB-GOCL-4 highlights misuse risk in an internal API, should it be repurposed beyond its current use case. TOB-GOCL-5 points out a missing check for a limit that is impractical to reach.</p> <h2>Timing Side-Channels</h2> <p>Findings TOB-GOCL-1, TOB-GOCL-2, and TOB-GOCL-6 concern minor timing side-channels. TOB-GOCL-1 and TOB-GOCL-6 are related to functions which we do not use for sensitive values, but could be used for such values in the future, and TOB-GOCL-2 is related to the assembly implementation of P-256 ECDSA on Power ISA.</p> <h3><code>crypto/ecdh,crypto/ecdsa</code>: conversion from bytes to field elements is not constant time (TOB-GOCL-1)</h3> <p>The internal implementation of NIST elliptic curves provided a method to convert field elements between an internal and external representation which operated in variable time.</p> <p>All usages of this method operated on public inputs which are not considered secret (public ECDH values, and ECDSA public keys), so we determined that this was not a security issue. That said, we decided to <a href="https://go.dev/cl/650579" rel="noopener noreferrer">make the method constant time anyway</a>, in order to prevent accidentally using this method in the future with secret values, and so that we don’t have to think about whether it is an issue or not.</p> <h3><code>crypto/ecdsa</code>: P-256 conditional negation is not constant time in Power ISA assembly (TOB-GOCL-2, CVE-2025-22866)</h3> <p>Beyond the <a href="https://go.dev/wiki/PortingPolicy#first-class-ports" rel="noopener noreferrer">first class Go platforms</a>, Go also supports a number of additional platforms, including some less common architectures. During the review of our assembly implementations of various underlying cryptographic primitives, the Trail of Bits team found one issue that affected the ECDSA implementation on the ppc64 and ppc64le architectures.</p> <p>Due to the usage of a conditional branching instruction in the implementation of the conditional negation of P-256 points, the function operated in variable-time, rather than constant-time, as expected. The fix for this was relatively simple, <a href="https://go.dev/cl/643735" rel="noopener noreferrer">replacing the conditional branching instruction</a> with a pattern we already use elsewhere to conditionally select the correct result in constant time. We assigned this issue CVE-2025-22866.</p> <p>To prioritize the code that reaches most of our users, and due to the specialized knowledge required to target specific ISAs, we generally rely on community contributions to maintain assembly for non-first class platforms. We thank our partners at IBM for helping provide review for our fix.</p> <h3><code>crypto/ed25519</code>: Scalar.SetCanonicalBytes is not constant time (TOB-GOCL-6)</h3> <p>The internal edwards25519 package provided a method to convert between an internal and external representation of scalars which operated in variable time.</p> <p>This method was only used on signature inputs to ed25519.Verify, which are not considered secret, so we determined that this was not a security issue. That said, similarly to the TOB-GOCL-1 finding, we decided to <a href="https://go.dev/cl/648035" rel="noopener noreferrer">make the method constant time anyway</a>, in order to prevent accidentally using this method in the future with secret values, and because we are aware that people fork this code outside of the standard library, and may be using it with secret values.</p> <h2>Cgo Memory Management</h2> <p>Finding TOB-GOCL-3 concerns a memory management issue in the Go+BoringCrypto integration.</p> <h3><code>crypto/ecdh</code>: custom finalizer may free memory at the start of a C function call using this memory (TOB-GOCL-3)</h3> <p>During the review, there were a number of questions about our cgo-based Go+BoringCrypto integration, which provides a FIPS 140-2 compliant cryptography mode for internal usage at Google. The Go+BoringCrypto code is not supported by the Go team for external use, but has been critical for Google’s internal usage of Go.</p> <p>The Trail of Bits team found one vulnerability and one <a href="https://go.dev/cl/644120" rel="noopener noreferrer">non-security relevant bug</a>, both of which were results of the manual memory management required to interact with a C library. Since the Go team does not support usage of this code outside of Google, we have chosen not to issue a CVE or Go vulnerability database entry for this issue, but we <a href="https://go.dev/cl/644119" rel="noopener noreferrer">fixed it in Go 1.24</a>.</p> <p>This kind of pitfall is one of the many reasons that we decided to move away from the Go+BoringCrypto integration. We have been working on a <a href="https://go.dev/doc/security/fips140" rel="noopener noreferrer">native FIPS 140-3 mode</a> that uses the regular pure Go cryptography packages, allowing us to avoid the complex cgo semantics in favor of the traditional Go memory model.</p> <h2>Implementation Completeness</h2> <p>Findings TOB-GOCL-4 and TOB-GOCL-5 concern limited implementations of two specifications, <a href="https://csrc.nist.gov/pubs/sp/800/90/a/r1/final" rel="noopener noreferrer">NIST SP 800-90A</a> and <a href="https://datatracker.ietf.org/doc/html/rfc8018" rel="noopener noreferrer">RFC 8018</a>.</p> <h3><code>crypto/internal/fips140/drbg</code>: CTR_DRBG API presents multiple misuse risks (TOB-GOCL-4)</h3> <p>As part of the <a href="https://go.dev/doc/security/fips140" rel="noopener noreferrer">native FIPS 140-3 mode</a> that we are introducing, we needed an implementation of the NIST CTR_DRBG (an AES-CTR based deterministic random bit generator) to provide compliant randomness.</p> <p>Since we only need a small subset of the functionality of the NIST SP 800-90A Rev. 1 CTR_DRBG for our purposes, we did not implement the full specification, in particular omitting the derivation function and personalization strings. These features can be critical to safely use the DRBG in generic contexts.</p> <p>As our implementation is tightly scoped to the specific use case we need, and since the implementation is not publicly exported, we determined that this was acceptable and worth the decreased complexity of the implementation. We do not expect this implementation to ever be used for other purposes internally, and have <a href="https://go.dev/cl/647815" rel="noopener noreferrer">added a warning to the documentation</a> that details these limitations.</p> <h3><code>crypto/pbkdf2</code>: PBKDF2 does not enforce output length limitations (TOB-GOCL-5)</h3> <p>In Go 1.24, we began the process of moving packages from <a href="https://golang.org/x/crypto" rel="noopener noreferrer">golang.org/x/crypto</a> into the standard library, ending a confusing pattern where first-party, high-quality, and stable Go cryptography packages were kept outside of the standard library for no particular reason.</p> <p>As part of this process we moved the <a href="https://golang.org/x/crypto/pbkdf2" rel="noopener noreferrer">golang.org/x/crypto/pbkdf2</a> package into the standard library, as crypto/pbkdf2. While reviewing this package, the Trail of Bits team noticed that we did not enforce the limit on the size of derived keys defined in <a href="https://datatracker.ietf.org/doc/html/rfc8018" rel="noopener noreferrer">RFC 8018</a>.</p> <p>The limit is <code>(2^32 - 1) * </code>, after which the key would loop. When using SHA-256, exceeding the limit would take a key of more than 137GB. We do not expect anyone has ever used PBKDF2 to generate a key this large, especially because PBKDF2 runs the iterations at every block, but for the sake of correctness, we <a href="https://go.dev/cl/644122" rel="noopener noreferrer">now enforce the limit as defined by the standard</a>.</p> <h1>What’s Next</h1> <p>The results of this audit validate the effort the Go team has put into developing high-quality, easy to use cryptography libraries and should provide confidence to our users who rely on them to build safe and secure software.</p> <p>We’re not resting on our laurels, though: the Go contributors are continuing to develop and improve the cryptography libraries we provide users.</p> <p>Go 1.24 now includes a FIPS 140-3 mode written in pure Go, which is currently undergoing CMVP testing. This will provide a supported FIPS 140-3 compliant mode for all users of Go, replacing the currently unsupported Go+BoringCrypto integration.</p> <p>We are also working to implement modern post-quantum cryptography, introducing a ML-KEM-768 and ML-KEM-1024 implementation in Go 1.24 in the <a href="https://go.dev/pkg/crypto/mlkem" rel="noopener noreferrer">crypto/mlkem package</a>, and adding support to the crypto/tls package for the hybrid X25519MLKEM768 key exchange.</p> <p>Finally, we are planning on introducing new easier to use high-level cryptography APIs, designed to reduce the barrier for picking and using high-quality algorithms for basic use cases. We plan to begin with offering a simple password hashing API that removes the need for users to decide which of the myriad of possible algorithms they should be relying on, with mechanisms to automatically migrate to newer algorithms as the state-of-the-art changes.</p> </div> <div class=""> <p> <b>Next article: </b><a href="https://go.dev/blog/error-syntax" rel="noopener noreferrer">[ On | No ] syntactic support for error handling</a><br> <b>Previous article: </b><a href="https://go.dev/blog/testing-b-loop" rel="noopener noreferrer">More predictable benchmarking with testing.B.Loop</a><br> <b><a href="https://go.dev/blog/all" rel="noopener noreferrer">Blog Index</a></b> </p></div> </div> </div> </div> http://127.0.0.1:3001/inn/12/3045 2025-04-02T00:00:00Z surdeus http://127.0.0.1:3001/user/1 <article class="message is-info"> <div class="message-header"> <p>Info</p> </div> <div class="message-body"> This post is auto-generated from RSS feed <b>The Go Blog</b>. Source: <a href="https://go.dev/blog/testing-b-loop">More predictable benchmarking with testing.B.Loop</a> </div> </article> <div id="blog"><div id="content"> <div id="content"> <div class=""> <h1><a href="https://go.dev/blog/" rel="noopener noreferrer">The Go Blog</a></h1> <h1>More predictable benchmarking with testing.B.Loop</h1> <p> Junyang Shao<br> 2 April 2025 </p> <p>Go developers who have written benchmarks using the <a href="https://pkg.go.dev/testing" rel="noopener noreferrer"><code>testing</code></a> package might have encountered some of its various pitfalls. Go 1.24 introduces a new way to write benchmarks that’s just as easy to use, but at the same time far more robust: <a href="https://pkg.go.dev/testing#B.Loop" rel="noopener noreferrer"><code>testing.B.Loop</code></a>.</p> <p>Traditionally, Go benchmarks are written using a loop from 0 to <code>b.N</code>:</p> <pre><code>func Benchmark(b *testing.B) { for range b.N { ... code to measure ... } } </code></pre> <p>Using <code>b.Loop</code> instead is a trivial change:</p> <pre><code>func Benchmark(b *testing.B) { for b.Loop() { ... code to measure ... } } </code></pre> <p><code>testing.B.Loop</code> has many benefits:</p> <ul> <li>It prevents unwanted compiler optimizations within the benchmark loop.</li> <li>It automatically excludes setup and cleanup code from benchmark timing.</li> <li>Code can’t accidentally depend on the total number of iterations or the current iteration.</li> </ul> <p>These were all easy mistakes to make with <code>b.N</code>-style benchmarks that would silently result in bogus benchmark results. As an added bonus, <code>b.Loop</code>-style benchmarks even complete in less time!</p> <p>Let’s explore the advantages of <code>testing.B.Loop</code> and how to effectively utilize it.</p> <h2>Old benchmark loop problems</h2> <p>Before Go 1.24, while the basic structure of a benchmark was simple, more sophisticated benchmarks required more care:</p> <pre><code>func Benchmark(b *testing.B) { ... setup ... b.ResetTimer() // if setup may be expensive for range b.N { ... code to measure ... ... use sinks or accumulation to prevent dead-code elimination ... } b.StopTimer() // if cleanup or reporting may be expensive ... cleanup ... ... report ... } </code></pre> <p>If setup or cleanup are non-trivial, the developer needs to surround the benchmark loop with <code>ResetTimer</code> and/or <code>StopTimer</code> calls. These are easy to forget, and even if the developer remembers they may be necessary, it can be difficult to judge whether setup or cleanup are “expensive enough” to require them.</p> <p>Without these, the <code>testing</code> package can only time the entire benchmark function. If a benchmark function omits them, the setup and cleanup code will be included in the overall time measurement, silently skewing the final benchmark result.</p> <p>There is another, more subtle pitfall that requires deeper understanding: (<a href="https://eli.thegreenplace.net/2023/common-pitfalls-in-go-benchmarking/" rel="noopener noreferrer">Example source</a>)</p> <pre><code>func isCond(b byte) bool { if b%3 == 1 &amp;&amp; b%7 == 2 &amp;&amp; b%17 == 11 &amp;&amp; b%31 == 9 { return true } return false } func BenchmarkIsCondWrong(b *testing.B) { for range b.N { isCond(201) } } </code></pre> <p>In this example, the user might observe <code>isCond</code> executing in sub-nanosecond time. CPUs are fast, but not that fast! This seemingly anomalous result stems from the fact that <code>isCond</code> is inlined, and since its result is never used, the compiler eliminates it as dead code. As a result, this benchmark doesn’t measure <code>isCond</code> at all; it measures how long it takes to do nothing. In this case, the sub-nanosecond result is a clear red flag, but in more complex benchmarks, partial dead-code elimination can lead to results that look reasonable but still aren’t measuring what was intended.</p> <h2>How <code>testing.B.Loop</code> helps</h2> <p>Unlike a <code>b.N</code>-style benchmark, <code>testing.B.Loop</code> is able to track when it is first called in a benchmark when the final iteration ends. The <code>b.ResetTimer</code> at the loop’s start and <code>b.StopTimer</code> at its end are integrated into <code>testing.B.Loop</code>, eliminating the need to manually manage the benchmark timer for setup and cleanup code.</p> <p>Furthermore, the Go compiler now detects loops where the condition is just a call to <code>testing.B.Loop</code> and prevents dead code elimination within the loop. In Go 1.24, this is implemented by disallowing inlining into the body of such a loop, but we plan to <a href="https://go.dev/issue/73137" rel="noopener noreferrer">improve</a> this in the future.</p> <p>Another nice feature of <code>testing.B.Loop</code> is its one-shot ramp-up approach. With a <code>b.N</code>-style benchmark, the testing package must call the benchmark function several times with different values of <code>b.N</code>, ramping up until the measured time reached a threshold. In contrast, <code>b.Loop</code> can simply run the benchmark loop until it reaches the time threshold, and only needs to call the benchmark function once. Internally, <code>b.Loop</code> still uses a ramp-up process to amortize measurement overhead, but this is hidden from the caller and can be more efficient.</p> <p>Certain constraints of the <code>b.N</code>-style loop still apply to the <code>b.Loop</code>-style loop. It remains the user’s responsibility to manage the timer within the benchmark loop, when necessary: (<a href="https://eli.thegreenplace.net/2023/common-pitfalls-in-go-benchmarking/" rel="noopener noreferrer">Example source</a>)</p> <pre><code>func BenchmarkSortInts(b *testing.B) { ints := make([]int, N) for b.Loop() { b.StopTimer() fillRandomInts(ints) b.StartTimer() slices.Sort(ints) } } </code></pre> <p>In this example, to benchmark the in-place sorting performance of <code>slices.Sort</code>, a randomly initialized array is required for each iteration. The user must still manually manage the timer in such cases.</p> <p>Furthermore, there still needs to be exactly one such loop in the benchmark function body (a <code>b.N</code>-style loop cannot coexist with a <code>b.Loop</code>-style loop), and every iteration of the loop should do the same thing.</p> <h2>When to use</h2> <p>The <code>testing.B.Loop</code> method is now the preferred way to write benchmarks:</p> <pre><code>func Benchmark(b *testing.B) { ... setup ... for b.Loop() { // optional timer control for in-loop setup/cleanup ... code to measure ... } ... cleanup ... } </code></pre> <p><code>testing.B.Loop</code> offers faster, more accurate, and more intuitive benchmarking.</p> <h2>Acknowledgements</h2> <p>A huge thank you to everyone in the community who provided feedback on the proposal issue and reported bugs as this feature was released! I’m also grateful to Eli Bendersky for his helpful blog summaries. And finally a big thank you to Austin Clements, Cherry Mui and Michael Pratt for their review, thoughtful work on the design options and documentation improvements. Thank you all for your contributions!</p> </div> <div class=""> <p> <b>Next article: </b><a href="https://go.dev/blog/tob-crypto-audit" rel="noopener noreferrer">Go Cryptography Security Audit</a><br> <b>Previous article: </b><a href="https://go.dev/blog/coretypes" rel="noopener noreferrer">Goodbye core types - Hello Go as we know and love it!</a><br> <b><a href="https://go.dev/blog/all" rel="noopener noreferrer">Blog Index</a></b> </p></div> </div> </div> </div>