Efficient Proof Composition for Verifiable Computation

Efficient Proof Composition for Verifiable Computation Julien Keuffer1,2 , Refik Molva2 , and Hervé Chabanne1,3 1 Idemia, Issy-les-Moulineaux, France 2 Eurecom, Biot, France 3 Telecom ParisTech, Paris, France {julien.keuffer,herve.chabanne}@idemia.com [email protected] Abstract. Outsourcing machine learning algorithms helps users to deal with large amounts of data without the need to develop the expertise required by these algorithms. Outsourcing however raises severe security issues due to potentially untrusted service providers. Verifiable computing (VC) tackles some of these issues by assuring computational integrity for an outsourced computation. In this paper, we design a VC protocol tailored to verify a sequence of operations for which no existing VC scheme is suitable to achieve realistic performance objective for the entire sequence. We thus suggest a technique to compose several specialized and efficient VC schemes with a general purpose VC protocol, like Parno et al.’s Pinocchio, by integrating the verification of the proofs generated by these specialized schemes as a function that is part of the sequence of operations verified using the general purpose scheme. The resulting scheme achieves the objectives of the general purpose scheme with increased efficiency for the prover. The scheme relies on the underlying cryptographic assumptions of the composed protocols for correctness and soundness. Keywords: Verifiable computation, Proof composition, Neural networks 1 Introduction While achieving excellent results in diverse areas, machine learning algorithms require expertise and a large training material to be fine-tuned. Therefore, cloud providers such as Amazon or Microsoft have started offering Machine Learning as a Service (MLaaS) to perform complex machine learning tasks on behalf of users. Despite these advantages, outsourcing raises a new requirement: in the face of potentially malicious service providers the users need additional guarantees to gain confidence in the results of outsourced computations. As an answer to this problem, verifiable computing (VC) provides proofs of computational integrity without any assumptions on hardware or on potential failures. Existing VC systems can theoretically prove and verify all NP computations [8]. Nevertheless, despite the variety of existing solutions, existing VC schemes have to make trade-offs between expressiveness and functionality [20] and therefore cannot efficiently handle the verifiability of a sequence of operations with a high 2 Julien Keuffer, Refik Molva, Hervé Chabanne variance in nature and complexity, like the ones involved in machine learning techniques. Even if expressive VC schemes such as Pinocchio [16] can ensure the verifiability of a machine learning algorithm, the cryptographic work required to produce the proof prevents from dealing with large but simple computations such as matrix multiplications. On the other hand, some schemes like Cormode et al.’s CMT [6] are very efficient and can deal with large computations, e.g. large matrix multiplications, but cannot handle the variety of even very simple operations such as number comparisons. Hence there is a need for a VC scheme that achieves both efficiency by handling complex operations and expressiveness through the variety of types of operations it can support. In this paper, we propose a scheme that combines a general purpose VC scheme like Pinocchio [16] or Groth’s scheme [13] and various specialized VC schemes that achieve efficient verification of complex operations like large matrix multiplications. Thanks to our proof composition scheme, the resulting VC scheme: 1. efficiently addresses the verifiability of a sequence of operations, 2. inherits the properties of the outer scheme, notably a short and single proof for a complex computation and privacy for inputs supplied by the prover. In order to highlight the relevance of our proposal, we sketch the application of the resulting scheme on a neural network, which is a popular machine learning technique achieving state of the art performance in various classification tasks such as handwritten digit recognition, object or face recognition. Furthermore we propose a concrete instance of our scheme, using a Pinocchio-like scheme [13] and the Sum-Check protocol [15]. Thanks to our composition techniques, we are able to achieve unprecedented performance gains in the verifiability of computations involving large matrix multiplication and non-linear operations. 1.1 Problem Statement Most applications involve several sequences of function evaluations combined through control structures. Assuring the verifiability of these applications has to face the challenge that the functions evaluated as part of these applications may feature computational characteristics that are too variant to be efficiently addressed by a unique VC scheme. For instance, in the case of an application that involves a combination of computationally intensive linear operations with simple non-linear ones, none of the existing VC techniques would be suitable since there is no single VC approach that can efficiently handle both. This question is perfectly illustrated by the sample scenario described in the previous section, namely dealing with the verifiability of Neural Network Algorithms, which can be viewed as a repeated sequence of a matrix product and a non-linear activation function. For instance, a two layer neural network, denoted by g, on an input x can be written as: g(x) = W2 · f (W1 · x) (1) Here W1 and W2 are matrices and f is a non-linear function like the frequently chosen Rectified Linear Unit (ReLU) function: x 7→ max(0, x). For efficiency, Efficient Proof Composition for Verifiable Computation 3 the inputs are often batched and the linear operations involved in the Neural Network are matrix products instead of products between a vector and a matrix. Denoting X a batch of inputs to classify, the batched version of (1) therefore is: g(X) = W2 · f (W1 · X) (2) In an attempt to assure the verifiability of this neural network, two alternative VC schemes seem potentially suited: the CMT protocol [6] based on interactive proofs and schemes deriving from Pinocchio [16]. CMT can efficiently deal with the matrix products but problems arise when it comes to the non-linear part of the operations since, using CMT, each function to be verified has to be represented as a layered arithmetic circuit (i.e. as an acyclic graph of computation over a finite field with an addition or a multiplication at each node, and where the circuit can be decomposed into layers, each gate of one layer being only connected to an adjacent layer). Nevertheless the second component of the neural network algorithm, that is, the ReLU activation function, does not lend itself to a simple representation as a layered circuit. [11] and [6] have proposed solutions to deal with non-layered circuits at the cost of very complex pre-processing, resulting in a substantial increase in the prover’s work and the overall circuit size. Conversely, Pinocchio-like schemes eliminate the latter problem by allowing for efficient verification of a non-linear ReLU activation function while suffering from excessive complexity in the generation of proofs for the products of large matrices (benchmarks on matrix multiplication proofs can be found in [20]). This sample scenario points to the basic limitation of existing VC schemes in efficiently addressing the requirements of common scenarios involving several components with divergent characteristics such as the mix of linear and nonlinear operations as part of the same application. The objective of our work therefore is to come up with a new VC scheme that can efficiently handle these divergent characteristics in the sub-components as part of a single VC protocol. 1.2 Idea of the Solution: Embedded Proofs Our solution is based on a method that enables the composition of a general purpose VC scheme suited to handle sequences of functions with one or several specialized VC schemes that can achieve efficiency in case of a component function with excessive requirements like very large linear operations. We apply this generic method to a pair of VC schemes, assuming that one is a general purpose VC scheme, called GVC, like Pinocchio [16], which can efficiently assure the verifiability of an application consisting of a sequence of functions, whereas the other VC scheme, which we call EVC, assures the verifiability of a single function in a very efficient way, like, for instance, a VC scheme that can handle large matrix products efficiently. The main idea underlying the VC composition method is that the verifiability of the complex operation (for which the GVC is not efficient) is outsourced to the EVC whereas the remaining non-complex functions are all handled by the GVC. In order to get the verifiability of the entire application by the GVC, instead of including the complex operation as part of the sequence of 4 Julien Keuffer, Refik Molva, Hervé Chabanne P1 (EVC1 ) t1 = f (x) + proof π1 t1 , π 1 t2 = g(t1 ) P2 (EVC2 ) y = h(t2 ) + proof π2 t2 t 3, π 3 P (GVC) Proof that:   Verify(π1 , t1 , x) = 1 t2 = g(t1 )   Verify(π2 , y, t2 ) = 1 Fig. 1: High level view of the embedded proofs functions handled by the GVC, this operation is separately handled by the EVC that generates a standalone verifiability proof for that operation and the verification of that proof is viewed as an additional function embedded in the sequence of functions handled by the GVC. Even though the verifiability of the complex operation by the GVC is not feasible due to its complexity, the verifiability of the proof on this operation is feasible by the basic principle of VC, that is, because the proof is much less complex than the operation itself. We illustrate the VC composition method using as a running example the Neural Network defined with formula (2) in Section 1.1. Here, the application consists of the sequential execution of three functions f , g and h (see Figure 1), where f and h are not suitable to be efficiently proved using GVC while g is. Note that we consider that g cannot be proved correct by any EVC systems or at least not as efficiently as with the GVC system. The ultimate goal therefore is to verify y = h(g(f (x))). In our example, f : X 7→ W1 · X, h : X 7→ W2 · X and g : X 7→ max(0, X), where X, W1 and W2 are matrices and g applies the max function element-wise to the input matrix X. In order to cope with the increased complexity of f and h, we have recourse to EVC1 and EVC2 that are specialized schemes yielding efficient proofs with such functions. πEVC1 denotes the proof generated by EVC1 on f , πEVC2 denotes the proof generated by EVC2 on h and ΠGVC denotes the proof generated by GVC. For the sequential execution of functions f , g and h, denoting t1 = f (x) and t2 = g(t1 ), the final proof then is:  
?
?
? ΠGVC VerifEVC1 (πEVC1 , x, t1 ) = 1 ∧ g(t1 ) = t2 ∧ VerifEVC2 (πEVC2 , t2 , y) = 1 . (3) Here the GVC system verifies the computation of g and the verification algorithms of the EVC1 and EVC2 systems, which output 1 if the proof is accepted and 0 otherwise. We note that this method can easily be extended to applications involving more than three functions, Section 3 describes the embedded proof protocol for an arbitrary number of functions. Interestingly, various specialized VC techniques can be selected as EVC based on their suitability to the special functions requirements provided that: Efficient Proof Composition for Verifiable Computation 5 1. The verification algorithm of each EVC proof is compatible with the GVC scheme. 2. The verification algorithm of each EVC proof should have much lower complexity than the outsourced computations (by the basic VC advantage). 3. The EVC schemes should not be VC’s with a designated verifier but instead publicly verifiable [8]. Indeed, since the prover of the whole computation is the verifier of the EVC, no secret value should be shared between the prover of the EVC and the prover of the GVC. Otherwise, a malicious prover can easily forge a proof for EVC and break the security of the scheme. In the sequel of this paper we present a concrete instance of our VC composition method using any Pinocchio-like scheme as the GVC and an efficient interactive proof protocol, namely the Sum-Check protocol [15] as the EVC. We further develop this instance with a Neural Network verification example. 1.3 Related Work Verifying computation made by an untrusted party has been studied for a long time, but protocols leading to practical implementations are recent, see [20] and the references therein for details. Most of these proof systems build on quadratic arithmetic programs [8] and we focus on zero-knowledge succinct noninteractive arguments of knowledge (zk-SNARKs) schemes [3]. Proof composition for SNARKs have been proposed by Bitansky et al. [5] and the implementation of SNARKs recursive composition has later been proposed by Ben-Sasson et al. in [4]. The high level idea of the latter proof system is to prove or verify the satisfiability of an arithmetic circuit that checks the validity of the previous proofs. Thus, the verifier should be implemented as an arithmetic circuit and used as a sub-circuit of the next prover. However, SNARKs verifiers perform the verification checks using an elliptic curve pairing and it is mathematically impossible for the base field to have the same size as the elliptic curve group order. Ben-Sasson et al. therefore propose a cycle of elliptic curves to enable proof composition. When two such elliptic curves form a cycle, the finite field defined by the prime divisor in the group order of the first curve is equal to the base field (or field of definition) of the second curve and vice versa. Although proofs can theoretically be composed as many times as desired, this method has severe overhead. Our method has a more limited spectrum than Ben-Sasson et al.’s but our resulting system is still general purpose and enjoys the property of the GVC system, such as succinctness or efficiency for the prover. Furthermore, our proposal improves the prover time, replacing a part of a computation by sub-circuit verifying the sub-computation that can then be executed outside the prover. In SafetyNets [9], Ghodsi et al. build an interactive proof protocol to verify the execution of a deep neural network on an untrusted cloud. This approach, albeit efficient, has several disadvantages over ours. The first is that expressivity of the interactive proof protocol used in SafetyNets prevents using state of the art activation functions such as ReLU. Indeed, Ghodsi et al. replace ReLU 6 Julien Keuffer, Refik Molva, Hervé Chabanne functions by a quadratic activation function, namely x 7→ x2 , which squares the input values element-wise. This solution unfortunately causes instability during the training phase of the network compared to ReLU functions. A second disadvantage is the impossibility for the prover to prove a non-deterministic computation, i.e. to prove the correctness of a computation while hiding some inputs. As a consequence, the verifier and the prover of SafetyNets have to share the model of the neural network, namely the values of the matrices (e.g. W1 and W2 in formula (1)). This situation is quite unusual in machine learning: since the training of neural networks is expensive and requires a large amount of data, powerful hardware and technical skills to obtain a classifier with good accuracy, it is unlikely that cloud providers share their models with users. In contrast, with our proposed method the prover could keep the model private and nonetheless be able to produce a proof of correct execution. 1.4 Paper organization The rest of the paper is organized as follows: we first introduce the building blocks required to instantiate our method in Section 2. Following our embedded proof protocol, we first describe a VC scheme involving composition in Section 3 and then present a specialized instance of the GVC and EVC schemes to fit the Neural Network use-case in section 4. We report experimental results on the implementation of the latter scheme in Section 5 and conclude in Section 6. A security proof of our scheme is given in Appendix A and prover’s input privacy are considered in Appendix B. 2 2.1 Building blocks GVC: Verifiable Computation based on QAPs Quadratic Arithmetic Programs In [8], Gennaro et al. defined Quadratic Arithmetic Programs (QAP) as an efficient object for circuit satisfiability. The computation to verify has first to be represented as an arithmetic circuit, from which a QAP is computed. Using the representation based on QAPs, the correctness of the computation can be tested by a divisibility check between polynomials. A cryptographic protocol enables to check the divisibility in only one point of the polynomial and to prevent a cheating prover to build a proof of a false statement that will be accepted. Definition 1 (from [16]). A QAP Q over field F contains three sets of m + 1 polynomials V = {(vk (x))}, W = {(wk (x))}, Y = {(yk (x))} for k ∈ {0, . . . , m} and a target polynomial t(x). Let F be a function that takes as input n elements of F and outputs n′ elements and let us define N as the sum of n and n′ . A N-tuple (c1 , . . . , cN ) ∈ FN is a valid assignment for function F if and only if Efficient Proof Composition for Verifiable Computation 7 there exists coefficients (cN +1 , . . . , cm ) such that t(x) divides p(x), as follows: ! ! ! m m m X X X p(x) = v0 (x)+ ck · vk (x) · w0 (x)+ ck ·wk (x) − y0 (x)+ ck ·yk (x) . k=1 k=1 k=1 (4) A QAP Q that satisfies this definition computes F . It has size m and its degree is the degree of t(x). Q In the above definition, t(x) = g∈G (x − rg ), where G is the set of multiplicative gates of the arithmetic circuit and each rg is an arbitrary value labeling a multiplicative gate of the circuit. The polynomials in V, W and Y encode the left inputs, the right inputs and the outputs for each gate respectively. By definition, if the polynomial p(x) vanishes at a value rg , p(rg ) expresses the relation between the inputs and outputs of the corresponding multiplicative gate g. An example of a QAP construction from an arithmetic circuit is given in [16]. It is important to note that the size of the QAP is the number of multiplicative gates in the arithmetic circuit to verify, which also is the metric used to evaluate the efficiency of the VC protocol. VC protocol Once a QAP has been built from an arithmetic circuit, a cryptographic protocol embeds it in an elliptic curve. In the verification phase, the divisibility check along with checks to ensure the QAP has been computed with the same coefficients ck for the V, W and Y polynomials during p’s computation are performed with a pairing. This results in a publicly verifiable computation scheme, as defined below. Definition 2. Let F be a function, expressed as an arithmetic circuit over a finite field F and λ be a security parameter. – (EKF , V KF ) ← KeyGen(1λ , F ): the randomized algorithm KeyGen takes as input a security parameter and an arithmetic circuit and produces two public keys, an evaluation key EKF and a verification key V KF . – (y, π) ← Prove(EKF , x): the deterministic Prove algorithm, takes as inputs x and the evaluation key EKF and computes y = F (x) and a proof π that y has been correctly computed. – {0, 1} ← Verify(V KF , x, y, π): the deterministic algorithm Verify takes the input/output (x, y) of the computation F , the proof π and the verification key V KF and outputs 1 if y = F (x) and 0 otherwise. Security The desired security properties for a publicly verifiable VC scheme, namely correctness, soundness and efficiency have been formally defined in [8]. Costs In QAP-based protocols, the proof consists of few elliptic curve elements, e.g. 8 group elements in Pinocchio [16] or 3 group elements in Groth’s state of the art VC system [13]. It has constant size no matter the computation to be verified, thus the verification is fast. In the set-up phase, the KeyGen algorithm outputs evaluation and verification keys that depend on the function F , but not on its inputs. The resulting model is often called pre-processing verifiable computation. This setup phase has to be run once, the keys are reusable for 8 Julien Keuffer, Refik Molva, Hervé Chabanne later inputs and the cost of the pre-processing is amortized over all further computations. The bottleneck of the scheme is the prover computations: for an arithmetic circuit of N multiplication gates, the prover has to compute O(N ) cryptographic operations and O(N log2 N ) non-cryptographic operations. Zero-knowledge QAPs also achieve the zero-knowledge property with little overhead: the prover can randomize the proof by adding multiples of the target polynomial t(x) to hide inputs he supplied in the computation. The proof obtained using Parno et al.’s protocol [16] or Groth’s scheme [13] is thus a zeroknowledge Succinct Non-Interactive Argument (zk-SNARK). In the zk-SNARKs setting, results are meaningful even if the efficiency requirement is not satisfied since the computation could not have been performed by the verifier. Indeed, some of the inputs are supplied by the prover and remain private, making the computation impossible to perform by the sole verifier. 2.2 EVC: Sum-Check Protocol The Sum-Check protocol [15] enables to prove the correctness of the sum of a multilinear polynomial over a subcube, the protocol is a public coin interactive proof with n rounds of interaction. Suppose that P is a polynomial with n variables defined over Fn . Using the Sum-Check protocol, a prover P can convince a verifier V that he knows the evaluation of P over {0, 1}n , namely: X X X H= ... P (t1 , . . . , tn ) (5) t1 ∈{0,1} t2 ∈{0,1} tn ∈{0,1} While a direct computation performed by the verifier would require at least 2n evaluations, the Sum-Check protocol P O(n) evaluations for the P only requires verifier. P first computes P1 (x) = tn ∈{0,1} P (x, t2 , . . . , tn ) and t2 ∈{0,1} . . . sends it to V, who checks if H = P1 (0) + P1 (1). If so, P’s claim on P1 holds, otherwise V rejects and the protocol stops. V picksPa random value r1 ∈ F and P sends it to P, who computes P2 = tn ∈{0,1} P (r1 , x, t3 , . . . , tn ). t3 ∈{0,1} . . . Upon receiving P2 , V checks if: P1 (r1 ) = P2 (0) + P2 (1). The protocol goes on until the nth round where V receives the value Pn (x) = P (r1 , r2 , . . . , rn−1 , x). V can now pick a last random field value rn and check that: Pn (rn ) = P (r1 , . . . , rn ). If so, V is convinced that H has been evaluated as in (5), otherwise V rejects H. The Sum-Check protocol has the following properties: 1. The protocol is correct: if P’s claim about H is true, then V accepts with probability 1. 2. The protocol is sound : if the claim on H is false, the probability that P can make V accept H is bounded by nd/|F|, where n is the number of variables and d the degree of the polynomial P . Note that the soundness is here information theoretic: no assumption is made on the prover power. To be able to implement the Sum-Check protocol verification algorithm into an arithmetic circuit we need a non-interactive version of the protocol. Indeed, QAP-based VC schemes require the complete specification Efficient Proof Composition for Verifiable Computation 9 of each computation as input to the QAP generation process (see Section 2.1). Due to the interactive nature of the Sum-Check protocol, the proof cannot be generated before the actual execution of the protocol. We therefore use the FiatShamir transformation [7] to obtain a non-interactive version of the Sum-Check protocol that can be used as an input to GVC. In the Fiat-Shamir transformation, the prover replaces the uniformly random challenges sent by the verifier by challenges he computes applying a public hash function to the transcript of the protocol so far. The prover then sends the whole protocol transcript, which can be verified recomputing the challenges with the same hash function. This method has been proved secure in the random oracle model [17]. 2.3 Multilinear extensions Multilinear extensions allow to apply the Sum-Check protocol to polynomials defined over some finite set included in the finite field where all the operations of the protocol are performed. Thaler [18] showed how multilinear extensions and the Sum-Check protocol can be combined to give a time-optimal proof for matrix multiplication. Let F be a finite field, a multilinear extension (MLE) of a function f : {0, 1}d → F is a polynomial that agrees with f on {0, 1}d and has degree at most 1 in each variable. Any function f : {0, 1}d → F has a unique multilinear extension over F, which we will denote hereafter by f˜. Using Lagrange interpolation, an explicit expression of MLE can be obtained: Lemma 1. Let f : {0, 1}d → {0, 1}. Then f˜ has the following expression: X f (w)χw (x1 , . . . , xd ) ∀(x1 , . . . , xd ) ∈ Fd , f˜(x1 , . . . , xd ) = (6) w∈{0,1}d where: w = (w1 , . . . , wd ) and χw (x1 , . . . , xd ) = d Y i=1 2.4
xi wi +(1−xi )(1−wi ) (7) Ajtai Hash Function As mentioned in Section 1.1, our goal is to compute a proof of an expensive sub-computation with the Sum-Check protocol and to verify that proof using the Pinocchio protocol. The non-interactive nature of Pinocchio prevents from proving the sub-computation with an interactive protocol. As explained in Section 2.2, we turn the Sum-Check protocol into a non-interactive argument using the Fiat-Shamir transform [7]. This transformation needs a hash function to simulate the challenges that would have been provided by the verifier. The choice of the hash function to compute challenges in the Fiat-Shamir transformation here is crucial because we want to verify the proof transcript inside the GVC system, which will be instantiated with the Pinocchio protocol. This means that the computations of the hash function have to be verified by the GVC system and that the verification should not be more complex than the execution of the 10 Julien Keuffer, Refik Molva, Hervé Chabanne original algorithm inside the GVC system. For instance the costs using a standard hash function such as SHA256 would be too high: [2] reports about 27,000 multiplicative gates to implement the compression function of SHA256. Instead, we choose a function better suited for arithmetic circuits, namely the Ajtai hash function [1] that is based on the subset sum problem as defined below: Definition 3. Let m, n be positive integers and q a prime number. For a randomly picked matrix A ∈ Zn×m , the Ajtai hash Hn,m,q : {0, 1}m → Znq is defined q as: ∀x ∈ {0, 1}m , Hn,m,q = A × x mod q (8) As proved by Goldreich et al. [10], the collision resistance of the hash function relies on the hardness of the Short Integer Solution (SIS) problem. The function is also regular : it maps an uniform input to an uniform output. Ben-Sasson et al. [4] noticed that the translation in arithmetic circuit is better if the parameters are chosen to fit with the underlying field of the computations. A concrete hardness evaluation is studied by Kosba et al. in [14]. Choosing Fp , with p ≈ 2254 to be the field where the computations of the arithmetic circuit take place leads to the following parameters for approximately 100 bit of security: n = 3, m = 1524, q = p ≈ 2254 . Few gates are needed to implement an arithmetic circuit for this hash function since it involves multiplications by constants (the matrix A is public): to hash m bits, m multiplicative gates are needed to ensure that the input vector is binary and 3 more gates are needed to ensure that the output is the linear combination of the input and the matrix. With the parameters selected in [14], this means that 1527 gates are needed to hash 1524 bits. 3 3.1 Embedded Proofs High level description of the generic protocol Let us consider two sets of functions (fi )1≤i≤n and (gi )1≤i≤n such that the fi do not lend themselves to an efficient verification with the GVC system whereas the gi can be handled by the GVC system efficiently. For an input x, we denote by y the evaluation of x by the function gn ◦ fn ◦ . . . g1 ◦ f1 . In our embedded proof protocol, each function fi is handled by a sub-prover Pi while the gi functions are handled by the prover P. The sub-prover Pi is in charge of the efficient VC algorithm EVCi and the prover P runs the GVC algorithm. The steps of the proof generation are depicted in Figure 2. Basically, each sub-prover Pi will evaluate the function fi on a given input, produce a proof of correct evaluation using the EVCi system and pass the output of fi and the related proof πi to P, who will compute the next gi evaluation and pass the result to the next sub-prover Pi+1 . In the Setup phase, the verifier and the prover agree on an arithmetic circuit which describes the computation of the functions gi along with the verification algorithms of the proof that the functions fi were correctly computed. The preprocessing phase of the GVC system takes the resulting circuit and outputs the corresponding evaluation and verification keys. Efficient Proof Composition for Verifiable Computation 11 Prover Verifier x, r $ r ←− F t0 := x For i = 1, . . . , n :  t2i−1 = fi (t2i−2 )  compute proof πi for t2i−1   t2i = gi (t2i−1 ) t2n := y EVCi t0 , . . . , t2n π1 , . . . , π n Compute proof πGVC that: ? VerifyGVC (πGVC , y, x, r) = 1 y, πGVC ( For i = 1, . . . , n : Verify(πi , t2i−1 , t2i−2 ) = 1 t2i = gi (t2i−1 ) GVC Fig. 2: Embedded proof protocol In the query phase, the verifier sends the prover an input x for the computation along with a random value that will be an input for the efficient sub-provers Pi . In the proving phase, P1 first computes t1 = f (x) and produces a proof π1 of the correctness of the computation, using the efficient proving algorithm EVC1 . The prover P then computes the value t2 = g1 (t1 ) and passes the value t2 to P2 , who computes t3 = f2 (t2 ) along with the proof of correctness π2 , using the EVC2 proving system. The protocol proceeds until y = t2n is computed. Finally, P provides the inputs/outputs of the computations and the intermediate proofs πi to the GVC system and, using the evaluation key computed in the setup phase, builds a proof πGVC that for i = 1, . . . , n: 1. the proof πi computed with the EVCi system is correct, 2. the computation t2i = gi (t2i−1 ) is correct. In the verification phase, the verifier checks that y was correctly computed using the GVC’s verification algorithm, the couple (y, πGVC ) received from the prover, and (x, r). Recall that our goal is to gain efficiency compared with the proof generation of the whole computation inside the GVC system. Therefore, we need proof algorithms with a verification algorithm that can be implemented efficiently as an arithmetic circuit and for which the running time of the verification algorithm is lower than the one of the computation. Since the Sum-Check protocol involves algebraic computations over a finite field, it can easily be implemented as an arithmetic circuit and fits into our scheme. 3.2 A Protocol instance In this section, we specify the embedded proofs protocol in the case where fi are matrix products fi : X 7→ Wi ×X and where the functions gi cannot be efficiently verified by a VC system except by GVC. We use the Sum-Check protocol to prove 12 Julien Keuffer, Refik Molva, Hervé Chabanne correctness of the matrix multiplications, as in [18] and any QAP-based VC scheme as the global proof mechanism. We assume that the matrices involved in the fi functions do not have the same sizes so there will be several instances of the Sum-Check protocol. It thus makes sense to define different efficient proving algorithms EVCi since the GVC scheme requires that the verification algorithms are expressed as arithmetic circuits in order to generate evaluation and verification keys for the system. As the parameters of the verification algorithms are different, the Sum-Check verification protocols are distinct as arithmetic circuits. For the sake of simplicity, the Wi matrices are assumed to be square matrices of size ni . We denote di = log ni and assume that ni ≥ ni+1 . We denote by H the Ajtai hash function (see Section 2). The protocol between the verifier V and the prover P, which has n sub-provers Pi is the following: Setup: – V and P agree on an arithmetic circuit C description for the computation. C implements both the evaluations of the functions gi and the verification algorithms of the Sum-Check protocols for the n matrix multiplications. – (EKC , V KC ) ← KeyGen(1λ , C) Query – V generates a random challenge (rL , rR ) such that: (rL , rR ) ∈ Fd1 × Fd1 – V sends P the tuple (X, rL , rR ), where X is the input matrix. Proof : for i = 1, . . . , n, on input (T2i−2 , rL , rR ), Sub-prover Pi : – computes the product T2i−1 = Wi × T2i−2 , (denoting T0 := X) – computes rLi and rRi (the di first component of rL and rR ), – computes the multilinear extension evaluation Te2i−1 (rLi , rRi ) – computes with serialized Sum-Check, the proof πi of Pi evaluation: fi (rL , x) · Te2i−2 (x, rR ) where x = (x1 , . . . , xd ) ∈ Fdi . Pi (x) = W i i i (9) – sends the tuple (T2i−2 , T2i−1 , Wi , πi , rLi , rRi ) to prover P. Prover P: – computes T2i = gi (T2i−1 ) and sends (T2i , rL , rR ) to sub-prover Pi+1 – receiving the inputs {(T2i−2 , T2i−1 , Wi , πi , rLi , rRi )}i=1,...,n from sub-provers: • Computes Te2i−1 (rLi , rRi ). • Parses πi as (Pi,1 , ri,1 , Pi,2 , ri,2 , ..., Pi,d1 , ri,di ), where the proof contains the coefficient of the degree two polynomials Pi,j that we denote by (ai,j , bi,j , ci,j ) if: Pi,j (x) = ai,j x2 + bi,j x + ci,j • Verifies πi : ? ∗ Checks: Pi,1 (0) +  Pi,1 (1) = Te2i−1 (rLi , rRi )  P P ∗ Computes: ri,1 = j rRi [j] j rLi [j] · ∗ For j = 2, . . . , di : ? · Check: Pi,j (0) + Pi,j (1) = Pi,j−1 (ri,j−1 ) · Computes: ri,j as the product of components of the Ajtai hash Q3 function output, i.e. ri,j = k=1 H(ai,j , bi,j , ci,j , ri,j )[k] Efficient Proof Composition for Verifiable Computation 13 ∗ From T2i−2 and Wi , computes the evaluated multilinear extenfi (rL , ri,1 , . . . , ri,d ) and Te2i−2 (ri,1 , . . . , ri,d , rR ) sions W i 1 1 i ∗ Checks that Pdi (ri,di ) is the product of the multilinear extensions fi (rL , ri,1 ,..., ri,d ) and Te2i−2 (ri,1 ,..., ri,d , rR ). W i i i i • Aborts if one of the previous checks fails. Otherwise, accepts T2i−1 as the product of Wi and T2i−2 . • Repeat the above instructions until the proof πn has been verified. • Using the GVC scheme, computes the final proof πGVC that all the EVCi proofs πi have been verified and all the T2i values have been correctly computed from T2i−1 . • Sends (Y, πGVC ) to the Verifier. Verification – V computes Verify(X, rR , rL , Y, πGVC ) – If Verify fails, verifier rejects the value Y . Otherwise the value Y is accepted as the result of: Y = gn (...(g2 (W2 (g1 (W1 · X))))...) 4 4.1 Embedded proofs for Neural Networks Motivation In order to show the relevance of the proposed embedded proof scheme, we apply the resulting scheme to Neural Networks (NN), which are machine learning techniques achieving state of the art performance in various classification tasks such as handwritten digit recognition, object or face recognition. As stated in Section 1.1, a NN can be viewed as a sequence of operations, the main ones being linear operations followed by so-called activation functions. The linear operations are modeled as matrix multiplications while the activation functions are non-linear functions. A common activation function choice is the ReLU function defined by: x 7→ max(0, x). Due to the sequential nature of NNs, a simple solution to obtain a verifiable NN would consist of computing proofs for each part of the NN sequence. However, this solution would degrade the verifier’s performance, increase the communication costs and force the prover to send all the intermediate results, revealing sensitive data such as the parameters of the prover’s NN. On the other hand, even if it is feasible in principle to implement the NN inside a GVC system like Pinocchio, the size of the matrices involved in the linear operations would be an obstacle. The upper bound for the total number of multiplications QAP-based VC schemes can support as part of one application is estimated at 107 [19]. This threshold would be reached with a single multiplication between two 220 × 220 matrices. In contrast, our embedded proof protocol enables to reach much larger matrix sizes or, for a given matrix size, to perform faster verifications of matrix multiplications. 4.2 A Verifiable Neural Network Architecture We here describe how our proposal can provide benefits in the verification of a neural network (NN) [12]: in the sequel, we compare the execution of a GVC 14 Julien Keuffer, Refik Molva, Hervé Chabanne protocol on a two-layer NN with the execution of the embedded proof protocol on the same NN. Since NN involve several matrix multiplications, embedded proofs enable substantial gains, see Section 5.2 for implementation report. We stress that we consider neural networks in the classification phase, which means we consider that all the values have been set during the training phase, using an appropriate set of labeled inputs. The NN we verify starts with a fully connected layer combined with a ReLU activation layer. We then apply a max pooling layer to decrease the dimensions and finally apply another fully connected layer. The execution of the NN can be described as: input → fc → relu → max pooling → fc The fully connected layer takes as input a value and performs a dot product between this value and a parameter that can be learned. Gathering all the fully connected layer parameters in a matrix, the operation performed on the whole inputs is a matrix multiplication. The ReLU layer takes as input a matrix and performs the operation x 7→ max(0, x) element-wise. The max pooling layer takes as input a matrix and return a matrix with smaller dimensions. This layer applies a max function on sub-matrices of the input matrix, which can be considered as sliding a window over the input matrix and taking the max of all the values belonging to the window. The size of the window and the number of inputs skipped between two mapping of the max function are parameters of the layer but do not change during the training phase nor on the classification phase. Usually a 2 × 2 window slides over the input matrix, with no overlapping over the inputs. Therefore, the MaxPool function takes as input a matrix and outputs a matrix which row and column size have been divided by 2. Denoting by W1 and W2 the matrices holding the parameters of the fully connected layers, X the input matrix, and Y the output of the NN computation, the whole computation can be described as a sequence of operations: X → T1 = W1 ·X → T2 = ReLU(T1 ) → T3 = MaxPool(T2 ) → Y = W2 ·T3 (10) 5 Implementation and Performance Evaluation We ran two sets of experiments to compare the cost of execution between our embedded proof scheme and a baseline scheme using the GVC scheme. The first set focuses only on the cost of a matrix multiplication since these are a relevant representative of complex operations whereby the embedded proof scheme is likely to achieve major performance gains. The second set takes into account an entire application involving several operations including matrix multiplications, namely a small neural network architecture. 5.1 Matrix multiplication benchmark We implemented our embedded proof protocol on a 8-core machine running at 2.9 GHz with 16 GB of RAM. The GVC system is Groth’s state of the art zk- Efficient Proof Composition for Verifiable Computation 15 Table 1: Matrix multiplication benchmark (a) Matrix multiplication proving time n 16 32 64 128 256 512 Baseline (GVC only) 0.23 s 1.34 s 9.15 s 71.10 s 697.72 − 0.281 s 0.924 s 3.138 s 11.718 s 43.014 s 168.347 s Embedded proofs Time division 0.28|0.001 0.92|0.004 3.12|0.018 11.65|0.068 42.71|0.304 166.88|1.467 (b) Matrix multiplication key generation time n 16 32 64 128 256 512 Baseline (GVC only) 0.28 s 1.56 s 10.50 s 76.62 s 585.21 s Embedded proofs − 0.37 s 1.03 s 3.54 s 12.95 s 47.52 s 176.41 s (c) Matrix multiplication key generation size n 16 32 64 128 256 Baseline (GVC only) PK 508 KB 5.60 MB 26.9 MB 208 MB 1.63 GB Embedded proofs PK − 757 kB 2.24 MB 7.87 MB 30.1 MB 118.7 MB 472 MB Baseline (GVC only) VK 31 kB Embedded proofs VK 512 123 kB 490 kB 1.96 MB 7.84 MB − 32 KB 123 KB 491 KB 1.96 MB 7.84 MB 31.36 MB SNARK [13] and is implemented using the libsnark library 4 while the EVC system is our own implemention of Thaler’s special purpose matrix multiplication verification protocol [18] using the NTL library 5 The proving time reported in table 1a measures the time to produce the proof using the EVC system and to verify the result inside the GVC system. The last row of the table breaks down the proving time into the sumcheck proof time and the embedded proof time. We note that the sumcheck proving time brings a very small contribution to the final proving time. For the value n = 512, the proof using the GVC is not feasible whereas the embedded proof approach still achieves realistic performance. Table 1b compares the key generation time using the embedded proof system with the one using the GVC. Table 1c states the sizes of the proving key (PK) and the verification key (VK) used in the previous scenarios. The embedded proof protocol provides substantial benefits: the protocol improves the proving time as soon as the matrix has a size greater than 32 × 32, giving a proving time 7 times better for 128 × 128 matrix multiplication and 16 times better for a 256 × 256 matrix multiplication. Embedded proofs also enable to reach higher value for matrix multiplications: we were able to perform a multiplication proof for 512 × 512 matrices whereas the computation was not able to terminate due to lack of RAM for the baseline system. 4 5 Libsnark, a C++ library for zkSNARK proofs, available at https://github.com/ scipr-lab/libsnark V. Shoup, NTL – A Library for Doing Number Theory, available at http://www. shoup.net 16 Julien Keuffer, Refik Molva, Hervé Chabanne Table 2: Experiments on 2-layer networks (a) NN-64-32 KeyGen PK size VK size Prove Verify Baseline (GVC only) 59 s 148 MB 490 kB 25.48 s 0.011 s Embedded proofs 44 s 123 MB 778 kB 16.80 s 0.016 s (b) NN-128-64 KeyGen PK size VK size Prove Verify Baseline (GVC only) 261.9 s 701.5 MB 1.96 MB 149.5 s 0.046 s Embedded proofs 5.2 162.7 s 490 MB 3.1 MB 66.96 s 0.067 s Two-Layer Verifiable Neural Network Experimentations We implemented the verification of an example of 2-layer neural network, which can be seen as one matrix multiplication followed by the application of two nonlinear functions, namely a ReLU and a max pooling function as described in Section 4. For our experiments, the max pooling layers have filters of size 2 × 2 and no data overlap. Thus, setting for instance the first weight matrix to 64×64, the second weight matrix size is 32 × 32; we denote by NN-64-32 such a neural network. Table 2a reports experiments on a 2-layer neural network with a first 64 × 64 matrix product, followed by a ReLU and a max-pooling function, and ending with a second 32 × 32 matrix product. Experimental times for a NN128-64 network (with the same architecture as above) are reported in table 2b. Experiments show a proving time twice better than using the baseline proving system. The overall gain is lower than for the matrix product benchmark because the other operations (ReLU and max pooling) are implemented the same way for the two systems. It should be noted that the goal of the implementation was to achieve a proof of concept for our scheme on a complete composition scenario involving several functions rather than putting in evidence the performance advantages of the scheme over the baseline, hence the particularly low size of the matrices used in the 2-layer NN and an advantage as low as the one in table 2a and table 2b. The gap between the embedded proof scheme and the baseline using a realistic scenario with larger NN would definitely be much more significant due to the impact of larger matrices as shown in the matrix product benchmark. 6 Conclusion We designed an efficient verifiable computing scheme that builds on the notion of proof composition and leverages an efficient VC scheme, namely the Sum-Check protocol to improve the performance of a general purpose QAP-based VC protocol, in proving matrix multiplications. As an application, our scheme can prove the correctness of a neural network algorithm. We implement our scheme and Efficient Proof Composition for Verifiable Computation 17 provide an evaluation of its efficiency. The security is evaluated in Appendix A. We stress that the composition technique described in the article can be extended using other efficient VC schemes and an arbitrary number of sequential function evaluations, provided that they respect the requirements defined in Section 1.2. Our proposal could be integrated as a sub-module in existing verifiable computing systems in order to improve their performance when verifying computations including high complexity operations such as matrix multiplications. Acknowledgment The authors would like to thank Gaı̈d Revaud for her precious programming assistance. This work was partly supported by the TREDISEC project (G.A. no 644412), funded by the European Union (EU) under the Information and Communication Technologies (ICT) theme of the Horizon 2020 (H2020) research and innovation programme. References 1. Ajtai, M.: Generating hard instances of lattice problems (extended abstract). In: Proceedings of the Twenty-Eighth Annual ACM Symposium on the Theory of Computing, Philadelphia, Pennsylvania, USA, May 22-24, 1996. pp. 99–108 (1996) 2. Ben-Sasson, E., Chiesa, A., Garman, C., Green, M., Miers, I., Tromer, E., Virza, M.: Zerocash: Decentralized anonymous payments from bitcoin. In: 2014 IEEE Symposium on Security and Privacy, SP 2014, Berkeley, CA, USA, May 18-21, 2014. pp. 459–474 (2014) 3. Ben-Sasson, E., Chiesa, A., Genkin, D., Tromer, E., Virza, M.: Snarks for C: verifying program executions succinctly and in zero knowledge. In: Advances in Cryptology - CRYPTO 2013 - 33rd Annual Cryptology Conference, Santa Barbara, CA, USA, August 18-22, 2013. Proceedings, Part II. pp. 90–108 (2013) 4. Ben-Sasson, E., Chiesa, A., Tromer, E., Virza, M.: Scalable zero knowledge via cycles of elliptic curves. In: Advances in Cryptology - CRYPTO 2014 - 34th Annual Cryptology Conference, Santa Barbara, CA, USA, August 17-21, 2014, Proceedings, Part II. pp. 276–294 (2014) 5. Bitansky, N., Canetti, R., Chiesa, A., Tromer, E.: Recursive composition and bootstrapping for SNARKS and proof-carrying data. In: Symposium on Theory of Computing Conference, STOC’13, Palo Alto, CA, USA, June 1-4, 2013. pp. 111–120 (2013), http://doi.acm.org/10.1145/2488608.2488623 6. Cormode, G., Mitzenmacher, M., Thaler, J.: Practical verified computation with streaming interactive proofs. In: Innovations in Theoretical Computer Science 2012, Cambridge, MA, USA, January 8-10, 2012. pp. 90–112 (2012) 7. Fiat, A., Shamir, A.: How to prove yourself: Practical solutions to identification and signature problems. In: Advances in Cryptology - CRYPTO ’86, Santa Barbara, California, USA, 1986, Proceedings. pp. 186–194 (1986) 8. Gennaro, R., Gentry, C., Parno, B., Raykova, M.: Quadratic span programs and succinct nizks without pcps. In: Advances in Cryptology - EUROCRYPT 2013, 32nd Annual International Conference on the Theory and Applications of Cryptographic Techniques, Athens, Greece, May 26-30, 2013. Proceedings. pp. 626–645 (2013) 18 Julien Keuffer, Refik Molva, Hervé Chabanne 9. Ghodsi, Z., Gu, T., Garg, S.: Safetynets: Verifiable execution of deep neural networks on an untrusted cloud. CoRR abs/1706.10268 (2017), http://arxiv.org/ abs/1706.10268 10. Goldreich, O., Goldwasser, S., Halevi, S.: Collision-free hashing from lattice problems. Electronic Colloquium on Computational Complexity (ECCC) 3(42) (1996), http://eccc.hpi-web.de/eccc-reports/1996/TR96-042/index.html 11. Goldwasser, S., Kalai, Y.T., Rothblum, G.N.: Delegating computation: interactive proofs for muggles. In: Proceedings of the 40th Annual ACM Symposium on Theory of Computing, Victoria, British Columbia, Canada, May 17-20, 2008. pp. 113–122 (2008) 12. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press (2016), http: //www.deeplearningbook.org 13. Groth, J.: On the size of pairing-based non-interactive arguments. In: Advances in Cryptology - EUROCRYPT 2016 - 35th Annual International Conference on the Theory and Applications of Cryptographic Techniques, Vienna, Austria, May 8-12, 2016, Proceedings, Part II. pp. 305–326 (2016) 14. Kosba, A., Zhao, Z., Miller, A., Qian, Y., Chan, H., Papamanthou, C., Pass, R., abhi shelat, Shi, E.: C∅c∅: A framework for building composable zero-knowledge proofs. Cryptology ePrint Archive, Report 2015/1093 (2015), http://eprint. iacr.org/2015/1093 15. Lund, C., Fortnow, L., Karloff, H.J., Nisan, N.: Algebraic methods for interactive proof systems. In: 31st Annual Symposium on Foundations of Computer Science, St. Louis, Missouri, USA, October 22-24, 1990, Volume I. pp. 2–10 (1990) 16. Parno, B., Howell, J., Gentry, C., Raykova, M.: Pinocchio: Nearly practical verifiable computation. In: 2013 IEEE Symposium on Security and Privacy, SP 2013, Berkeley, CA, USA, May 19-22, 2013. pp. 238–252 (2013) 17. Pointcheval, D., Stern, J.: Security arguments for digital signatures and blind signatures. J. Cryptology 13(3), 361–396 (2000), https://doi.org/10.1007/ s001450010003 18. Thaler, J.: Time-optimal interactive proofs for circuit evaluation. In: Advances in Cryptology - CRYPTO 2013 - 33rd Annual Cryptology Conference, Santa Barbara, CA, USA, August 18-22, 2013. Proceedings, Part II. pp. 71–89 (2013) 19. Wahby, R.S., Setty, S.T.V., Ren, Z., Blumberg, A.J., Walfish, M.: Efficient RAM and control flow in verifiable outsourced computation. In: 22nd Annual Network and Distributed System Security Symposium, NDSS 2015, San Diego, California, USA, February 8-11, 2015 (2015) 20. Walfish, M., Blumberg, A.J.: Verifying computations without reexecuting them. Commun. ACM 58(2), 74–84 (Jan 2015), http://doi.acm.org/10.1145/2641562 A Appendix: Embedded Proofs Security Our embedded proof system has to satisfy the correctness and soundness requirements. Suppose that we have a GVC and n EVC systems to prove the correct computation of y = gn ◦fn ◦. . .◦g1 ◦f1 (x). We will denote by EVCi , i = 1, . . . , n the EVC systems. We also keep notations defined in Section 3: the value ti , i = 0, . . . , 2n represents intermediate computation results, t2i−1 being the output of the fi function, t2i being the output of the gi function, t0 := x and t2n = y. The EVCi and GVC systems already satisfy the correctness and soundness requirements. Let denote by ǫGVC the soundness error of the GVC system and ǫEVCi the Efficient Proof Composition for Verifiable Computation 19 soundness error of the EVCi system. Note that while the EVCi systems prove that t2i−1 = fi (t2i−2 ) have been correctly computed, the GVC system proves the correctness of 2n computations, namely that the verification of the EVCi proofs has passed and that the computations t2i = gi (t2i−1 ) are correct. Furthermore, the GVC system proves the correct execution of the function F that takes as input the tuple (x, y, r, (ti )i=1,...,2n , (πi )i=1,...,n ) and outputs 1 if for all i = 1, . . . , n, VerifyEVCi (πi , t2i−1 , t2i−2 ) = 1 and t2i = gi (t2i−1 ). F outputs 0 otherwise. For convenience, we denote by compn the function gn ◦ fn ◦ . . . ◦ g1 ◦ f1 . A.1 Correctness Theorem 1. If the EVCi and the GVC systems are correct then our embedded proof system is correct. Proof. Assume that the value y = compn (x) has been correctly computed. This means that for i = 1, . . . , n, the values t2i−1 = fi (t2i−2 ) and t2i = gi (t2i−1 ) have been correctly computed. Since the GVC system is correct, it ensures that the function F will pass the GVC verification with probability 1, provided that its result is correct. Now, since the EVCi systems are correct, with probability 1 we have that: VerifyEVCi (t2i−1 , t2i−2 , πi ) = 1. Therefore, if y = compn (x) has been correctly computed, then the function F will also be correctly computed and the verification of the embedded proof system will pass with probability 1. A.2 Soundness Theorem 2. If the EVCi and the GVC systems are sound with soundness error respectively equal to ǫEVCi and ǫP GVC , then our embedded proof system is sound with soundness error at most ǫ := ǫEVCi + ǫGVC . Proof. Assume that a p.p.t. adversary Aemb returns a cheating proof π for a result y ′ on input x, i.e. y ′ 6= comp(x) and π is accepted by the verifier Vemb with probability higher than ǫ. We then construct an adversary B that breaks the soundness property of either the GVC or of one of the EVC systems. We build B as follows: Aemb interacts with the verifier Vemb of the embedded system until a cheating proof is accepted. Aemb then forwards the cheating tuple (x, y, r, (ti )i=1,...,2n , (πi )i=1,...,n ) for which the proof π has been accepted. Since y ′ 6= comp(x), there exists an index i ∈ {1, . . . , n} such that either t2i−1 6= fi (t2i−2 ) or t2i 6= gi (t2i−1 ). B can thus submit a cheating proof to the GVC system or to one of the EVCi system, depending on the value of i. Case t2i−1 6= fi (t2i−2 ): By definition of the proof π, this means that the proof πi has been accepted by the verification algorithm of EVCi implemented inside the GVC system. Aemb can then forward to the adversary B the tuple (t2i−1 , t2i−2 , πi ). Now if B presents the tuple (t2i−1 , t2i−2 , πi ) to the EVCi system, it succeeds with probability 1. 20 Julien Keuffer, Refik Molva, Hervé Chabanne Therefore, the probability that the verifier Vemb of the embedded proof system accepts is superior to ǫEVCi , which breaks the soundness property of EVCi . P r[Vemb accepts π] = P r[VEVCi accepts πi | Vemb accepts π] × P r[Vemb accepts π] = 1 × ǫ > ǫEVCi Case t2i 6= gi (t2i−1 ): This means that the proof π computed by the GVC system is accepted by Vemb even if t2i 6= gi (t2i−1 ) has not been correctly computed. We proceed as in the previous case: Aemb forwards B the cheating tuple and the cheating proof π. The tuple and the proof thus break the soundness of the GVC scheme because: P r[Vemb accepts π] = ǫ ≥ ǫGVC B B.1 Appendix: Prover’s input privacy Prover’s input privacy The combination of the proof of knowledge and zero knowledge properties in zkSNARK proofs enables the prover to provide some inputs for the computation to be proved for which no information will leak. Gennaro et al. proved in [8] that their QAP-based protocol (see Section 2.1) is zero-knowledge for input provided by the prover: there exists a simulator that could have generated a proof without knowledge of the witness (here the input privately provided by the prover). Subsequent works on QAP-based VC schemes achieve the same goal, with differences on the cryptographic assumptions and on the flavor of zero-knowledge achieved: for instance Groth’s scheme [13] achieves perfect zeroknowledge at the expense of a proof in the generic group model while Gennaro et al.’s scheme achieves statistical zero-knowledge with a knowledge of exponent assumption. We now sketch how the zero-knowledge property is achieved for our embedded proof protocol. We first have to assume that the QAP-based VC scheme we consider for GVC can support auxiliary inputs (as in NP statements), which is achieved for Pinocchio [16] or Groth’s scheme [13]. Leveraging the zero-knowledge property, the GVC prover can hide from the verifier the intermediate results of the computation provided by the sub-provers EVCi while still allowing the verifier to check the correctness of the final result. Therefore, the overall VC system obtained by composing the EVC systems inside the GVC achieves zero-knowledge: the simulator defined for the GVC system is still a valid one. Note that even if the simulator gains knowledge of the intermediate computations performed by sub-provers EVCi , the goal is to protect the leakage of information from outside, namely from the verifier. In detail, keeping the notations of figure 2, the verifier only knows t0 , i.e. x and t2n , i.e. y, which are the inputs and outputs of the global computation. Intermediate inputs, such as ti , i = 1, . . . , 2n − 1, are hidden from the verifier even though they are taken into account during the verification of the intermediate proofs by the GVC prover. Therefore, thanks to the zk-SNARKs, the intermediate results are verified but not disclosed to the verifier.