From c0b9b567b6ea091bb6d930c567e7dbc20b66f0b2 Mon Sep 17 00:00:00 2001 From: dawidkopczyk <32499114+dawidkopczyk@users.noreply.github.com> Date: Mon, 28 May 2018 23:59:53 +0200 Subject: [PATCH 1/3] Add files via upload First version --- docs/qcl.rst | 245 +++++++++++++++++++++++++++ docs/qcl/qcl_classification.png | Bin 0 -> 23725 bytes docs/qcl/qcl_classification_data.png | Bin 0 -> 15561 bytes docs/qcl/qcl_loss.png | Bin 0 -> 17990 bytes docs/qcl/qcl_regression.png | Bin 0 -> 20036 bytes 5 files changed, 245 insertions(+) create mode 100644 docs/qcl.rst create mode 100644 docs/qcl/qcl_classification.png create mode 100644 docs/qcl/qcl_classification_data.png create mode 100644 docs/qcl/qcl_loss.png create mode 100644 docs/qcl/qcl_regression.png diff --git a/docs/qcl.rst b/docs/qcl.rst new file mode 100644 index 0000000..0fe4f75 --- /dev/null +++ b/docs/qcl.rst @@ -0,0 +1,245 @@ +Quantum-Circuit-Learning (QCL) +===================================== + +Overview +-------- + +The Quantum-Circuit-Learning (QCL) [`1 `_] is a quantum/classical hybrid algorithm that aims to perform supervised or unsupervised learning tasks. In supervised tasks, the algorithm is supplied with data :math:`\{x_i\}` and corresponding labels :math:`\{y_i\}`. Then, the algorithm learns to output :math:`\{f(x_i,\theta)\}` which is as close as possible to testing labels, not used in training. In this hybrid algorithm a quantum subroutine calculates the output :math:`\{f(x_i,\theta)\}`, whereas the optimization of :math:`\theta` is executed in a classical optimization loop. + +The outline of the algorithm for :math:`N`-qubits circuit is as follows: + +1. **Encode input data** :math:`\{x_i\}` into a quantum state :math:`|\Psi_{in}(x_i)\rangle` applying some unitary transformation :math:`\U(x_i)}` to initialized qubits :math:`|0\rangle`. + +2. Apply :math:`\theta`-parametrized unitary on input state :math:`|\Psi_{in}(x_i)\rangle` and **generate output state** :math:`U(\theta)|\Psi_{in}(x_i)\rangle=|\Psi_{out}(x_i,\theta)\rangle`. + +3. Measure the **expectation values** of a subset of Pauli operators :math:`\{B\} \subset \{I,X,Y,Z\}^N`. + +4. If necessary, make a transformation of expectation value by some function :math:`F` and calculate an **output** :math:`\{f(x_i,\theta)\}=F(\langle B(x_i,\theta) \rangle)`. + +5. Calculate **loss** :math:`L{y_i,f(x_i,\theta))` and **minimize** it using classical optimization algorithm such as gradient descend. + +The quantum subroutine of QCL amounts to encoding an input data into a quantum state, applying an :math:`\theta`-parametrized unitary and calculating expectation values for each training sample. Additionally, a gradient descend method requires to calculate gradients, which are also obtained by calculation of two expectation values :math:`\frac{\partial \langle B \rangle}{\partial \theta_j}=\frac{\langle B \rangle^{+}-\langle B \rangle^{-}}{2}`. +To calculate :math:`\langle B \rangle^{\pm}`, one has to modify the output state by inserting :math:`\pm \frac{\pi}{2}` rotations +next to the :math:`\theta_j`-dependend unitary :math:`U_j(theta_j)` (assuming that :math:`U(\theta)` consists of +a chain of unitary transformatiosn :math:`\{U_j(theta_j)\}`. For more details see the original paper [`1 `_]. + +Below there are simple examples of regression task and classification task presented. + +Regression example +----------- + +In this example, QCL will try to learn a simple quadratic function :math:`x^2`. + +Firstly, we generate a small dataset: + +.. code:: python + + import numpy as np + + np.random.seed(0) + m = 8 + X = np.linspace(-0.95,0.95,m) + y = X**2 + +Next thing is to write a pyQuil program that encodes input data into a quantum state :math:`|\Psi_{in}(x_i)\rangle`. +We will do that applying :math:`R^X(\cos(x^2))R^Y(\sin(x))` gate on each qubit in :math:`N=3`-qubits quantum circuit. + +.. code:: python + + import pyquil.quil as pq + from pyquil.gates import RX, RY, RZ + + n_qubits = 3 + def input_prog(sample): + p = pq.Program() + for j in range(n_qubits): + p.inst(RY(np.arcsin(sample[0]), j)) + p.inst(RZ(np.arccos(sample[0]**2), j)) + return p + +The output program is generated by evolving quantum system accordingly to a fully connected transverse Ising model Hamilonian +and then applying :math:`R^X(\theta_{j,1})R^Z(\theta_{j,2})R^X(\theta_{j,3})` gate on each qubit. This procedure is reapeated :math:`D` times to increase the learning capacity of QCL. The Ising model Hamiltonian is expressed by following formula: + +:math:`H = \sum{j=1}{N}h_j X_j + \sum{j=1}{N}\sum{k=1}{j-1}J_{jk}Z_j Z_k` + +and then the evolution with time :math:`T=10` is expressed by :math:`\exp(-iTH)`. The dynamics of this Hamiltonian generates a highly entangled state and is a key ingredient to successfully learn an output. To generate a Quil program responsible for the approximation of Ising model evolution we use a helper function: + +.. code:: python + + from grove.pyqcl.qcl import ising_prog_gen + ising_prog = ising_prog_gen(trotter_steps=1000, T=10, n_qubits=n_qubits) + +The output state is generated with the following function (:math:`D` is denoted here by depth variable): + +.. code:: python + + depth = 3 + def output_prog(theta): + p = pq.Program() + theta = theta.reshape(3,n_qubits,depth) + for i in range(depth): + p += ising_prog + for j in range(n_qubits): + p.inst(RX(theta[0,j,i], j)) + p.inst(RZ(theta[1,j,i], j)) + p.inst(RX(theta[2,j,i], j)) + return p + +The last thing is to write a function generating Quil program responsible for gradient calculations accordingly to formulas presented in the paper. This is done by inserting :math:`\pm \frac{\pi}{2}` rotations: + +.. code:: python + + def grad_prog(theta, idx, sign): + theta = theta.reshape(3,n_qubits,depth) + idx = np.unravel_index(idx, theta.shape) + p = pq.Program() + for i in range(depth): + p += ising_prog + for j in range(n_qubits): + p.inst(RX(theta[0,j,i], j)) + if idx == (0,j,i): + p.inst(RX(sign*np.pi/2.0, j)) + p.inst(RZ(theta[1,j,i], j)) + if idx == (1,j,i): + p.inst(RZ(sign*np.pi/2.0, j)) + p.inst(RX(theta[2,j,i], j)) + if idx == (2,j,i): + p.inst(RX(sign*np.pi/2.0, j)) + return p + +Now, it is time to run QCL. We initialize :math:`\theta` parameters with random numbers drawn from uniform distribution on :math:`[0,2*\pi]`. The output is taken from :math:`Z` expectation values on the first qubit and we use mean squared error as a loss function minimized. A number of training iterations (epochs) is set to :math:`20` and we use full-batch gradient descend. For mean squared error loss function, the expectation value is multiplied by a coefficient which is also optimized, however, this is done inside the code. The aim of this multiplication is to scale the expectation value. + +.. code:: python + + import pyquil.api as api + from pyquil.gates import Z + from grove.pyqcl.qcl import QCL + + qvm = api.QVMConnection() + + state_generators = dict() + state_generators['input'] = input_prog + state_generators['output'] = output_prog + state_generators['grad'] = grad_prog + + initial_theta = np.random.uniform(0.0, 2*np.pi, size=3*n_qubits*depth) + + operator = [pq.Program(Z(n_qubits-1))] + est = QCL(state_generators, initial_theta, loss="mean_squared_error", + operator_programs=operator, epochs=20, batch_size=m, + verbose=True, qvm=qvm) + +We fit a QCL estimator based on our data and labels, get the results for inspection and predict to produce a plot. +(Training can take a while as :math:`3*n_qubits*depth*m*3*epochs=12960` expectation values need to be simulated on QVM machine.) + +.. code:: python + est.fit(X,y) + results = est.get_results() + + X_test = np.linspace(-1.0,1.0,50) + y_pred = est.predict(X_test) + + import matplotlib.pyplot as plt + plt.plot(X, y, 'bs', X_test, y_pred, 'r-') + +As we see the QCL fits nicely to the data. Assuming, we have a real quantum computer, we can increase significantly number of qubits, depth and number of samples to cope with more complex regression tasks. + +.. image:: qcl/qcl_regression.png + :align: center + +History of loss values is presented in the plot below. + +.. image:: qcl/qcl_loss.png + :align: center + +Classification example +----------- + +In this example, QCL will try to perform a simple nonlinear classification task. The data points belong to two classes 0 (red dots) and 1 (blue dots). + +.. image:: qcl/qcl_classification_data.png + :align: center + +The algorithm structure is very similar to the QCL regression example. We generate a data with sklearn make_circles method: + +.. code:: python + from sklearn.datasets import make_circles + np.random.seed(0) + m = 10 + X, y = make_circles(n_samples=m, factor=.1, noise=.0, random_state=0) + +Next, we produce a function generating input, output and gradient states. The default methods of QCL can be used. + +.. code:: python + from grove.pyqcl.qcl import (ising_prog_gen, default_input_state_gen, + default_output_state_gen, default_grad_state_gen) + + n_qubits, depth = 4, 4 + + ising_prog = ising_prog_gen(trotter_steps=1000, T=10, n_qubits=n_qubits) + state_generators = dict() + state_generators['input'] = default_input_state_gen(n_qubits) + state_generators['output'] = default_output_state_gen(ising_prog, n_qubits, depth) + state_generators['grad'] = default_grad_state_gen(ising_prog, n_qubits, depth) + +We increase the number of qubits and depth of quantum circuit in comparison to regression task to get a better fit. The output is taken from :math:`Z` expectation values on the first and second qubit and we use binary crossentropy as a loss function minimized. A number of training iterations (epochs) is set to :math:`20` and we use full-batch gradient descend. For binary crossentropy loss function the expectation values are transformed by softmax function to get valid probabilities, however, this is done inside the code. +(Training can take a while as many expectation values need to be simulated on QVM machine.) + +.. code:: python + initial_theta = np.random.uniform(0.0, 2*np.pi, size=3*n_qubits*depth) + operator = [pq.Program(Z(n_qubits-1)), pq.Program(Z(n_qubits-2))] + est = QCL(state_generators, initial_theta, loss="binary_crossentropy", + operator_programs=operator, epochs=20, batch_size=m, + verbose=True, qvm=qvm) + + est.fit(X,y) + results = est.get_results() + +Now, we can plot the decision surface of a fitted QCL estimator: + +.. code:: python + import matplotlib.pyplot as plt + from matplotlib.colors import ListedColormap + cm = plt.cm.RdBu + cm_bright = ListedColormap(['#FF0000', '#0000FF']) + xx, yy = np.meshgrid(np.linspace(-1.0, 1.0, 10), + np.linspace(-1.0, 1.0, 10)) + y_pred = est.predict(np.c_[xx.ravel(), yy.ravel()])[:,0] + Z = y_pred.reshape(xx.shape) + plt.contourf(xx, yy, Z, cmap=cm, alpha=.8) + plt.scatter(X[:, 0], X[:, 1], c=y, cmap=cm_bright, edgecolors='k') + +As we see the QCL fits sufficiently to the data. We can increase the predictive force by increasing number of qubits and depth of quantum circuit. + +.. image:: qcl/qcl_classification.png + :align: center + +Links and Further Reading +------------------------- + +This concludes our brief introduction to QCL. There are many areas of machine learning in which QCL can used, thus if you have ideas how to expand the algorithm feel free to make suggestions. + +Link to the original paper: + +- `Quantum Circuit Learning `_ + +Source Code Docs +---------------- + +Here you can find documentation for the different submodules in pyQCL. + +grove.pyqcl.qcl +~~~~~~~~~~~~~~~ + +.. automodule:: grove.pyqcl.qcl + :members: + :undoc-members: + :show-inheritance: + +grove.pyqcl.optimizer +~~~~~~~~~~~~~~~ + +.. automodule:: grove.pyqcl.optimizer + :members: + :undoc-members: + :show-inheritance: diff --git a/docs/qcl/qcl_classification.png b/docs/qcl/qcl_classification.png new file mode 100644 index 0000000000000000000000000000000000000000..012b619e4ffadd5836d53cf906da7e64a60089e1 GIT binary patch literal 23725 zcmeEubyQUU+wB>KlJ16q0RcrCq#2Y}Bm@;jq((v%L8N1bZiY}nq!c6sMGys~8x#pa z=^RQzx(B$Q`TpKocfIdjciq1)Ybnb)XU_Tbv!A{9W0bL>E-e)o6$C-F`g&T|A&3ME zL9hWza`4GJ<(F3A7pePYeKSh%<4=i>1phwcs(0HRg6M6Ce_#cud?yIvhxD~BnR%zL zjQjYT?Yurc-7~l>d_nlanKOM*5P$YVE}_snWKry)@67Ig9HU~{a42z=&w&LOwJtV8Lf~Qj_Wttv-1y-%R)WTjg4Hx1+neElu88`QfL)V_zEXi!@PDQ5;H7geJW4wD8?bFe7>w zu9I48YQlAJd~vK|H$y)kPt^KJ2OK^%NRvBL?YY|YzG5PJxul-&aqZ3vuUGUa0>j`v zKk@mVEL@NJ5sj>US3@kP?D|}a71xdocwS`a6j&Ocd_|Dcu8QkFuS(jq#d2P%sKUpq ze00O>)Th3ZcV>I=;K4_yeqpFVe(-KccsO~@$>H+Eqc$7@CvDeeJ@@T-TP;LfJTa{n zwQwC#2??FRz`Dr*(>#Ml(43`;YnLuvigVjmOV%Vp&;P^{!sss+JTNr}gmgaD4bsYw&|*h>NL- z3FEd~iiBk}7`OI>0AcWQ_tnXIghRSp_^)c4GiTwwqG4fSfjQxyKYw1FtVi*2E09^@ ztIaDN=D)fAf_<)x*BmoKCG4OM=uFEBk%f~96=#((>E zrT*x=m{{VoXQ3Y~tHZ~H)zs7$@n7#sRFe4HdB&{;yIiDdNc(b6aomGOjftsvf)xpA z;66nx9eR&n-CI2L?psnbn;g_my!vDF)Gpik6vnoy|B&eux!)<~Zb=>6UwueC0tUU; z7ul1RKG#{E|7C01;tIL-y?gf}Ous&ysNwYS@d+GdVP!qwWgiOUMwaDIXi|99F`m?E$&k2#I+=bh_j`o5^vpSuk9Bo?_wV1I!0r7VkFl)weC@S1K^!(JYHGKs z23WCWH5=rLk5ic0Xh*cKR!qiQ9WUJ)D!Wg8&wHBe;aFwg5l8*OkH@alVeB)nBlw!O z|MYK+x%W)>8=+d0jnthEmy`Q~=KHZf)%l-5D@VmAN17_38|>u=V&?5A!ngnZGq#hBTZqq+q|v=)`i^ zOi&9V%Q`A_akh=&gT{jckz(>wG|(sxq1(7e;kuC>REb$?jlAXRfj*jo55%Zrr*$)u z|EgkhAE7lZSS2bXAnKjH*Q6E$!(C@b*pe{%X(cpecVA|^D6|~8} ze|m9lgwv?YgUog7rNhkpu<{CvM?o=z@Im30rMYdf_q-Wds^Ug%KH1tuCr zCFyOwvosn%e}DxF8Ny8|?R9SEa@?Pd>95FD1_95*+N?mjOizwRtxSxZ|4)3(rLI`v|g>wyCIs{QjZK8WUw-bIF3xkAa&8rl48zCW0S1v`*I>EqL)OA*P&%{VO;$;MQ8eY!zAxB^? zRU2|*vBFwfF2uAbbjp#rt z!YtWBqzEl79|dLjnvEpDC+WPzPl_2`hNPEP-pXrtp)OXN6>{suq7FbCZCjq#9?m%O zk};bMRej~G5HyiFZ}6V(YFk!12Nq4m$_~E!X7Ki`^va6nZb3{WZf~{gvZX+cE+O96`R$D!v^F_+zLo>qh#`HI2%f z#pXdla0;ZfG-Fj&)&4#n7viGg_4lqKPdr`4rylju>L0`q@2@T`-4~l;Ymr|aeb*4mRiZzgwNm9PhV8F>|Q-%6<5{biRHyDJjYL z&Yh*>p=no+J>$Snx{;*ol>$BV z@yXm3aL6+j-70CDX^l~Nn3bL$J3EVBYbTzl@qB7a$G-Ft%|YASPwl)W=C$HcD(UUG zMiGf{zD{9jXQ`gOe1>WG$G30E&YnFhdf~$M#KKeM#H=jb%*@P3KD%G zkGFSrb~vQ0p)nVQZneF;go7T;?ki!JlBz1sA3uIf;6T7d`pfSkZ3%_yBkfbn)PguO z#=)WzFGkA3OZ!mqyLXo@EVzIEB=pccd+{Q?r2gb@Fx-buGDES%)NVjo12?82%}UfSaSq@JF}5$5Q;EO4?`4v z{K!B?M)ps`0a9@4ioa@*yKgE^7|}(9^xUOULtYPChovSZ1y@!on%%s4IBs?O_VAw# zzqCA?ouyyArpY7VLt5QZ@CYs%g@Pn@Re_j0#1br4$^_ zCy0w=5%7nEg@pJ;L@2biwI@teD@Nnqyg^>LTPZa==ra0+7R2~e6ecjgr~miwaLKxT zlK#AF{rBrvBRBFTC;QaHvrZJ#bUpc(Gm z>`$INSqD=-Q~aqr+WQnBy~{X6ernOxy{DBE@4sCVy|NG6Y)3RpUIfGcW?f0{T7U7x zncKg9{Sr`8;%rsu+9J31`}LMLV&cY`%WYZe9D#|`8$K-W%LK1M*cJ7QUzhrEf>>Vz z__B{urLk#)AfH9yclXd4pVJTT-)mpLo;2Mff;^`K-T$o~Z^kKrCbY&+v-9X~i85t= zT#jx2gPs3P_;O2~T#44;k`#39_icu63?Z@gL*T%%Su~NF*5P%PS$Q-5<#*@O1r;@KYB<9~4(*7=C+G_83VbWnof`J2jUDobW=bsODFCq~9 zfrMeFw3J2>gidB%?p?i^CEsH`Vr z3!T3(!jC3+7|yNRtw%*Aics8VPfB>gfxXDo9n54hgPV5XlNe)w!;RFC5<-klT&cya zP|T=3|1+#Y0+CD@nQ*Wb%>jyd^*SdcojeaA4Rt<2 zwq!jNc|3o`@))24vlj@J0>MnuJ3KTRWi-u_GJ#-lVz@MMXf{#uoG1;kTktKRUR!G@ z>1=T#5*`DvSbK5%7|LGJS1Z+$1M$Wbr1$E$H1s^X~aDNtm&%rdiWe91?)9SvuZ1wg0otu1XB2Ngz#5vWP zkFuB6lk0D&;fmzqz*5DyEB946Bh{U0c)UfRl_!d>COE#YCFye{PWW#-ctqfv?WkjB)ntO&Yy5 zx%5T@?4G2NT9xb7%a{X&%=uQ46Bc4%kiQ(p2k%i`&5yhvrE!ldzNr<;P4(6}=@Pz6 zkS2&yHQM4{i_rC8s9z$bxK!zS139oPvPB=3o#k0|jyVG7ZB{zpJJ*;l;o+JyZrCp^ zP;SAn&(f{keY<^5G2<2K?b$*3iF4!6q)en}G@_$3@&oV3qr$VKM<@_>MwYP}*U8`s zZ|vI(t!{(0mygoP3XmqzC;LfgyHjdQMoz~Q+0vplJ3l{z$D!!HuKs>+0cZ&2vGeE8 z6TVo~2dahBa)*OEVtJI99@lJy_epA=vB#Z6$?S;P5uWopZMJ1K|FI^tW)~EHTl+0P z@yAQhZ^M!Y$<=ekTCjCzNvsPo&%%2y&{^%2uq7%g+A!`l8JD5rWxKuD2eL#eI$o*( z=_LT0cvQW6{gy}f%ruC=r*e11SsHWV@4q|jkSc9Q4x)SzKsrAcQ*uaUbx)x-t6bIT zg(&rKtEc`U^iT+n5Xlpym6148G1N?tiY23Fv!1B&QCu$mL_|(cmw(d&z(IH55pcAt z?d$t>Z~kefZ`Jn{$@YOl(~auMV>%*eK%A^o5NRN9!bW>spwjhE52hoYqoeD0r*#wU zwoQaMf`r7TF*%>h%d>NHNe>PVCXUh#MK-s#Ozz(0*>rA?h$P*n8k8tEPRLkF_kP4rXI6RsOI{%G>TTNGR9-Yzk~$c^XJb4 z_io+1i3F_5N;{{53(o}RG+?d$&!0bxsJ!oU!Qbboj+AcLMf9Qiz?x%M&&&%e=D$L^ znbt7CIgJtgFZk#Pz|83A=!BFM_FeqS$Y<)~!=0_b^puo^y;SI<&(?e-U%y1L`F`X6 za^%LKlHqC+rCBPS@Xi?&2_*zzU8<}DU0XbFj2phqX1MIWqoZR(jn5X8o}Q-WItBgy zU`Y)Tp?O(ZnJZXHIfaF1jMR2mzC0Wbu9yrQJxcvQQYM|wEYB_T0y!~HW<(KD-xNsR zzM>PhNI*g*Rr6dt>i!&F;~wl_BBboQI3A9acUDK@ry4^n!9>k+77WGN`q@{IA@KG7JEu7_-RC5_ME`gA=75B02_zfDEv1?k_T(V)`%RXF+kS>xq!gmuaO7A;B# zF&y`iu_O7?C>1<`kg9e_kt(|*wpoC%&I0{q%n*7=n)GO0LCWWL>CzxgENYodSxGi= zd?U0EeKsCSnk>i~-f-An%<9;1NLk;q-e&X=+S0QIe4HLBQkj(|Dg@?ecQZ6gvMuW? z2$q6WS&57t&l^B0+8bU>hcaaklsYkJT^Edw?-C3W5+ToyGv^J&l9|>Ao5-JV1ZtJy zcS1T1Xf(JP1PK1a9!s}ciLLt6<9=+d!_OH7E0F4I>ghJEf7gm=#&wuyQj&Jn21?x> z_m**d1>#o6N`F;oGq`>pT<_t#H(ZLEU6AbgA?IYY64y?aln5d!qcnK3+tRJ);K_yO z%og8n>qRat2iwT$=;Hnv(omSFQeU1AymWcs@o=JWIO4X%NC)-Vw2~NME?Br=L!yQZ zIo$ZVOen&7bD)HOBfqHt(QKry6PlU%W;o*AAzS>#lSoEAUpwfE&WrV}`}z12G!8_A zAuXJ0t@crxnOb-vxNHkux!kt?X>_2ZiwG-X#xEN^etiFc8 zj{rZ+`0i*Uv3a%W5-(`sy~ua1SrJP)m`o?DKIxO~#`RC^Mx;!`FnBLXU4I=5Sq$!& zIba*N!LAaXRoi)yN{@w>JX_n0`&icFa1bdIdm?>e+q^CqPm*N0ZrOxBk2@(;Jr-+@ zwraE522(r9jX9^r#DQV?B_Da8zZp@6ri+K}XngKNnch}B7V{+@!FcXKE$r7EEL}c4 z&Ek`tkBYcWW)Y4O*3v=sCkbbBVsIuuU zmX^i79O?VtSa_75icam0Ry=NRWw(k|nDB-J;vrSeiAkb%`#LeI2abXdronkRF*55#Yv0Xw48=Xcg9<~=w z_Ewgq#G=33!7Rsn(_}a--|d>$il7;$C$HVPbI17l^6{u8qxKxyY>mM771qXAzg@YM*+2KB_DDvVVh13bj zhQZ#G7F<_@iN^J)^)=BOH*PpOJ8Ry#lQwzm=zf@GRJE8D& z(YW|tEmuj+Ce_IX4%4veY1b5bX8+`9EkwQh=i94^>FF!A_wC71@{rLHlZsd#Mw{cFyHl5L;sG6XdEX8)y?&j6oju|o zZADB>Ohqklt!g2&d1ufHx3fHWbWMN)w2nR9y<16COpN+#+`n*ubN#B+VgAT=Nk@J_ zsY>Q|oR&2hd3}H{#DxBnCiyQgV0$?;VD~nW^a5(xg*>CIi9cKApFdB>NY-fpKx8{& zr4oDcvT@~)RbqbNs_J1oBZ}$T6&z`yE{xYOBB6e*!@mg!k!W1H1e@=A83Oby+p$VF zA_(B(;;JbMrv6VorJ>VlQptVZ8gO-hH`(D68~W^g3j|j<^tX2JUMvE|H#|ICP*~Uq zr&q-S`NV!Lpzl701#@#V3?{+P7*}H#9BGHdi zBlymo`EvWfd3$cMroUb|4vO|&$r{%=^-f20=Dw@bu7n<{PPfgjJTt85ALA-J3U!++ zn#bE@P<&in6p&k8^{)oxtYS??UH__%>X)*ufSAOE?M6<$yg4%9!b4 z5pK(JT)B7%f-EOEc>vBBxbzMC_vTt~p;_7qnfFT8bH(wiT4-9on(wI%bU7G%$7`Hj zg;w0Hh+Df!U1%#tgI#-^7uU})C(z8Pfs)Uv8@jn9I{ZKsCfwDt;+6ecgR<+V)j(=&yf>8}SZqrl@4-k8F zbNK?-k}nMSQ6MeP+S9%WJY;8Z-ZuJ0d|lr;(6v6mgZPpg63;D)0tEo@1eL>36hiRN z6Je1gQ;S)p92l)2FacU@k9X)2D$!XdbBkFU==geO@oW2klaXf6UO-&EULB6;xf6=7 zm0$1FI_F}a@trAn_qfzoAu`w&O8QK5@A!Kk3g5wl(XBh;TDx${n-gsUjzw)uZsaY`mG-%+mp9kQO(>RVv>Eg@p7)_%;!FMvnbaJwKXUwl zpq!fQ9m~4xj-i>dD3^Yi#MXaW9DqO*r5`GRBWbK6Ti>J-&MJxM*Y%!N2xu zRAH?)`bC?L8~!!(0fnQqKNseg@SpxmzLJNHFVurhwqf4;bwn5);9-JcKEk{ z6?+k`@c`Q+tm$qQx27J@voelE$~@;<)6+tqzYS8vDXpur*a?9ziy>cr+kf(mi0ohj zvI$ds^(rZsGcJ41yY??GLx!K5ZovW?gQ*uaPW!_H4jq zluy>4^B&Ri9y>pBX-Rqfp>=SrF8D?G14v`!r4onyu`)V6qv*RkO~zBQx^Xu+HUc7i z7wK8fTnG2o}vJ&sS;@;SWS4_pk^}K#NeFrLm%)>M%TkN(3 zD`sBpSq;wl9NOcWI-K00F}esl8SD+{gj`Vqw(bm`|JJ2cUrXqGJanBRFC7{QPr)Q- z^!1xtJvyX_Ul9w*djxmQ+zY(+irN~cet2};!E?+kl=)k1enELJ+A7Cup27d%_m<-3 z_zr}<=RngweCnn^9+yQGp;=4@JSG4&CM9WEz>2AI@-Xrh+;1(c!tyLBCihi3&rGRT zj`w7*gv4=3@1Q(1nk^sw$=%kti}+$9QE|?b$X|F<+aIeKWLSCMAR<=RZh1 zW^&C%?2EI%XWIO-5uTAmG1E*-j#^ezC{ zHf&gy=QqyD9y^DZ3)s0%pL$8oFx?}Qg5oQsT@@*y)bMA;1_pI=fgifhB7dH_1!Fcb zwl}h%2go(1$2DXR6Cit7E9Dfo_)=YFBB16TvAV3sqT9Gy0JJqI5jN# zD%n2pL2EQChoUP#6Ekz-t5-2N)QRHku0@4QD+s*QDc{3ZRvs4L{*>2zae1(${i98D zLTYMcqM+_foWg{Z<66{gbI9)ymA5S=M|&(3N?>i-jeqs-!ZRO`2A z3#=qWeCN435(WSy>!1}zZNhh1eC;AJlzph(z0;M2LSzN(auT^5PFee@?~?U$%lFT; zFN$G@qRzLUzEnFDiCHz8h}%rEg^IGBZuE`>LhGz)IGko*PDRdxwff=-?KcTLh7k<6 zt{v8wc;bWhV%#)eaUa3_*F5nQ72FkBia~yBwd|OK?^fX6`$}?WHkB0=*cURq`1mjc zHwX({FZTArE_Da?7w*+mHt%|kxlvqn8Czs`|78E;^0jNU7axs~LY4Dr&b__8*g-3l zM?fD?QS5;JVP5s*zWLFOTel*B?zVzCR2T9{J|&>v-6b&$)#g7>nti(nbKiZox@6-J zu|VR@)==!3^(d&>ht3f3GPqM=^6^P`V(+^p3s)bPyy>2P@20rU;d2=-`Bn1hy+5-; zi$B`?-aatO$Nl;zRrjp3{8y?@4=GN2+AU3<1jTT3Z|p&%Y>@e{T9v?pVlChlQ3+Vl z^PZ@8c6J6{m)vE$@0a(z3SPWmd-m)ZKk9UVmE_+EDjx55Z!0T@9{WK=Z^q>A^uzvp z?(VWnOG^ZocT$JjlJ<-OCuR-%)Glixuclq|;Faea9GJ7KPN^p`C+gtL`Y|DoEcEyM zNb)=`uJUZh!ef~6VDA)fr9i+L1opqdwfpiw3xZXY0B~YwXV>v4SWOtZ0Cc_1&dv!7 z`@9WICvff5Yh^e)6;tqVLNmb+`w+cjihWPVy2g=7-KFzvkj5#0)?Q8X?}35TGL$Fa zpQ3Yt-UW20iP4ke2RuS0-xjk(i9v4$*PC=`Px5yhiYFhQfwzcclHb}x%X-D=DZ_m; z37iM~9SbPJ2IkYJPuGSK)r=>W=g*!cb8~YmDBiBualC*3?9cvwc;Gc(U*Cq+Cu{0X+fwN&P!!TXRkwKzzibTxQ$ft?I`toHGAdVTEna$Srta|2v z1OPaNVD-Y1l09HQ*GRjTL2dvKl-?yxBl=~$;bQO*=p9CsD~=ua`vP#TdyXk zny4jPH0jks|MchMi8(-iR$#P9mjBPBLC3TGI;RHau!o=Dl$|f~60ipu?btfOIU#j` z`_zBcX@IYW2bzjCYRTSU7j&4%2$WLfW1^a*RprT?OVWdMW4xx?0*-3!mXa}trVqh# z!MEcoo}QWs|Id;nWBb2JlH=+i@8DK&X?avJ9zORy1s^Nh-C$~@76$lN?~0dz_XOm2 z#M!Garn*(uS`q{_*EhdQ+O>iHc$GtDoSSQsn>I(x^%z+UWvt07)$- zCIy9lILv$(JdpHL5LU7>^!Qt$HTsGD6@+TwA2~B#RQ$VeeCqmoPDe+_N8gHPBCS4< z_SqvJ=!fdYC(Z?kMF6^g4?;E-TFF&#A9=d84q9BCTHKpNN;HNA*-196cizI65A=(V zDl9W=((rjzeH5Pdg-Sf`^ZsA_54M$z<8peGPIlbztXBnZ=aGxHw29NU(KGxh`fSsp zm9rf@o_8cB8OEW<#?$TY>e$0>|M$h}e)wd=v%eFs7TnQBs&Of4Ks#x+I~8RpV#g?^ zXIg2C3w(L7I~>2wB1-U=CW#top3J+L`;)lc)lZB_Et#rc_h1=a6;Bw^F+H!V>H(Rx z^)z52eCdjRHD28Rs85(jg%I)qCmzz>)Wa@ljL0-?WVjtygpGQN#BX4b>4&X$EoTR9 zN57qtuj@iFZcBsx>Wb<08iSenIj2g@9zGvl;M>=yM~*(0psBN~Fg1fgvO1?MsF!{s zOh(Iv-?t*aDa>5fw(^6Ue#KY6Bd@7{B+^~Kepk>lF46!wslulHqigHLXMKo~YOtX+ zS^BVS8AjlvThM^qVFqw1j;Q<_(=!%`fw{x4kKT)WAH5zmuKPsV%ST~Mo6XjcthnCl z^3Zq6{{Ag)U6~nn8eK314U8HxnY0ox5kbcM*KKz_)nwWUEC8Zi-Z|;%J=+)tQ>w5r zo;dr}@;<)WD8>~%nWE@(JmIFERmGR)r}_ZH5u%2u3$jVJU)uCXMtZhLa>SD|b>d)- z%r`+o-MYluAcHs+`LqjV~p9pPsW0ZSz7uypLN%7uat1r}omUsi} zID)jE*g;HxG``?ZwLpIJeIw^rZ}}L5MS2|mb5`0Hl#Y}P!T-RJLY5?I7CBhWrX7|a z_*#Q_tLC2iIV?V_QNfkKJnomDR-vgJv%^vENMrD)+m0kks2M6ha|4_TNcma9AM6+2 zhFvI299FI%Jxg%1U%T0*O0C)j=*dhqf$j(|1X-QCQh2*NOO`g}C--lj_j`#uj!w|(r~6ne(eq=LJCXK} zJXZ%EaG>B067uX|#Y0`K#(f=YIf0IS!Rv6iv4AJnvEZ9UVh^O_$Yo9 zjOmOKJO_$DF)0T)DdDGJwF^QsXCpnpOfW)zV`84;Q_$EzhwaG6{YQN_GUUGA&@R18<62hnZVi#tnaR z5IlG8jGEtW3*lhP#L5cs^yv5Rn`3V6yhPXcSFg4B*jSpoKpq0Qm6qT=GM_fsr?+l&I`a_S#a^muRb@$6LrV5?>d?#&Mx zuJuy@*fi8g{eaZ2J)R6w-5s(^z4(ZW$R486=%@Q>0G0yh@$gz*I|9RlZ;R9P@liV5 zTU)EG2Pp%Q3f7Z#fTJhg3)=e0o@0$(f- z9fj}i${(*&*&fx9XU|w-IiyK|om(-G;N8*g@ZX_+zk<9zzbStRm};~6y)bHH8I z1srRNsH^jVbU+8VpUiQ60|TKde=Ffc13a+4)$Ep3?wE36<|cX zx_bIYMxu*K>bTPKrb{r#MUxBuI6)pTb22hAu6^3QSneD4*+`6^F@~jy589^;k3FYX-mCb(k1Zy0umt6MO>_L#ottE-Ol?=%uM&bQ`TDo4!ZY<{D z4j-A`xf2Z-WC(yrigmzalR$v~#8-?1Q6L6L)<^*X*jmjN-3xJxa8SH}wzszh+$`{g zM=433ze3!eNTeDi0t?nn+gFd}%Pu)iX?v&iJc_+*cnuj@+0sX&7cF;tX#UyArZ)np z*|7-;p_wFkro>|j5EnKOReTNx&32DDHL^n;xTvNU5u9~mm)@rwvhLGvdLCMwDa&c?_^mJg6HuqTtX{WG*OB>z!&Lvp2ytE&$sGY$wZRaP3t z#A}q8vkWR5JbUf?$)@--3SZm4pDNq{?fGYpxwWo?@FS#rF!`RPaX%)0o-SY*^~s>y z5audFa^KU(MHqx}+mts*$LiAW3o)jS#C}fv?@v#^byd0;(2!n z9)%-tIYfo8!iBR(qS+A(w6TnOESyC%B}AhON~_bHMk7YZp7y51iQan?4J^2L=60Z= zT!ru$4b@8!*I|?Ec7#^pYNIz&?J!@^4ikG825VBLun^OFA#NZ#DK0r6Vl|L{CK4xU zxGdG;h-WR393#F*-Ru<=dFAC)QI5>bFf{n=48F(w%{91RfgIIhBWsgZ{`vwI-3rO2 zI+Rl2_GIG=C}fKi!KHIHNWMK?fD5=Ha{_Zmf*Gv96as zI9Y3qTXh4eQH-&E;eDEiqlz!(HNPrzt&21o@FAs>V&syXRQ9f!R0E~Z(e!|1Ar@93 z$hW5ZxFkSdw>qL3afV-kDB6lSKBUCIAbu zvL25VC4}#J@aFWoX~6lRkes00^MOXmSHLA3f=Q(rAKA7wY0^71;#hFg8d~E2kgulW zDeCFTNk+;Ml*`7N7Z~LajO*uy1DL|p=z2Bh+~Hr^DTP=gPmV{un;sRRB=ix$!5mMv zK=SO2V1l0i7qz*oTN}v>PtV>5+${s4l`(DnP14KN8QjAlnBEnp0Is2Ci5x5+ z53SJ4^`mEIZIIAhWS}w!eRUj(uM~+s`FX?RZ<`U)FM;rm7ElqV+-Ri~;w)gn@RbpQ z4B7xmSB5@x9{kJL!61~l{!9{00rE;v%VXb@83%Oy?UiBXt%V-lfPjFGl9wu862y4* zXlbA@a8yoSemd`!%jGQffZiNk`mS^(X5i(}sbi5zG{DS_c%0`@{H)mG6+~&5P%JdM zm&JSPEffIu`sB z->Q7ERXyM6mq);gL%@oCsB9`oxioOD`V_er%HLpesVuy>{do8Qrx$&%=9db?x4+}n zrJ%UUf3s7B=x~LmO56FG%bohy!HEXyv;-FiLFG&=5L1oq7 z>NWL;z$~0{_sN-&k&znQtAqq>rkdYjfF;lcKg(~+4ge>$JAV5jnq z*sFgu9EELZ0-EnCiR;EpD=(W<2W*!*1aET?Nrw(=T|`M?}q0)im~DJdx}{(MUf zY(`S<^9GQHlhe6x-@hjU%TcQsvahp@qlKM4G&CYcNA!*ij!zdT=vAL1<72)Ya=6H3 z_O40%-^E#IX?gh{CmAAOo3ZZbuL>ysng#@euBYckP#Hi1=@}Zf5iusv$^m|1C(7Xp zkBP4JN)q!+W#Z&8MOGeK39i?A9sXlkA}9NP!L|ZVA(93n00Z#IW^=Zai1)ftr~ax3 zv&n&@z+V3bXmb!zl_g@S6iZLeZq+RVBF_r??7ddIN634O@R?;KtT>>WL9ljfBxC0X z=mx02Qahg$eaeo6u6SI?roe4P(Xq!FE23jjpdzXD+iRyZ|F!7gf`18%@4SHU)9-6u zt8)3lHBSG{Q*&<#4IJ=d|GWm@GS>*GDZ>v0iyP7>^Xi@=(@0#@`F2@ch1#L;YAWv* zJF#Zp8uS04DAD^{QITsG(4vX)f{AV{u1`FvbYUJ1e{41rlulGZH1`E&Vtp)Q0m6`e z6qITKGt-ZK!Eq_5^Cp^?Oz2peboKRdByg?h=8C*7Av2&V-A`*+?ex9F0jUZbJFNDc z8{)%pvRD;(r^xU7yZw^n3<3#m`x5v1O3`DR!TKxHR~Cb>3TNO5S+ODgPW}c@3rF0a zNJsKBE-EpMxCc9OUhdtBsRw4p$3j@?f)b5V22mwARhA4(gUY2elXbF;8KX z!*Jjby;ygo{-JuV-fd;yEjAjb%PXh4yS=68HN;R;{(#faXl?W$q(!fsl602tQnnop z3C(3e(irMVW}}FZQg5c_5AMEUR^?iEYINIdp!0t}m0WTA6<6Ona8ZGeBEe^dR1O-2 z5f&6CXMQSCZ)~)AMB)@qEH1Tx$I)EM#&g4UDSogzrH69q$A*d8bbgG;ZjL7Lzf&8SFHMkHDj?oXoy9KmP7e-_ zhlk_i@>Y{26LsORQ-*DWoDPm)NSETAG7akZrH{XrTz|dlNss!zg_RB9XgMxU(Xc3y z=w4OB+D0%t&~l>x#viPolrip0_JGr4f@5GTjj4SA;G2yJZ1UqPncTp$wYTtn&!NXr z=h|r!fdd;|J)Lq3e+oT()_V?h~)6=d&r4JZ%a|0ol4jKJ)09i)vXe1UFoB*m0r4+|8DJo)-?E zF0_xdb=WDqi#&mUK2XfZJPHS{?$A&PZ;zcPsx?cTs#UvW#p?c>LnT?U>fDmYz7i4{ z{z0vNi$>dcl7K?KV8}>bljU4db5BZQfBhp}dd}S;G#z<1ZROoFqJ$EkN_?AA!MhJZ z-=AY=iCfX)QYT9gR_$_cNQ{0Y3G+EZSCeT&2e7RR0^dr2sjo#Xw@sor3Cu4nn*u!9 zQ^a;(TlVVz*uo#%$^#r#oL(P8$D4*7)l3Ft)4WZAVZWtPA6QPJ4I$i@Suqf8c?Ra#~&MH>3 zEc(Lqg4mlslcMpbcq(goDM2XDk(RJA^Ez2CY7 zRR9n9^>y=-bt0M=Xb|8or2D?xVG0tI<~bEgaZyT<+O;YQsqNqUaLvAESMr}$nVuzq zg=aHzL=14=Xp%f%LwNNbU^UX#C)B8ZKLWml&)!C{ z4i$f)?wiEH0qyF(+gEYGEhpLU$pteK??3dZXMsAJ(HzR zZ<{W{AG);kpbJnI!0AD(sv=gaA08eiSm9k2NhbKbr(kOrnMDj&)(^NJ*Yqu>2ws>> z_Pe+`9s)x1L$9VDi5q@~WHwuFqX?1Yagn|?@^pXZ&05ApH-F6Uk<$x0aVeM_1TgI? z`t3d>8e(24Jl1h`7MuU^DuUak61M#NJ@uhH1wG&P(f$vGNg@{`4Ptfu5j(Jb0Yz`{ z$gRR0JmV&?$M%4jUPudA7qR6GL{XV15D_sZB`!(D+Y21=Ef)emNw zV|i8Bp!Fd;oXtqNllOWfMPgzi^H=L%-vLc?WG4 zD7U9dgBztOqqdvR-IICMYuW%%9T=2+g%gM$g+hQGrsBB*j9Z|>yc85agLqrtypZWL z&zYN>YYp@SP+!-wfq$J(NPG3l+xNr<5&zwI(*0t^}TcsplG+P#{>8@13J| zf)m63veSno?$vMrcEPISk>}dP`|s}Wf!_>3d>z|2r}V>sqnbKC71WK_($OK8__~N( z8U92OLP76xDg|Nxi&I?F-ab#_$SF)uXNMz**UeW^W}5UhMRS6gt;aSanJswBopMU| zg>KraAK8~e5m;c#C3Ky0!t!z!@sjA9$z2%y$>kEzG!B|WITVPme<1^>#2OpT1fFcN44?eXR(@0 z86W%F`|FKWE+gQe2+q|20CQ@#3k~0%i}eAzDU6!k_IL5^Krw6tsEv4gHDzW2by~Go z2MmD3yoX+G+c=_q{F4JcQEqbT&wFi@rM^$t>Tw%=R52Y)Eztn#9^Z@%0%!EkW7pOv zsu2sJeUPRULp&8%&L@Qd0IV*d+7*RzT%#4r{|(C@_;vbUrr>vd^7e027_~G2)gUz? zrf)d)!-C0+U>FPDzu&1Q8uCK&oQTB}DysHCYMk&{;Jsk+g@%@vwxg?y$k&vjJChbY zua_x!)Y8rGWH9Yo(|EGDn)RljCsD}dhuc!uV|5-#4pEl(_Oauatkk+m64QQ#Fp6>j z|GBDy<})A|CxFt^Z^x4Y=g#RzGivnEmzfk*F?}LIpw39q!8f>hX+qr`3mxA`^Uz$C z4s(Bpe61;&fWS+>_*Jj5vc{PhOA_z>@V-G0E8V|U;3p-~VAL7G;UG%ghENVzF|NP3 z$`$dt?s0HD$Du7h*S7)%znmtSJ4mv^=L<2pN3>JbuCfn+4mME~UDf(t5c<@*O4 z%BLq+Rf;vXTg_R*9bXGvNr>KEL&`VKlmOS49vO~|c-M{ z*EiF?u-{!$Cs9m0_}qJO2b~-HBDm{WcnxSTm9B5 z)p9hB;*?og92f!7h9*q3Vg}PpoE=tcG~yYm0@`|-s10@ByR+K@Gg69=otHfe%cz$c z;`)*O_kOQ2y`+Zm=G_D4ww`zEyQU_7RJOji+z)fu80!gw!CkU z6){vZ#$jR^MkMAh5D1rVUR8Xpb#p@1w;jP2&fIY*c@!nw*GufI)4pH$_9W(G{>~-` zV)9f~Br55Ewklelag3GaK&hld_&k5yXE0nKyL7Z=U@1y;t^rFG^9EWwihXD2OU+BI z%2f)J3qs_k1M_ILU<%6ZKa9<<^xcbFHsa|ta=yb%wSDKAj>Aj8tX-M-Kdl^8N~mz0 zWk=ySJu2Gpx6gW9sI#sLD|x2FTBt)^xH#w>DdIHf=#*(CLGp?ud^$Jk^1;zAamp$E z;Nxx6qyCN^s?e2@;2!!(3?h0d?dcZpv}NMQ!caSbxMff=l(`&85@v0OF|6^QaL=>l zOHK53xTQF%5`t;5c{Z{L_Rw{*3uL9In`dU)&1z%O)C=4E{4alo!rV4-Mk0NNz!AX4 zfNRvJW6(9A*nItkHQTBApd4vB>v}v?vawN(aaM9yObx5*Y*Y*O zrY&AgiPrAvLdW`wVe<#lLz8({x7N=!YuheW?s)~Wqv-;&w#udYfU z$Z7+D#o$Dk z#4xLg4Q6gSqWL+a%a(rn1-~;;O`LwQm1}`8zI6+s{MdGPU|^s=vjAF0DE5HYar&ew zL(|zmI{Ndlnlu}_VKTwwmHET@Jp!S--$=h{6h63h7=lp7H}>^%7w1m=JdQZ%8>LV# z5RcVVUEZ5l&8nb^OxiGV>g)Z@6?d|8jVW|*U^#RfOSkykmUvXPJ<}>J)&1k`A|9VY zU=zwziPOBHV#*Twj9sEVczcZ9D{ZXK&lVa|!BqqQq<`F@uq}ray~LE`K`N?mqWr4N z8mI&k6m){}0LBEl|L}(e)+d1LIrj5)Whh*kl?|h$f4W?T!{NvfFf3dgD#_eF0?`3@ zX0_MpYHQPi71jZo3={|r)@L+qY|hcp(A1w9`u2^~7=!lCyV`g-3&;>6;4=|?+8XalVd+zH0fO3(Fz7UphHZTBk7Cot(gV`@d z)6wq|vj1a*xK(B;&Tb)Y@_0W!K0c9{y#;i!@-~;sf39YiJVZ^XeE&6i=>1-^u0uGA zvb4UQ2Ye3TM%BS41K!f(2(TPO$DWb6DFZWP5M2Ud5R=fHcI4@%)h{59g_3lkaKe#x=-E@s)<6hY^zoSL3?OnFX@2d%e32hY9b0^HQHyKOXure%D~!QM3%)%$C8 z6neA;?cybP@epNpKma8aW5<>e*K z0d|2zH(i@p>4bZZ`V7Cpc*0sPdiYxoPJ@H~6YXVtx>v2Y&qaNk%Ksjf?sI=2$y3!p zj>t}&Vi>Y?^t+u@H@5B&$+@=vy5WGqVcqNd*#5{}fW&i0-0Nbt0+C1nZGDao0u0jG zoXxDP^hlahCmF7T3B0~YG`Pvk=qYOGhC;U zfj&w0>!&@uit_WbVHf-0Xe}Zo6{jP}iXR$!cK6Tv#JXYA7cqgZ^B!Jq&dy%@JKifD z0a|6nEg0N}qMjdWo3GNxhN;0aD&>}+g0L20z?;!%x-#$sY@G=Gn*Ue|%4 zI{T?pPp1J+wpYk-+Knq{yE_Fb)qpoy!unL3Mp5zj{F2YrVOQBrP(4>;;m|G1%^ibo zges(V-`f)R{=KMi{w>!^vzm&Ec+dphF2MhqoXi^8nF!gB0X*8jGwI?e9^V%9=jT*k z-;E=vpW&|n_V#jg6vhBxLi!8_?7U)$T1pUUSp#03BNoq|KCS-nA;H|-d>6D~Q$QK0 zFwCFL4B69+%76E6^7Cg6gF_w$^(@2GxT+diSy?IO`yL)8!pUW2lIPG*AN-cp$Kpi3 zu0tpm;+Fg&NnMY=lKfg;F z3$q=-Bq8vxbSTsyJ*srPprDWj&v76-l=ADc$+Nakz)!uomTYEuXatTm#)5pj92`vT z^R;>C+?T*_F|rAAb&dWVh{ON|!F>jPJvTjF-IFoMjaJrr-&Hdk1r#cigh3eS0P(sB zr0gN-HQV*w_zW29Bx`GFg}dpW=_T(Tu@K^FCPlt0@EFCO31?;)4 zji+iB>+0%$0Ct5G2i+DRIXF1#hX)%L6rh~7RAppjRB0kU&0S>jHp2Dy8^E>M4_~WX zzy8@(R)06>?_t**f4&^x5kH+qKB1Pe_G-ZI?8wnGGR8x4jj^urieO=3aqQ1gOGr*u z&&bFK$*8Nck8=qOyb8mneh`H%&VGr{`R8hTpbn24cFlu+uD|-5^YXJhNRzO{S@uT^ zf~@OE&}TBobhNgf*ayx|{w>jD)Vp|F^6n|mK2VgY+S=IIyumFgzQzGHV-fN`mY!QS zs-2`o)6&p5+hC>d(374GZO-!Q>RVu>G<8pKH_k&3p$#BWR@*c7_%U}ZNY@|p1}>f} zA9iKq;u5W?-OJ5`aY`Uw3u;W7RVfM?ACu>D=;p~fqne?`SaPFd>53-gA6C#-b|D}yfL{@0s5=i zT>kx$esVRtjr?uk~^FupV+F;R`A$ApXk6Pv}{npf}j=_o1S z>1`Cm#Gd{1RahNwNF!-U<EFI_o~o=i8IZ^^L3O2n2NrEpTYOXD=_28 ze14jnTU(Wgk(Ut{*W)7UgM%A7*y&6! zoJtl`!w!s&cB!@kW$I5rqqHQ>LMUCq@A*lZ<3o77WX0CmM!&f188#j9tuni~dmImJDl!3Wh@|7eM>{bp({1=%8j|O!lU#A}OH(Zp0e zG_Tb$mcj++OV=snVjeyuTU%V5gkXFP5U`(*kEgmi6(b`ftt4gGLl$xa8huw&J-PGo_z^{Qb@c!^9@PWG3?{ajYT8;BO(7X6 zZw>xiN(=Dy7-GoU^71qUT7_}J$)twwf4gye{P>_5m1$~r7Y6*JV`JT*gLO|%IzdZ2 z1F!np+S>eS1z(N3PShUWNPoiF#}N_t0|IK^h3H{#Mu~`sOzrIjLbV?qAMb&*ZWn`Y zfka8x>mlXgJqC2hp~1mlkaWI6hc^cV3I=NGDWF!;N*WdxR0Zhy?=4~FrKNGJl5fp$ zQ&Tom>w?T1UFGRctE(qZ``OurTL;XYYy@9kIX=EeLnYR|?BW(DZWylt8{-Kr4UIoI&hT(@ ze(6PSr@!mJ=i8l_5-`yi%)`qozDFTF1?sKIL@|?cJ*;h|SvSaF>Vd@~Z%x%VVo_q% z@Rd<%=Pcx~%tWp8LIMJwHa6LvSIwc=R0aIm-mYtC(r@h5ew#;XM|O7nNv(?v*DkRC zJ}E(o2AOisPJ26APha05QL7C$SA5&ne%_;TH*&pk-yQxsJvH?!6ugwV3yL$k#qjs; zD-LC*;je(1YAI)>qeF)}AhhH>AYX{PQj?UFC7*g{yr{dQqo;fx+sg9l z;n0$gm%qOp2139+RtTS}r|2S&Kip{YEvDo?F1>l7=_?&rZzEV9JwTAbYDY)faR>ThDAK?NiS6DZjx zXOP@2G|*(|=6y%!*3R9#FVt4;t9?~(s&vW@r_Voq-!Gk~n(CK#GqN)x2(lZad_fyQ zXaf+0rhUgYcqLmp+5-MZ@1lg!-2wmk?zr&;{=M_IvXKjd?74>ip~+OputAWc2Lazw_2>cVFMG(VX zG<)F_*LMWKO9##)^a%2XjrJ&l1bo?xz;{nR`u|_~U(MJ)0%_a5=(^GG^P_bgZ?2v1 zv6t{Krh^Sq!q$*+NrM7(%0Op<^Jzj@YsF*8QOfk2YjyoY!) zuuIsVn6BeG`uNkxB&?O9UdezXFAtCBP#x|GyzRH%-8FI>U6CDU^r`ES;tH#<4d0yp zy;AfSk)u$d|gh2CIVU%9X+LDT&h3QbK=e!N*UZI?CMaPMKG%-isM;h!PYL z(PCj?A#}e@(a$qAq8otwrH@S75n$btpqHT@FXl8@A<)4k>#0fESTSfxkSnsRV^)h1 z!&ufn|9+VZJBI5ivra&TdhfqcIoQz*{p*4s}D+FpMRX4nW>y(kRQpj`_P#S@WUOCx%{SZ)4fGG zp`oE-{?tu@r62C)tcMQqM>Tv=gAGZ)bFljU7~^nSeM5tew|7~KVWK=XJSxfqBM!gl zrWNnI>v`L>uBz(u7DM=}-?7+lb!2--zD4y%_KY^85x>B|Ks{5_pV%4q@|@y3u)~ct zkLM$u;YI~k{AbSK%jXOfVd76`htZstkkEhl@S(gV{N;T1aJEIYUz+&aw{KZzJ=Pbe zol8vz$EUuhbPUPD=XdA8=TDtdlai8>x3ufb)iO5j3(}&#{K7u#ZgOh_n+Obxi1GRug}-_KVjjnSQ@PjYKyY)%eu_zb-`z4wu4Ey zBI#yBEFfQeQ$G<;># z^DfP6Ji-Z_$kzeo!yr3@KMaqpZ5O6J60J;L$&k+E(qPu78csQU*8oJM7(_YdXh(O zu3(-V>StLnOQLSl7(GBLy8!TF&579APpAvvj!+CBJ=^$gxCB!g$0GWVLYr`_7$4= zO84@-r_UC=9vnc3M-f`ZBW^@oG2!7s^H$kdHMR3fuGOjMLSuSndi&nImNzY`f9Ih{ zt6yh%H4iC%Nw;Ggx@`9tm==x96Fz?Y)ZtWv1aZn4cbWU(s!ixN@2xn|4zrGI-+=@S zf7+1bqJd*)$o${0{O*iF&N>asnt2SkJ$v#*)7Ccoy?u}NwV6zJG zxVmRl%H@!3t&~e;lLg zi``1P7-hM%o(&h%K_OkJ95Iqkv!Z@*9>tvoO?U3x$$s{BlKV`r-n8Gd zz`$|$a_j*p4p;Zh(agmcx$hJcu=#Xfwb%j~EyK&*I zmxKOW*h=a;aluoT(20w!S|{W%u87-pp1gSJl6TaD2M-!z_roQ*7<`cm?fk0s?BwKR zv0JT26Xy#H#e&#GlrR_!VX(3s&I;?1BRB@81G**o(`^|I?BIn8_gxZv^Qbutt4p5b9(DTRfFSBeMTT)V$a8p>~MYz*Yg z$jw!i@tD;$HSK3z*@Yl`ztU0}lX%o%HKAG-Yh;hC^>5rpv+~8`QELx@)3+E!!xG?_NaTjQ))qsis0vu+ysh0MF>v zY;Tp8YDov#_G8U)$y2N>3}0_srqr}ZzH!& z=c&}CeE+OZOI<|Cn>W2N+LitxK0Im>Vwbhe%@g1P-e2lyk(%u64T=og9fs2Z6{|l2 zpS|l@Os4v;hWW{23V4eccXyCoCf75h;C8IxcD~m3n^MXOby>!93smY)oa#_;aj`2E zgm=5(-6u_s21dL@72Y2E_r10-(d}CFH3*)#+_!X2`AKtlL7pytp;dFKjI!M8$OLbT zz}sgW`_HmIesJqSQ+j~{hj65rI{r?`7kOg3jF_i$;lcxL9i1ZR zxr}x$E-p9GS{J9Dp)s&9-VpKXRpyf?JKTPDYY7_U1)zdqHGt&zQu@G zK6>xa2M9+xOCwQU+H2uPS|98F{ZpTx?i)wiep731zl4HR%BwTN>~f z;ujFe?#D9!4LThm{vA4faTo){#07|Hm~j~9DIzg3AIc!<)qtY`RKSIW1%UwxetyiI zJ9h{J<;4}O6tYO-BNVqL%d_Y%b zvL)S(*r~Ri`EYBSh61jdGUF6gIRx;_W}-2=DIPy{v#e?1Ed z!Y%k|Yik$!Z~2jk-QC?PgOiKXZGcNE8osAo&hhnyZz{6P(VRYcQuW)nZ}s24%?HEe zjgO6;ICTnFztQv7^6`%!KPvLu@Oa;4ehk30EZ0P+i2&~8VV^3#+4(Fa;J8LY?n~~= z*(y;&S1_3GqAjv11=cOLi&Ni2{VL9Q{OlH~T<*0CO+{T&(R>`a(A5f z^3Q49H1wE3U$UCqa<`@T;`hrP>1rGR_Nr1S2)VfLjLe_TCoPU!I-uDsuXBjPvJ`I4HRPJqMGNp4XP#sCtyqO9CXw?{U~rY#+y z#}R4Q))#-CUj&+_vOdo^oQet4)zx+F!TwW8Uxvj~oEf3W?8rkBU|hnE3YJ^QR%(p z+NLIHvqhz_cB&@In+ZiVlphA9N7#o}lf5)t$fu7V!{g#|6Q`%Eo0^f}eFZi~=PMD*O;HRKCF4H;7Jq&5=lVJ^e+ibpVcJyysxI)} z&MGX_-N&ce`e5h2ictVq?LWQ*@?wUtEAaq^0eQMLiFcil zk}^WO$WoVuMUD$BbEd<&`NK7eUtAu*#K>3pZ~x$dz#M5-{+FpaJC{7@5Y2dKi{^f7 zH*pr#z&n?I_L#u}YMGkG0nG^rxCW||za&d8Fz{pC+)$;*ZlB#8V7}@F!|ojZ;E_|( z%^i{v!V~**Ib{CQ^UxjNG{+P9LndFPdhp(C z7?n!@@xyh%&EQeh=7I`}2WajhJCf8rM;;#3HD(vGj8rb0u(nIS(P$zdXqNM6`pxei1tXrB!Mvqc+>W%soZ{GAEe%hmuJ@XzpdXxnuy`4}jLJ0=iS{A&#X+<5( z2))=W?KaI(Mn=X$`<^qUrb0yyefsAgGvk(p%{_>b$->o}ifL&r%8YAYde1z3Kcy=P1Jh&lF?_H%3qqCKR;JNN0nq$nN{U~W zXZkV#q+BsRYV%XL)B8s8P3V_Y2y{uBJxjS{z?(}-F0fkiDa4xu~hIZthZ zP1XLwW6pOV59Rvs1J3LBCdr}PD(r>;&sdcIHF`z2Gqop< zAHNJoF%;lbzIkPMWF%i~lRmBvD)*B-q&Zenqsz=A{@Af&;W07UP%m|j6Juj@n&V{z zfs27`(+Q_rIMb-gxAc#F`&e06UI!nMu^q3E5O0?Y0Hjr9)5bUaX$bfXZg#*Y3wjmc z%V6rH{{*qCnp$LBWSaqVxK=Kg)ZAO{kuUw;DSxoi$8I)EgD@Q}y2Q0L&qe4>3k>Fz zBaoH}asQikw}(?U6Oc*{@VH(8N_=WDsj&5?(B|1`547sjqun}IR_X7ZhVstc-(~~A zb^S-6u&tdjLaU0wJoR3wI?o|F8yb##_by-0*jov2gky8SdHM6$rD zNuZee@-n9g^v}6N8~~eu7Kf}XLHn{>Um=ln!eU}<{WkiB^h<@K8gB9&ifYgscR=X# zK<4DYTv~5eo;cXx!6#PuHd@Rkw7QR;5C=8;qu86irgvq{A_I~U)PjvwzVA8@41ecBVz zsavRli4nJpgc@4OwC4>2AyOyf+1R{Terj^GdZK%Wu|j~}epc2j{E)2W*!pa_F|m#j zdhE8)zl4K>RpE^7r(mDx-@28Mig)hnv*+kYzL?YN!1yMh)1i-$xV*sq^kt!lm@W>e z!$3Nr1VBR&horu;@)JfTCTuMe6B8>3hsmr2NT1`KGKZN_F%=*43=s4`uya5ex1gX( zTABqW#-O=Z7DFOOeKTkrwknK$7Za0?v-3M= zXXgq_&XCI>oL*5;5%h=!EhNz!V_aYrq=D75MGRG-ii+Fdc`#u-luLMaOe~L@p0cu& z8z?nKMn--!ESZ6ib{$Hldi1}RnQ@mpF})8%02W-9BW3&X$kUGnW=6Jgp ziOSg_5Bdf%d$ybDs?pb*6Xb}bY)5GCAY86uYsjVcS?StloM<@!<9^-7N?@G$E(L3ddk^dWs4cShf3 zKYcaeuHryijnqM_eg z;kUK1eQA>{oSFNx`etL7z$uD8{-Cs*fZqLW1LH0ooQS4R39To?s=|(>>wvVryLA=4 z5OLv=TMR5pd)McrRqV*JJ{D%G;JUr0-g9e$5gNri=*vC!EbH^~RThj9b>sl*TFY)} zkN_Ntw;_?25qeE-;Nod7UfjYM7-)X~XaXI^ti#`KH==C&J^4#DSGKPzKEFO|RrmGl zrNtbJh~n9Luv}0#qn6EWw!h~c*F2(lv4j>n&Mu(YEG$dBO|2mzi|eIfjJvUy4jnpF zHW6c&J@X3f^?>Hl-o7;T{XA?A@F3735BY2{pG5+oXFWQ5n7Ju_<9X`CcRlMW#=O%^ z7-0b(9=(?@CqX6L((jvx69DdwB77(mT#MKKTy{~-@f-IinoKhNDBChqQ&W3`w<8~( z!^LWESXmKfiih<3-e2Zf3TxMr=(|8!?~=4>h*X22T`W36jQ1FX#<%sNj(j)uIFH*4 z*UD6dZ=-9PrKM8d8#Bc@zhS|yM%G~o=Oo+bBE6M9#emj8%6tv%+E(F`w>Q`L1U2d1 zyO0Jmr{WbnuDG|zkJ4+aAQ3S9aYIt_E|SS05vvA8IrF>aVOh@$0I76K21hZSQ5IDO zDRRK5hqL3VGsksYSL5ZM!!PrSJB!koAwA6_g-%YHhb zY2_ZYD@Y?y{ zG7r*;Y55-e-8Tx%cx^JNWQ9JvPmd0pVcjnJN6;g=4`UUP_Cq2fBF5#*+~~RCkg$It za;bRGI|mfXZT_%jXdc1^n@d5&bpsu8qkK2A!_b!=bkF~Cw-n`8pp}dHuHFXPdhy~# zeo;{!plDfds!~%gfQYUKM8vH_&wwAiMD9z^TS0-w61*3u{)}y!D0t2xnU|gcTwvG( zh!M(4fsfb#{aS}6vNT`EWe3%cwbOr#vV!U})YUo#TX1;lo168|ox3|*L2hQUEqM3N z6}Sj#VNzkMlrMu8#NNG1lmW$?P@hsjhb4k|uLFWF55{G(S+ClUB4*uu7>q`XH)C+-&q1V+n_A*$B*m4krNqEK?<&;n9-I6 z5M}lG!$*%$BLPaKUT#mLjLskq9EsK-jEfzp##Xa(w;fp1)B=pO?@o24^i118X|Jo_&VcXVzds8OuDWGrY#akRdN^o$zyL6K;Ot`{1X3xmH)w|#LeOhW)%7LdX4+qry`Shz1u z0rBn#k{q0hDurclj|`~Vd0~BCeE>QS;eDTD7@*}#mmUEumr9JecaIi$6$TFi*W!om zT*_cZhWZkKS?6=gd7l6w55$R%zce_Y80VnidjEV|XKM{X48-rozvq}j>qpMRzGVYD zaqgGet4aAUBfM?a`vv0s~gA~?BMikm>~4&zg6 z&oh$=;!-dl^4q|9n1LJza#MIh!t4p!`yXj?!QmTBlvJL2Z|}eEs^h zZF}a+ICblXeM>Wa2KTq`(Fcpnb-0E`Zhb1HY&2K~zM1jj#a^j+br6|BP_MTx2Z8&_ ziixRdE)*S-U2z{aS`_Ybj;q>~CQibkmqF5I)48YIgLISRKh^rWjNB~K@z(M()H-D@ zztz|B8$U(ICnn6bhTy)mp)`76l$A@8DR6#Q?_1;{>d|K|rd$fjE=&iH0aWddZ(+x= zwJItqZcqyttA`A}SFuYv=0fO#2&YBI-afZEQ6WD$`B-930Ni54uk-=xu&xyK&G6CF zA^}OGQ3xA2hEr2hft?^?WI?C`)eP0uBYA*kssp?R#Y#_p(7*zl=2T752faTu6^aH_ zbqxUC^{S%J_5KA5j^oyE|*Z;jIJ)A0%f>1>Z(Fj!^vaUl|zBVM1b)D1Ga zH3xdh%F2qi!{Q;o@&w-nOk$3Qh61I~!yjNP>N%T(C}bSZ0R>N?`?*N%kQ%)pko&J+ zm7(N7ll`6030NT^TRYg`m~@2v-5J`s8mpLF&z3S;7t$~~e*%66G)t-@_)qODrZl@w z*RrAJ$5Wh14oJJUonY*VJLN$`$;{1d{B73-AZ&`yMZB;6xPb0l$s@9vD?Hc}S(9%q zW0t>impM&of;;is(K<|XI=~lBWX#G(CT}hLAC_X8+Wwe)Wn-)7k(HG9;0ROtc%c4! z)7Kyi_)`c$tf$z4&onR6A}^6t#cKxMhU z4-{dyJ@5mjGqExT5pQi$9I|r!}RBRWA$0crJMC;m`*d z29D!n7{VnGJy%Q4F74i}mynuBRVFiIu;-QbkKDesxjOIn6TF3X2Dr$N+h@ZIuH9EM zey=NuIYsEuA%cP%mg+S6qtI$V8dhYjTn&!;*s97=8q@&lEwv6o%@R~Yfel*R?;N;o z7h6)RrusnyDRGvtd|v8#sDAVx;U8XzM3LN~v$D=nn=Zq4K_dU}#MICL0Q6{fdEXLn zkD)@L)}5$zvu;^Xla-){7a=R7LR`*!vE(YH%z( z0u6y!)&+M!n!sp80<{mkDj`B})5=PKoBNZsEHG#d+4%=6ZjhtOF7|?5i|lWuxeZ_h zjr^ch#cdjYF;vWB))a(N4VQ^VEni>X(mt^M*MVqIRyqo-HHz$e@YWR|&>U=sWp8(* zp|ITkj|!{6Ov;-#nu7YdnpRd548%dt(cR)8%S-m(zMrmcGT5Tb~pY(*>>xzI!fS9zBQR_T8FLT9*%1ld3k_!NmnwNCXF@3noraQ#0x3&!4}O z5KVDXWc+csv%QL&e;^rZvFSju4U7SHCD=-JCSMFh@$bb87gnE#9FghVXaU%w>+4$q z6g!P>J9DO9j{f+p9fYs`QU3Iml$44V;Y@0ppx0B4P6jNfXWaJgrV3080jwSQQ7DMg z016|j%2&P}Ps{`W2ufPzEeBAUcz)Rn+iK?KZa;kd*uoVJbKY$fra3JwZ45B$AP%as zj+WNT1u{7zF76}-g=gT7fISNlkdF_U;Kf|vmu7|T0M0`}PAm!Rgu&4l0OUypY(;?9 z3c`Ki&4vRG{UutEd}#+s8#{}rtE#q_ld6aXxW1WAFqXw3EyJ|zgSMr$u`skSLN*Qs zM7O|MDFY|NZ8S)-bF~8QZ<9ZXE@^Qzz6>B*+06UC_IwM40q2HO-qUGOzP6xMk2#n6 z)<&K*I%fGC!g=l=2y1uM-YYtm-WaNn9GvT7cr!e-ER6OZDI^ z(EhTHkHXLXH!=+m49|OlMTddvzgEtQ&;Yka+zdg31mKQ~Y>MIMJo(4)((%4 zpCNT3t1RGn5;9RlFMd0Tx;m9YaQpe{9d9LH);YhbGiT1AVU)GP3v{q z6)*(aeNu35kw?CxXnMaJODMG{Q+kf5W z2ah3~!TIxUd2w!uD!pU#x8{aF6b&2*Koo!d=0Z6Cx-=B&9((CUk_fj=O6uL$udmn- z!~j>_|LX#v5lvC5x6EQWBy?3U7{EaL*t1P|#O>QIagC3ikd^Hpa-f~N4?5o#Sz#;1 zaiA;t7flvuX%VTg=%z$sk3c{^9uG-+d(^pJ2cb%X6SK z(aaqWJSf}=Qj8Eg5+s6BKiKe#j}Zx%DB>?67jej#pW6B{)lj#Kv z55&l@CaWA*z?k_8#yRbOOF2*m{(xW;FVj-pxxZJthK61b0E#?tX-xokgJ>j-mzUSe z?w_FJ;7r3lPR?d-NOdT-l2wrB!NK~juC75p%OO_>NK_9jRLMD@*$#shMu2;wcpx8^ zV_xYEQ3@@fY&=;|JW+0MY-}8~2vmIU38=clUmz~w3LFJoOi+MsqV54t7IgSB2*B}D zJn8CkjS5`B3HfGKV1rM^tdq8=PT+0SZm_)qqXmV{_z=m5pl6{G8WgTWpXKx%rh&q( zx5j)GD%4e0Xu_akbmin%5wKYBg)yOjaYRVY65<=iAsf5!l^gYKYlDyE09Zy#BHA}s z4h4iRF1Bfa)PO?Czahik=-+9)CJ3prKu8rix;jTS+s;J?SU;d1&M#zz_=cNK{me&`-IVtyB5-70_hv(O^}M_tz!Z6J`fUcS3}nD2!kTAnsn@1RXl zBDQI0X7zUY02%{;kjkkg1_Qz7zGp6xuO!Luw@sLdk|k=VV0~eqMh!SNIeogrFvr+E z<1X&d+dkx=TR(E!kf6Y>@b*Uq?yU7wFO%Km_mMGi#d`Uhl+Q75p?cSR{7rE8Eo3NA zrZuRJwBp==kO7cr-pVYJpDzCEo1#v3R|h*|JRC;&(9~Pt2scdZzoh@qp>o9kIn>c} zmm+hD+qN^fpq);m3DsY&eWW+@nJGdSA5Hv%xEvE#&s*B=6%`(S3oei5&GX6#0ss;0 zpt*rqTcTl>6i@os{M8jAnwtM-jL zLxSKj8rOin4S61*F>Po@U8lfv{s&XD-GvJm)HFsz&FU-Qnhg|ERNM4o7Skdb%kLsY zsy6$rV$OBae=3FQ_pcFpw{Z`dq(k&+VE@HH!;c1<(eAtO?Klc#t-9Y_i*R%S=`8Uk zG>%%Uf@r`ho^~KdIzsdJC722ssA*)>3t10>qb~*xWGT80q$pooP`RxC_~Q>%ICeb{ z4nkA!5b7~eQy&jCB~%1Hd!`B@Eu%8mEVK}OYiJPV=jZ2>lIqD2deOWR%BzCLwv8Zo z+EZi~^{u}C45$O?P+R)f!I;K_L0Y4lAlvhEu(Gc)UZ(58g9oqBG^HJs&szYS8WXoR ztqXis9Le35A<=)C7@eJkLImwe3dEOU0GY*tY5B^ZLbR{~%#{v>zzRge!59N`uP>lw zCz1)S(sp)N^qjNE=GA_cFK+<)@SOlbX#%$srZ&q(NI+mAnh5(|4G3@J^FyK);va73 z8RVJTP>I#lh8xz_9GD4Uj<~|AyiLdL(zI9iq9XLm8z|Z(b!Y44Ja4iR1#FeG;J1zW z@GZE@anVljn8|R1`X>1u)6SD}paqNG9^fUYaC$u`P6}EX~`2^QBgs#p)%@r ztqlTeeR+eKOaA8(rc`o#Xy|o5)mOtVUrLQwPR!N3M}58hqZmAf)9M=Y)a22eJ`#`-MY$9+0sh*$*Edi2vf1hY^EO#CksTnq{GCvg z4R4%g^;+sMKp&ym%_-Y$y;-#}s{_Bq%Egu6qOfI(>foRQ03Xi5Z8f-+l4qgrY}><` z3;Cs~n!9vp4$iIah{p+GVGW?`pbCS9S$>r5%jeI!YeUv**}-Hig!Lfkr3ZdCd96I= zu5(rO+5`tYPXMs7)>;;@Bjol@d2m2Z%CQpAIp`Tl>q|rf!+kp@^~x%|w(xOMF0K$8 zN5$d6w~!dOfm2^LU&{d*7gQ35lAu=ZKEwE=+QLtY92^+fwz)>`NK-iobc#40WuYI! zDOc!4wgP*}Nc)SzzPq=p@)u3H6#Ptc^l^BABh#XsQ&C0=8`Nq^70upE}jF zQD#!?hzfTHP8ud1PxQre;mpyoXMxW!*fmh*)`n`q&}WHG{u{AeGC4>7Dd1k_nWrVQ zBY@^ceK zvx3NG3BR+XWA|k81!cDXvyUAkn;-SVd4dwgs}gyu@s)^imG2q@*F%c--x6pB{Vr~> zzqd?*GlIkD?`^StVXZ1uio>$-*7surV5`73%PEv-ZC)DBWUB7_5I{n53l0R;^?NTW(yz~tM#X5Gkg4DP66-o; zGX71e8$<ZG(&?6n03ci;L`c#35Q)L=dIE%%zD0qW*i%aZ!W;$EpZv%x_-i2|DjOBG1 z{QXdX#jMABfBt_A^W29akJrNsi!*&){Sel(f`d8%T_pnqu_lHP1{=r~ers-CKIBqr znhpeG0(_K;tBLS5f+Q+*pe;|zb1umXqzlv`X@cCIaf(an{wQe@)S6g@&5=EP5Jm=L zu{GDMyc1Hk@W5GHZJdlpb9obJEvlfeHiGNXi1thvAqJ$&j#9k|?Mbw#xlG1;#H6dm z#=xwu${tGw-<@rr=cErLap6&j{*&ev-Dqok4m!gHFr>I_L>zj>T;N`Oi2mNXpCK7$ zkMWheP3lD&J@yqn*`af!mZE^!$Xz0id+4#dQJdbZUZRWfK688{iLJ z1=*kp_`VuM%ArLY;9ByJ!Iv@;{dy}Pr)veey%pHT6F~FD+{*{hF`Dd~n%vNy+zcDq zoRH)b7cc6^Zu?XU+G$}=6}BAx^YR9Dy{kF~vg1C9%9;kCVIHA5t&9af7sVkHgq(Z{ zu)dyi%Js&{V|z0$g`&@N`PRa*y$T9FiEK3=1}NkeEP98SZTr^;j~~av^u@wscW09l z=sdH?RO$w)35;dORWLzMI_-fm3_E+VzWepHk zPJW;6rUgR-4Tv2xE=26gm#n3qSWA%c$g=@ubSD!N3SA~3?!A=90a43oc*;cs_$5(I zTRRke7pg(U@ZwLl$aY}jb4M1d3=zkqxf17T0B9=$+D=f@&cWt ze!${%95In|MCW^py%}|Y`3A_1Sp#;!B9Xb{RdoMPKOXyPYkSYGG3BgHo^cosg1M-E KAy4V*z5fAOK@6?{ literal 0 HcmV?d00001 diff --git a/docs/qcl/qcl_loss.png b/docs/qcl/qcl_loss.png new file mode 100644 index 0000000000000000000000000000000000000000..b89a917861c8bd8c6c02e40ecba216bd74e4fcba GIT binary patch literal 17990 zcmeIacT`mEmOgk00xAdwR8W$DNJa#ajG{ylBuW+)kc{LECNO{~5+oxzmy$DR2}%+u zLP-V@kQ@sn&pze7-M9PKcUI4=`Ey+BTi>aw6W+7my`P<)&2x<#iZllp4PlKQMGD5^>KL@hO|>+lL^GhXMlzm8`Q}@W$rYL=GltYkPGySG7t6 zk;7g=T#jD5X3>^t7QUyyRPyVdvn;Ggkq#OBqR0}lu)*6Zl*+EyvIAe9hPOScuirCI zi69ybq)D;Qf{WbX?0_cpY#On5?puAf^m_vqZ|~gs&kJg5CbR`LH|TC{@5G6>P{LcE zRzzIbu|cXfJxctBk%P=kcJ`LcatLyJAKa5c#%Ob1lKaf(?cMqlX^$!Q`y)u51i75p zuNmHA8_WFC`=34tBFKm1B<*VSv}7#O(|vBRguo!=cY1P4D+U$xZBe)p&pb))^b{+s z+ZEQWwTgh0EZL9>ZZCuc2`GdmYd=7w+^!2#<4ZtBB7Dq8k;%41Oi!L6@40hIOG`_A z<<6FATj}ZP(~*YuH*ei~cjd~J@Yk=CLQaWA_F7Ki(h7QQ%_EbNSS@Qq7+(g9BqC1p zaW2cLIfi5)^CO{D0Xe#Pnkp(U($y35gM+DEXL{7gC=YcljeX@E*u&DwcJY@EB(D#7 zt4~c$O%K=B+Sl&5&Gi-MwD^I>G&MEx=pSN|&}MPSP#`635l1)!AHvFuZ39Pl$%0{h z(}R_z6Roja$_4#%FP=Z|xEm^+-uRXapQ#WfU>qJEo@&*Zt}N`0ynXxDb#rw}qLf!q zQ2qV;_f4&>I!a1P@iW@#mg?N>@?Pu4QziX;g0x70p3~Y!|IspE+L3bS$?$}P{4Za= zgxSf+$f#Vu-nBN@Z=;R1FP&+LzF^&xs~_29rmOxc39+XYP$PoF+* zYHsE&ymZu^6y#v(xP^Eu|5B!>r{|WEs%lwVTy!+Iw8UpNthC8uwzqu-eb)y$`@sk^ z>tCLjnJXRcDKKxoSK8lQ>R{??TJJFv%y8itBcnk9ETwBst()#YZu>iPyg6zn+A^Wz zltb0~{;e=F|S^JSJ{KHA}_+Uu>3drrS)~a z_4&bK|Giz6_5IY;+P81tZauZYO4OzcAhFwWasggzvspz&MPYVsmBv<9RKZuKHKxF~f{_uQcXqHgk*`_d+Hq@||bU;W)N-JKJa zU%h6~8ht@=-@bi}l5V*i&iB`TccggD72`Rgu{9}*p*;iT&h#uSrcQac9UVVhk(cLG zZj;%{g{@8jHJVzT>WmS!i3Al&eK6KgKB1-?z;+ujhno2?@-{5Y<>#0EYOxp9dV70c zynNXiNOMZXp1QHA2{#I+xn^2)|BZlASrX`~&Gw%q&VI#E=Dal3D3!1q@e*l4%w9@x zopoPP7W&F|{Uuq>livxOL)n@GWba~m$0t=oPF0pjBS@0j;xkWVP{+50{n(K+@#5^h zJH=feQ?w3&ChM6o%h8rmwD+Hn9SOi88T|J2sq*e_n3>Yv-c?C}fgmvXxX{pJ^yc{& zh!2!QR*}CCy!vmB{%*`+L>eJ3LWNw8woZ)|mV+K?cv1ZjGU`7LBHf@0U{3O2uKRpt zDv0GjzhY&r^EvbEix4c<@ia-hydxL7>;wPhsGSTDG4~3xO857T=@4mhXA;1w9P~)O z4j#q>5Bo%XWAxHvly#jMX@!=?#3%Q{lh1HmwI29kWO?!=f?Rrp{ARPgNJI)z`0dH5 z%Ycsb=+QqRmeNc>?l0Ic^YsULzo0ajO8Wn?t9!^^*;ub9b8niMpuKnL^^In}nyiO9ImjcgAU#0tCR#w6fZ+rnnetk(QKz5HWUShnj1fJFgdn9!EvgXRf zgoM~5c&7?jEUP^^Z1$`FdNe@_mQDPQcx!g4v&i>ES<^hjQe6#=*FVR`ia}RZ*45P1 zx>5Z7@ImP5lPBlc*e;(Ev*#-v2Gi@#(&7TE%d)}edo5J2$88+v!=mzce4f&{8Fd$5+jwLk-przsuYV1zN(O03NdJ99bdQU1{*Qh2xgdxo96@sZX7%zkbAQg zf{>?yfjnYjdf-cnee_&QEknDCbGmY)SFyANw_D($$o1lD2cP$^BMFl3H^2n}#`E<~ zjE$vy`Eu26xTab`bk?o%dz^TRMQcoiZGVlgcXUjch~r2dkInJp$9V(=F)cq2)XUe& z?w;FOeXAM+yCn6lpR(>PmwI+)rqbx>sCh|HP!O-AWTk>=g8#1P_~hiweC2{bZ)AMD z@l4DGix!Rh_wP3zIeJv-M~AZ3tlnl$qZ)8Vr~gaTqeHsi|pdY~(JK=HgO3eE6_V zjoWxr1f!UJoG!-1*uG5c%$ejKQ8F?z`jaPbHM@_T<~LN?T6@3_MwlsGxZ|@!2dc`bl0II0t zig-_rvo%J5gcmogf8!F5;|vl3q|jJqsucY*G%+Xi$2b8@GC}4Ulmo8fC{HA zX1T_=g7J;XL_913^>-~T*X@3Mc>n9{tK3{ZCK;cydwbVj9-sRB^dO_GU)33QcGbxJ zN6&`Oc7k8!H?1*xd7NLf)PAtYqLnevxKdZo)YrATwzlFn3Jw~2%;#y7DxJoqt~gY4 z(y$2Be^pKN(ZmE%AayD5OVBEkT=kRfHtPK3?7&#Hm*^bbO0>0q{k2tqGmX*FFbsxa zS0K6-kR=HP%<&oCYYH(l)MW4mePFXfe}x7xa=T?8Y``>N*Qa*}0YJJSnte=!_W$-> z2a1-&?^K4#Ka)XJDyIgIzoi|%L73H_TD2m^avq0BlPC`!ii{DxfM=k9$BSM=Rvn4Y zum%_#{re1vAh8cnM9$;~xVno%VHf%x@wvqFexor$F?ubE%>jJ?S7kquKUi8^n{4G@ zpaPIK>P(+_(jMkA`_85xa9avo54P9#?_07V7m>fueT@7-i1?pLLwcL64uuoqksK}P z?|(ZY`k17PQ~@+{AXnYUf68@T5wS%+V>ww^rateR`tv}q6yB{&67DxL$#w1OyT78H zrbA9q&6EFqZClo|qv(qrBl+-f5-S}pQAiewL}cc?qiV1jEe6IZ|uN1ssNmXr?9 zy*d>uM~x){|9lAYLk^tIBT{a`w*UOb$Su07DC-}{e}Loz2@&Nrj>6raNJ5+{rv}TQ zBkr*1WyDsFC`;neU+I&>C8K9yum4_xMkd4~@H&6K?9ckp{`XNYkskpfHfQ%y#LRqs zRZf(-?_s!cY={|pg>>ltfR)KMy_udou(8xXe>~;}$_sJ6g#`sJfZW;gL5G%?mTo#Z zIhFfH3K;Xmw(45!KXAZl!Z7m$abL2>4wE=`XX~gaD|bM^6!GTGd;7r(tC6~g5`Nnr zPq$70(DCtISC=M)1JvtU4ZeO&XKrpD!54Ih$*IRMQ;et?(;iom?HSX(Y&0-}Tv2E8 zdwj9u=oMqPUUP#gj|We;z|v4#??!`D2xhH}V_VaoJ$n|$DJ3-t&$lCakAVAES*4(Y;le<`L0b4I-N z-FbP)2sl-*U%wt7y1++7m+3NJ#3k~{6Y>y7QI!V|9!%pgiJDqk^v92({w6XFeB<{n^^e@m`CjdfInXeSoWBv#5jbzFIZ&K;LpnLCLs=25! z+nLXCw2Pnjtvm;jAyFt4Dx=#6!Etwvo}jV!Tye|=%U1^u91wAvy`?->Oz*y?OyTefC6eKCU7cc4#`>w}xOm$^xdHsGLgkF=b7MpF^me5yXXQ(Wx=Fl* zYg2U4n)QF2bt6VZ+0)ZAEOoHbedak=LVI56Q1)X=S{_c$di2D4lEoH=sXc$RJir+z zK|Av4F{SE6xA#h`m}|dX=~SElZlSQSu**zdsl-aVN#29I+#=ddt)ku})vJfx!GQN? ztRWQSo}mH^J602C`KS3N*%1q79W5V*mML3YC_UptC^u#FX_M6 zKd?rm{I<)&!orFGGmZT=#=}w}Ue1Bs*`VAh?epi)ah@KSuS6gCObbKeGyCytcw!<3 z3J99AyMI#rwwDy=dJ7dn^Sri~zQ%jre@4{m`nW!_GBYfN-B6Xjp0TS=W6Ae;Nt@m6 z4Mw5c0T*a#{>plyi&+lj=kLGg)W5T{qdGzPWJL%~6M24iAs(G-VpgA@?1y^8t;P}v z#;kvSd0sJ<9t(=``wm(UI^zInbms<5z{*6cmWfG%L8*NngznjpJ6bjF51hPPI(*)rEk|X=1e6 zL@uB!`RT@`vJ}NuS1=nea|K_PRj7vOg)&QP!@uF65^Z=eB=ixnu{q#y^u#i*RC_5; zP2nmRR7wXYrh6dA@=9>rb-S2xhy+U@nu*a12ISNZq&q0A|xVG)R!2_DB8pj)WAv1$D*865Tk;7&o4A3A9@iKKrE5e zmr;Cp$U9Hc()@OL4{4SS3p)t2dF?8D(r|C2@!SD z4kOhFn#LxHkS3iVKAm_P(fbllBLE_J|2`t<*0Qt>Hw8dj^@_yp;>Zt-u9uWhO^Q1{pf1_Eb;Bpe<58{up zAQ-vx|Lefv=~kXewE-ZbCS;ElZgbgCuxrzAbuY(B2Lz7(UFz~Pq|CaT2QA(}^#yE9 z1N=Q?N^(pPdVk(Ic<>x>^1BV8(dS_TAcS z?^sLePvQC%{Ef{>8L!-o%DWzutqX#ofA?|+lNx#q{deyst^w(Bcud>=Fa3PqEEUnKwAWR2HT z&B7v`gBv0+@c9~WOiD`H1=V3?+>`xByD+A*tqIbGfWIA05xj(6%S5Y=jGj?s7*$oKC_1ExaGt7mJQ zTeCGUsjLYeuGnY8Nt)25wYjt4k3mbYP$90^oX(k<^X(QRRycmKtsoV=>amLF1l14F zh~igP(iB3B`kBGbKXrZ|{ywsXM?Ooiv1yJqOh{A*Ac%(`n1&fzE}}!KsfpDl_~Jb! zKs)*c$v^&&S}j`w7-_5TN1D z@09F?O;PHpZ->zqMEfEDM_n4W<`vp*kUE84IJ{xg?FZ$dLi8u-HyoAgw~bghdh{-O z3Zo3%WRVat@m&AkJ-Ow7I9}fgITdSqLNM}1eDxXjZurHVI;=>}J@(sd|^I5*$0P9~_++7bsNbUQ9LE%~<_Wjxa9)`KDHFDux@yc$M zs}r#n8stdQd+T$P4vJ9FBTTVNZ={y*E@E*@WBGQp0K@{MtQ^Xn8LD_A_V`O|I}>8` zvZ4^%h)|QuDTR@erQuDZws+EsZBUocE_4{V`7_2Q)iU8cp2fSxN9r_!-1ndgICmGv zu@X~PB{lVsvgMRKz!>%la+Sf)50`^c`Sb}l(TaPtFW>$!u{KIK?(&>%)Tal9SWNK_L`j<(#o<21QEN zLq?n1&QraX#)D^w6+zZ)dS(HBqx<=7@7U&kjP|X8WqhSWZnD>Ir-?YFil1N2you{A zBrYFHiLBPM(EowJ&UY)7TgzyW5n?+%a=ETso#{AU%)1%ED%PE?VWMy|Pr=#dBXUz> zwde6)Uqv`M8>7mv;SHzvt+c7y^(h6>svhO4s1OA=R03-+ZU{+BQ*GtG6Up~x$QMr( zLGfkeYW#rX20@Eg}zgfqu!^m26}q z{rY(a@{QRGpl?|p7Js%Sr=*oTr;TEb@iMUZ0r2RYNy zd;BZm+2F_=Z{2(&VxwCx%Ug4vLp*x)Hi8W7BcVIA^_zWaYk_yK-0U5m!f*BFPVs&S zX5J+=~Tn1);jmGkco8?5?q0~ zrpk-*wp8VbioyA&NK#~^I=m@P7eW4V0npBy@A-erLt+jVIm4$H%!vM6@A=KCpj(mNS!suy@I7|8UHVvNCGP4ZK<`fsR0X1XO~WV-oxWwi2aUA)h6Wx83ZUfC9cAjLZ*F+t;K7Kf zC_RhIb5cuBJeiVeYm2NSL_9utK1QpZvi2;9dV*9|a`Ks{K|xWy%*@RBv$L}YOy5Gy zV00H^@6D-x9aK}a}S8l8i@mBBEbfrG`j2{v7V`Gw>nARl$X8=_R z3DqlC9w~${ra`7mn8B-OWo5zgI#mU$wWn;-uSX{j3=D*ge|y7D`0+Dvd?GfDRyO?1 znN-1=*GG<|R(l%{@__ev4CRskv%Zl08GrTaRqZl=C8cMN9zW*w38{rTru#zv&epnK zUu0dK9OTX6`Wo&J9^gvsG=mO7hYLr*#}0$CPC9KNC+X;t5~RI(`1$kh#X_;&z#!(x zks~}lYG{RbhZ9`3TEx7dGSpNcAuK)PX`)|Uv8*yByZ7ejSQ?AJh&RQs=U21|pJe8| zg`897=H~X=7>^=M=U}YCseAo;qo}Q+k@WHIR~6wP3&jvdodUG^n+vVr-|n?r@in`f zHap9oAb|XzLes5J*c?E*-SE zang4(9+!MA-<}?NUQuYkbW~YcS>nOZCs18TpPjWuV?S&l4X2pbJ#+J^#ZfHhmZ|4> zgh8!e_1PP7nb76sk&rOh`}0*6E$oN+{@-->c6V7ttvRX*k*na>cri+l`MRuTuC8|K z$lgUuj0uV@WWV#@DSq<`Gz+Y;OcbqU5xZgVL2h~FnSVC8aFR&}S_aa<^$)$^*xno$ z0Cq*H|2FpY5gG5EZO@x;{uL$n5qm(Rk$x+urdq|)l31+Um(>yN=<>6E1CMgUXqni||d`Gl7_n~<#O^icc^ zgqoS)5u`pZjsg*aNO(BPt`=|YC+#RVjaOGuF2D3Z1VPS)dLm49QdbC=%x|;f%}v6X z4R}OhkdT<3*U$Yc;-AC1-ai(9;Bw`WYV1M8Q9iDAf54$p+HX5mdo>%|BX_^HC^kwj zJ=riCO=8t6rA9^sF8N;|52mnKylvIS46eS>KFXq17(+8q-r3DhhSd4eAsr(kcGidX zYF1tyJIn@+S@C1rTf?ePpXS>$AxOR=a`mz6)*LR%RE1qJruyLp3ngi9m8q$Gdj|Lw zB8kU6xzRJyn75Z5a`gdliUmu_K>tL>vV2jJT_p0?UE*VL|1yrtfSx(*pq|u6jMj z)UfoTH#Cgz-+xEZKr)~(U;c_EirF)DIzVr`>y4;Q%kxYU==e@Lcp;-{21P=T2W9v4 zH^#{*4!wKxiaVY0z9dTlkeyXuql4r5=xIz7sOh2ZchLWPNpFh@P12Pr9QmBGDx zt!vGVSrr)&0;^v^p6LDE=H(gWrCQuM!VH()&~-X;#6t#7j=^AnV^i*DMOD<>V&2Zm z&01G=Xfe_-bp{i4(Vhd+V5t)%bS4ge99LeApzbpC)y2}|z-X zY^}cO!Em7`)2<*A0iI77Ba_b*MhQZ_@#13OsTStXxPpSz$;7DRe6RgB8wYw)f;R4# zOU`HdonwF^z(3W%8`8Me1pd}ULHc9w`YT-Drr!`{k(`ttmO#q*zYjxg$rYFURKz)b zb*%8y@_Q>HE+%y3bmvmLyR6GRF@zLY+dUw zZZ0R5xL_?;MSXi5RbF&emWmg*tPe|;s-YPQ30HPwVIUbjK%rUQhx-iqA9oZ|~e~R@cWGHvq@!`!Km&&>|(d`|B8< zLD#5qlg&<_DdRe~$|85nYT}-e(eBppG9fv+WrOM^+2}(uP0b_j3}p8&UGhJQFnwmB z8@OgS!NA8aS!v7K^6j;N(YZ=>-vRTMMd5(WFB`GQZ@S%pksG;GH!a&#B;8G@pRoAd z|9;_CJhxV=U-kcyat8~NV>eJH- zo(g0jefu`;y*$%Fn^hrFInPy)lK zaSuAZJjN{W^l7&}Gd2*1RiZt68%*%8pd2(hD}!^Kr^6X2Kp^RA6I~+3-MP^W6j*H9 zaef{XT3WH7WJz~{{uR9#9kCCWDuX(D3DVM4UQWL1_LYS*l9h9DwVVvhcZRAw6`)LT z_~gm^M$qwSc*P`{J+KD_B_RlX?8(W=PePGNG?GtW(>@W}=h)vrKBf>)_9b*TK!j@3 zoemuZ`LH+8P0zuTBuuo5(XdPO3AIq&7dU@@Vy$9xcHKi>xm5`e zj1Buc(S-+o+UOUe#sjck8lN7txNlf9u7^wz#IZ_{_|Cx@);R7>Sg zXTv@1>izYiWTqxG>+v-bW6LC#J!`9*-|fAv;%cQw1V;YR?Su*-_kU1NFuzEXA-Ul_ zfBs`j3Frc(aOEvB5LS<`=~uehrP*=w@o59EuW=oclwvPIq=`{xfZCJZcuaSYqetgs zZ|dd8EfU>>b<4e$@r!;O={Th!pmEBY__?Wy|a`o!Fhxs2gHa8|;nyQvU zJKZ^}G45qlU)jesOxE+aTxn+5IIVu67Msexq6a2B3$G*kAAKX%U0Xk;(7O! z8edEYJ&pEXF9tSgVIPYx$DqgEIqMXuQrZA6^#tvkH{U?dw%L}JmKF~UFXK_0g2F=H z0a>7J$^yd%O>PFcfK7udX@S0No7Bp6rs4eoFQ{4UZxHSf_6-BoHspx=pw6z<*?O)b zac3SGAQC}3R<1pfonw6*(aSyOUm#3w*~?hpw6J2opDt@UgL$>ZA_ z1;Z?tZr}K09DmU+HC?S$NL$;iTADL*MAg{kS}_RArC>EJL1 zO&c8614~1{*D^e8BoKeP^47u-Znn1@zqd7P!xC9)`(|?;<5#O^lGX1Y71J|ZRsM5R z$5`L0RMoIFM!u!nbIWeUv!}kc7H5VjunT0f>pwf_>rNo(<-XorZ5RE}~hu3vUXn^7!(b5pR^=)q35&t?{&Q1`+K2K~y4 zOq9p;&r~zY=z7@Ow<*xBKVW+F@Zp=IyFjTraOhCJQ(}{~istRxNu)OY88mfi10RDr zx(Q7CF@&47s~>Ey*_WqT5Q3<)`WGz!EXH*2bz`he_nLUq8sFw%q@C^!_6@y%EaGZS zASi}p8}0}gr)DJP*X;iISd8%}bmK0xU0PT;V!lyULDIedf9#CY*PLZzd*8B4WFZM} z;qS2^H1ZE<$E4TandZf)=avO6;keSTDjHsBEGnEn5u}>s(_SE4rCIA|5~gDMRfp0Ta?JSUd3T`}d2~SRv>!*beyb?Qm|LP$#=! zh2yvjd2Pe?#tLs7MMGO#o5$|PWJtC5IwLgsEE!KCu6GE>#0fgX3@ehF!-Qh#>YBoS z`;&3CG+6ochi_7WZfgbf;im83b)oNU;xW3fes6Ezk-H7S9gyf|@Eewd&!W{LwYW<+ zO??Rw$tfwKuG88f3>VU`w?d15CMzQWhn@-5@R{<-gidHm8;o6ia7r9epKMfrxwyKS zFyLm1pHyc3K6##`ooR`Z90`~l-JT=CBT($DGa!dr^#9Xg3gWl`uirBjXrjC{gxgqW z|EZd5rIlw#Bu0P{9hM5szZ0cUH0|n%zF-M0`{`+mGV(T3w}3%=JY@e}sme!tZ7-_Y zOcE_Z0eHwCx5b}+&z+khNSDtgm<+$|pZ}M`+0E~{b|(W-0$yh3xs5*_lik2WlBoOo zRk5R^8&Pepzw9*qJ zrC%Wsx=sw);J+O965Run%g8Pu z>$sM$_1j(yVUZmg?Sse1NVsO_S1qgTqomA%5NLCZW$*qJdpIh_U%&35MY2nmZuxGl zb%PiTG7P$X1F5knjpG;tL)OU1ZO*f2r~4ghi*x;~t+OepPbTr^mtDCSs}>y{9oZ%0 zzgv~yv#jh2giavGXu^mHTGIl?kABY@dOx&Hn4%EURb-_FH?scviYc;Na1Lmp@G_lo z;?8_o(iSc6oc#A5Q_vI?%|ZXXo5_9PeS5oxsCAbTm{1R}ulch~J0|fUMKF@cF0d~= z+Ohxr`#$ zpm&s3tpUqH6bMbe%Z*%pMOJTt5Q`xpqsUB3stel$ZQ5)$-Ax9@Dk@a88!qDVTf-NM zSdm?VYp@m5X;CIPQM}Pzt3d(digUcYIrldHjD_;&_)d3m8@ms>F0ZfSw-#zKn3_$M zLd!N?sG0F+?V+1!wFQ+Z4&~a-Zp`U?Y3GUS{IL1ZAMYHh_7;VY((41xUQ6{SnDUKn z2g)9xi$e(wb3@gJWE9jIckU#kG|3rspU`-(dhzo?^!?ptf4m3LqeLz6;fWwJD6!38 zYE3R!elIGTgYv5muyfO)6a@9NG#KHCb*Sk8QJX3IDo?k+e+<~3fpS{LLS=QKQ7U@7 z4vD|?;B(k!{gBrTKPsjdFJ2Vn&V`BGZ%-0Vd`!=)drNn(c5erhU%Tt&I^CsgSYq38 z!qn$|mR1G?W8cCy)B58hBGO=pq1mEvDhj$9oCGaR%_1oN0-{|fftL7yBWwsHVNHnVfeUNnW1sbR8+mTm*(nR73(Dx01wsW|;}2*bcfRTX z^#Zm26J$;RwZ2>AFg)}F06R^efLZdyG63?hmA8u@62ijvMU@G&>2R6GdZf(4C0hp%g{hFk+@SON`JJeZZes~x{VAyBH@u+dJ)Ljb2gDy5zM~+fz6RQw zU`ScOsb&;+x)&1UX8jX3=_bacdNs#%M+LaY8ZfazSZ)?H)U7Q78?*u*o!7`iN1%ev(9Q1F0zOswZC9G=303RJ z67Y}3_dBkj)p_<%sbs+4Phk}>b^;lL9&i&;@&WogG%F%alC=zO+<4W4F~v}_Km-tj zqMzhKRSM8wnuMr*7e7(*o|lme*Sz#nP+8#0IW%vaEX_#jY^1R?3?T{hR$#} zRL7x-kzYA4VGL?M18?(7e#LxT-Yc2m#85{iDVYkRAe>8@8zyBXSdVb{8ty(=crVGKX$XbCK_s8_+fNor3Rx z(8E#}7Z-*5u~@9C-vqbc?uMoG#;>;(lL_8@zBi^Joq~a%yFmP(a`4}Zy=d2$3MgTy z={?aBy}Yv039GVKuEtT&Fs9;7ywza7Cq~>ky~KVn8+1P>GxOVMA83`HpI^=Ha#StA z#^p_X?QR+fS_>OQiw6YezwfIL2$>DjhRJ~QhPo?|*lNRO(=##UK|V~wD53;#jTiJ= zIv6zV1|XAXPz=>NE%1We@j%iCiFK_H`cWpdK*5)eW5ov&P!b zb8eL|?F7}ojkd7V`N2vXV8Wwds}RaO=eUD2@f@e8VU{Y>#p~&eL)_2~C#s`Va^nh@ zn?Q&UEo3(^2ex&cH6o!g{U)YVH<{p6i@w>{CJFJWqVVCQ`( za@*V8i2ybxj15df4WI|Q9eJSEXvlD=TTNXZ4`WWi%QaJOTTKIo-OYj_@czcmt)kZb zC3z*by~zOWz3=-#_rp-Pg+`5GC?Ou4KT3N+*{kvd!D5f4JE*4M_VCqze!krN^;UiL zyX3+-U}mDR3o1g0vVs(}w|90TVq&tvQZ#`3_7;P|049riERIMFo}0*ZKH8K**?v%Z z^LGj~IOAphjF7=7sG)%*E2|s?Jz5|!(Yp=AQ9nQl*D`xAuL@{O<}*useQRq|_w)1X z0|-ZBUYMQJU&d_X1ZuaPTz|J;-dbNMO*Of9?_N2bPq&VqzC;V3G8i&6^R zqg+gNC{dv-L%`W-6cE6p@XtMmH_b!!5SoBY;!am3-RHYDRwldC)lN0PVpE1Gh_Dv6 z5*(1ARs1IAUtovd%LhTgb5_-Rk0a}5iUQh>MTf@pOgyi_0=>o~^iJn*bttk-!BmSi zkfR$d{{T39{d~^VRpDy$^_1UpFpf+J<0E|nW5pZ)Njc1Tf#&I(j6ydYU%${#MYR0ld$>y^U$qwj2hBrJ zBW_bu(}M=yH^B_o7KSn3zA33rqY=t@y66u8xjT37Laboh7{+Pck^D4}n&FOqD+DZ& z-iIr}W3Fh&Iig1X?v`SbiiBs^!%TmhTs><1!9(cQ0yR53cPkv|uq z@;7V^@l3_f7rb$P*K=gBV4$@%pkl&_qo7fMOUiJcqNFr)UkG$JAr|D@pr#}z&jd0T zVVS`XZrugKXwI(;Z>fz1^OxLlJ`KPRb4k8ibETWm3$)(e$eSPdHI-=SUQhQSmirKM z{UK(lOqlo(wduJI7~zz;23v(Pg{pVf`mJ%P%8kWGf!7U5s#|6WhZ^v;fwZd&T8+`t9PkJ@}a21@9Ve;g2|WNnNkPF`c3fOMK*YW9RkL` zn~YZKcJKL%_5LNpobC%T>NT&w$njEZs%}Kr%?VI;K`rozy%#wC|4-8=RIG`YLGyD;0j6OAc+7$!ExQCd6mPAw&H#f44M2bofe~P|-hf)L z4M8#|e>aR%m1lbcuz)GsSSs)OwPrxy5yyUiASsOjNQ)#VCkMk~*@cC1Wn-@-gTWUX ztW}n4+oeCu%3cGxhCniN5=w%B5EdLhb;<@UC@Z+Rm899BrW*_ccnp^>UseECW-#oT zfx#XmG{TTb&a9T28t(vS|1MP26d;{DeC*ggm)Rd3@1(umLegJk=dXeH34z=TtuNmP zy7*F;mU{Q}9<8XjnD}P-HP4IK4~`NrqmGnG=aWG4|Nr7IH5`Pj7vh{A`0D}zJckhy zk&{pgc>~#RDK(?Ww{A5ur1Z|<(J_)b8u+XPo6KFv58}I)M}n+hrDYAE-~=Wl2-`do zQZ3*ZSf1d!l>{*fFC@C)@H=1%xT(qy;#V}YJ9heXE}GJ!HJ*sXL{p1I$XIfKhjRdC znqk0#*>|-=SXS0^Id#5j^~grOR<12In5WTJmQL>AUx zo*nM1bjNU@v{SGLZAkwJko;MLF85V=I!6bT-u^@i8M+DjRN%%i=b6@;FiE(aa}Ksl z@Kk_WPAMSf!9lN?y<&&20#jeiPJflP6#sQoL+QPiJnO|S%_m())I|4{C=XaaEGpyt zg)_KIC!+93Uh9^83Z0s$&YqmoWmSrjKih#LBY&1&vY1m0ZSDK)TRXhskuZl{8hGJU z9%o`=QqjkS_%tC&iXI7;zP`Q_74{>gm6bLY7GZpeZIrtg#KbI5ojPS<;pX9SAELw_ z+lxCZg@qShUOm%uU!@v@1DiF%cTJy`7H(_cGa*fKIQ zQYO#L+PbOM%~(%fUf#>gYtu;`X$M9$B3}VSSyKIaD<)yF1&$!lLNIhorHwyT+!bGrxZ;g@red7Znz= zx2PbFaKlA`o0dLPC_on#K#3H>wmA6^pB?a$(k|d&ZUiz|;AQ2asVoguGqr zds|zC5(lYwmbS|J`g)j5O?#d;Pv5|R_L(>Bjg8OGa&d7gf4p`5#EBDqeSNsu+1xuH z-xc)cf@HTXJrgo?bJBKpeCc0as~>ximzU>?xs3cqzP8MkzC;4-9ZxG}WMmi^8}H0% zA|rYrb?68a|2Rs(YcqJc`$9{k2#d$Re$9rKy`B(Rt*rN`0>D(+h<`?fiUD*Pk^8ny Z($!F!*6 z*7LsK{_}nB-pBsGJ&xz$fOX$j%r!ID%sJ<**Z0&Fh%ZoGKoEpjNl{h{L9hc51Z$80 z5B^dpnrsR`aGhn8v1KOZdgY!$=bUf^ zJuKz3e9|g5{G2t#m4+XQHxc;lGZ7Xkf;2xR2!KCv-9>N_#Niq?9fAaOk|J0LV)POr zfLD0`|119|i6s+EoWm_oOS`c;T6Xsen|4&|T$%Aj1YyB$b#$3+#u|FBdH2zywG~2o zgrX&aT8x~M(xQ$CM18T!ZmKXvzmdnbtiVe<+ z@R5L^4bLk{1Oazg80890O+~#bHlAc<<-ECmn2U{!w#@6yJ|++*SRHddGG%J_Id$=O z#6o7ReN4`A{WpaCs8n8sZKnkw7pGDcI9aV4$_=3`e57BB{tz%w4JUeHA<^; z$k|IPhCIL6$3$NSA(E-IQhKsxoO!%HuH5hA+MhY%+TWXb`c9|lfi@ZP(6rbbjhl)GIZ7r8rA_Uo`fB?W87t!{R&^&x2xg5&lO{ zXUm;umfci~n*bacY9WLmD*)4%OjTp7t3t{`=yCm zosRZ0y=*xB=lz>=s64ei-^JX&wVL#nhfa&9``zv2)*~s>!Vd!~ysL|qu@JkRy$LP! z8Pg;&jm_7Gva-`7)`L;vG~1q4)`^w8Q{l+fa6wyMf6aJq?x7FK?scv7yUWbBqYwRl zGLR9J)^_X=VZ&*cHgQe4XIe__MD6s5X^kp5W^1#OVKcG@CpQa{!b)nJdeC({T4YD+ zZfA$6gH2Qun$y@Ju)?#P-#pyk8W;K88$qr0ILngcy1AtHg`>2XoX^6}4X#E0R(B;k5nR*<=ZuQ{g;P-*W@ND5V zza;MUu_-mtDmO<3;-CgJj;pwk0B?Alj45d^>SjKP|7D4Q1dNzo-)*+IOu^J}^2xGk zJ7TAkyRnE6nKcQcOy$ulh#4%NNRG_M?_9AQ9=%jvFGYlQi)wvUe* zi-_Ur_4+3z$GdzJ{%NnP@uQGu=dgH2Pc}nbe!n8W#yV7NxO3dF6SGL1#;0)f_xj8b zKH`vRJCSpaQ1v*&JJc=hENW}8{1Tq z1iCa{xv;uDOY9kID)Lj#ys{ya?7uu?PlTkQNj5rsl}rPx^1RaxJ33_ zYmV*6==N7*Uy&74Pw*qkdzo^d|7P89M&80MDU+nw}%R3#n zKKXMF5%$*CH{UB|nDSn)k-(D(7zSNaat!g;aOvbUuRSY!g1WD91-%JletH;K@gyX< z+k3)xe(&lIVsIBBV)~wp<-;4P;PvOqz}d###Sq$1bP|H>Z>XPL<&kDHKANO?P9u%k zSI%d?0FB}L8kbgTbEorRcX5zVt4@mY{Za;E8lSbuxmB#LHlF#kLTjJxzpVw{z&vY&EQE)&KQGXMft85q*FJ^!-dfaonHMD$x4wFYbjD z(;;|-rfF#RtebbccE#3{YwaTgFDy_8g)fx9cgB=tbikxeYlwm@8 zl5XF9{rWYPh*OSksYx-DTFQIlI=6A1whzPhO{coP$9qG~fjE_Js|MXE0xIl!C4;ju zqWSL_Y$uWZ!f%`A3@`_a1)GQ54|mg z&h}n!*bi@bse9mcurE8!_=rh8j+f6D?Slj~yRRhiyO&t*FKB*>G%k!gA9y)i!;kzj z77ria{UY6BEg2b^HQPsz9w{p--g9zt3f#9GDvZYj3BiPv9ga70b8{_+Kj<&@y}SFa z*Lh|1_ER&n-*pGUr4~r$!my#*zMnw~zkSAfCJ91%kiaD{Q3=l1Wk?4UBMpDS{pKZP ze`PFtWyMCP*g#-rRi{W-$I8kIW|^w3%u`P<%#iRdbXi0d>K3Q4>3(=$_N0~KNIY91 zQdd<~b&av-7-!kKF$2pdODi|$&U4k&cDUnM?GLKG-C4Wv`va4k{U|SEZ5VwptX3zH zI{7FceibfMUCYg1-^V{a4SVUqEJObAVB@ECs4%3cQ}%imtpZG=H^758Fv^g(b)2zz&(-kK>2h0-oIsKj39^V)f4 z?7KVUwNlo~pp`Eu1#RVPfb zc(~MS|4>+-`B+X0TXtPVi*qA%8yxz#sD&R&UjH@cyM1H&l7JU-%EaJ>s3dX~n|G(M z!oU402=N#h84F-j=cea3ezt_ZYRmL*X=!0pikLen87;50jEzgT-9tCbVxuKWtC?N4 z%oaBup(SAakDv{Dj|oPDq6h+F;*yfypC@B_w)MN+pvkY{0Hj zoEs^*HM5P4V7ErooAWDfO>LBZV<07zc}(yi#%ph-V5-jD`rE5Z%4%vl19@td9-F@= zU*8mSU(@b5@2|hJK4Xf1@kU>&ePclr`gFbUL0OjF=bv~D2fxCqOk}>Ch~ON^K6+%H zyZ@-A%PgvHeQ$jm2U#(!e<147x6@Z?vsgF3xrgNCr7&8Y_}0b~dZq6S7KyXZ_8Hkcb?zzx)+yVhO~>k1)00iTB*@3*2(d1f zzU)Y4?xvNxdB+>&jEnu*DElTQN|S}U`It<}CLE(SBCHS&Ly3TxG0z-Bp#d799yI3F z`He@50~%@*i7ibrBHv%Xrt_0Sh|I>uJUc&*5`bsvp&Q6c-p0a0);>Oq9t&$!r$&b( zjx=vgFCjQ%wEjtJ8*PewH(Oo~7w7(X)BU>ne%@#S#JK*`V@(oX2Nsua@dciSB3Ik$ zTbg#4Rs1vKxhNj3@D1$kZj@3=1pG7k*)&!TkNXs3Ji7(< zVxsAI@i}DHn~*Za8R4$KSG%Q*7H9siCns?ZT(YveZOmRjdWbmS_>-WOZSdzUaGRyg zo#6sM+POVaZjp*v{=3E5JC@EjK5SRTD=?l|!Cy3{KoC+d z3JxCsg^?m=0o$df`CYu`a3WTTDgWMOEf}-XIscH6fFzNtanwz~+#*VBoqv@?s=|;E zX?C|ELMv+>ZlOHqlLcFvB^3}N?dhL0VO$r|I34S;9Qtor_R`cz*zT88FhW%HBlXSb6q`vKQic)a#h+|kKfP7$n>6*)2B_tGD=la770`n~}5};To{(YqsM& zC*pWDsrV;C$3IGQ7XN@~JQ5yggIKdDA(?{qnO%bpo1yN}uHItZx{V$V!cwy1V!8Qv zpi1C`{SPg655(Fbgro?L=G%*CFNl$++)^$1uHhmfw{} z6Srp|lDvZu$p(so*-Ss0jg;hVdJlJufOl2R+Hm*uMkzx)+tVI_L}8M?R#?YEze2aJ zI1b)uNshz$^IGw2sI@87r>9j#SV&qC4*u3l|2p?-!?16j0e+CY$O>2~Qj3o2zbJoy z>sF4~8jtPEKrA{B&Ib?8(I!4+gP+S*t^O4v=Mw}7yTkl7&fTU_F#q;u>)-Lce`)6( zAb(Ha-hewwnD;`0a$sj3HRkRqBsL^wFo8~U}{q8|4O_3uL{<0jzw_(|@2mVxQU zDLc-a;mE6B%hOXF(}yi2sPRgSOvr=X52~&8%8%vWZb01n3mFIByW4q&AL|xafQX6l5K@oE6Rq*;-ma>KJ%foM^6-Dn@Vmm2UZ>b< z`aw+0KU($*(txT7NFVDe$zT5akryHJ-Aqq3pCJU|;JUo4@s~DA<0uh6-Y&B%al+Vi z%DTN7f4$a>bRF;mq_?|;P$kA-sY*hV59?CCf@=U`nvCz^#cs^=?NIrf2x@C^joAd}oCY_OXV=-1kRV*sRzX7CrA6GiA z-lU;xx84(u9&^A~*>x4g+!U{{DcVkqm;CKoz}jJj^_X-fU=-usUDVfL%<(9Cwlt-S zsYFM+S+j=HNaR${^jz1Gd3_7X`t2$Hms0QX)vE}0;9Qb`LGJ9;v-q6llGv9-(J6wJ z#!8B}kgr1Z<$;tOQ5R`2&gk>h{!Hq-L8m`Cd#kp3Aa&%0y^&+Ec4-D=Tsw>TnEBkd zFQVO6SmIfqR8!#MV$=Dw%(bTyea-4a9%}GhM1P$p2-u)r=h*-iK0C$iqxxu5VX=B9 z=RHfoUHpk5I%!hmr`Vh8Dv*gKCz~r>MFP;CA^!G_V=EuAULgdg+k4o&*RLWcy^cA_ zd}X)kuKBHlx*Vob5J};P~G}zV6r81^H|S$0eAn=1h<3n#dV+TVLxS|guD_o28_t*cXV&o``yzu|_U8&eGH8#eg`;##-@za_lU=LQ-7M02O$LL#bLEAkDM#)Ih=+V-F>$>VV_GamXma~z*r0;M9f=5 zMJl%ANmcv5%@FDKt%y&PUPa{BEa6xQ26q+*bejBprF2&xPk&d!TztPeyW%7dzg1j2le^?`qt(%dQuFe! zxw)Y%Il_j+zo{`f5O4wjxcCL1cJSk)7IhInh^3c?iV_9wr#~piv!xF>o12-jFfqw5 zE-nIW$fbK)5}U!LQ}B4Y!tYGqYI4*~4k4|l;EEGMCMUjr{Tde)_4fDg-w;*%pZN&bPaAOqkiqqN7TegVozZKjU%~Zv+a>?_ zRlD1_wT}1lx_NjI^2mMP3PYiG(5I&=t_yv4N*j*~nKUwto^j7E^k-ioBa8KZ9Tv9jJajY9*@Bf# z`RETNAN^ItVK0H>cK(o&L*QU-<*F4agUl`D$B!Ru;^Ib@mX?Qh8Y(LHE3Jku)NAZ6 z4Y__fM+t&t+OJaKZqSaHdp1#Jdv4m|f2S*uA^kRaab4#{prk9m+w9 zSEyfZ9=QKHJY3tw#ifxj6chCcyDLN}!mTA(?(awet5w!|{5XqAHR)}8EVK3L(Jqy! z%lnJeqB&SNc#6CY?vUqpOEkK#Pm7{YwgnzGBgK$;nfmU=(|E7k#-!J5_w&Mw6Msrp zPu_~>WVfBR*~7;}$fvU8P+y({>Z|2b4`PvpgnDmf?0#weRwyQA%Tr6Wf^8VodpNFr zi%J)BXM>z)|6uD+r!{2m<4yieDMI!oV(azyA;Cv2mp0igGk(5b8SLV+6&#Mt3p||Mo78>Y?dTAf6i4}M{Ph2@i zEZKDTJwEB34|nkkd^{H;rE>UeX7G`NPUg&{jghy3gCCM%v1SATA4^rQl{18Y!zkbo zK5f#nr0-RcFZ%SzEglaGoZd9nKfgyIB*dPovPDsvCb_LoV^(qtKCZCmmQzrfs8$L7 z*T@x?=8GS!OXHUY%Z!PGNU=x*Z}Mu@?&=Ph!wq#|UidJ?KhRkkk)7SRh|`0=xlqwA^_-1j)r8Fxdv)@U!cG_eT6 ze39-~59T6s=UP{Xv5~;XJvYd(mxp41&4E{?gHUDGa^Q!JL!*6Wg$;-mq2uuwIjdXd zs`9&A29tVxZQ1t1%GapIEGZ~mEVoxmXtiz)Rw#oFQd)!*ZFjQf-%-Ito`%qIj;U{@ zOTS!I(H#OhkhHOD{MgLaKUvDe5E5W)bNceKJR-J0mc}>@Ti%W`J`pi|QYBhQfLVON?1hI>arH43g zoCuwT3(Ng1o>6Zi4bJ&=e z4JS(QZoY(58_5n4q1~84C!8I$U|yxQImM%e6WIXpXb(RjCsOj0!3E^tdrvy**Y~{c zVfzg%Y`P$va#&>h@p`IrreSr$(g0g51SUXyn3)w)g!?P-NvdF_ z6qpCJ7~z^7$fG2cW;y;vXt{09Dr&Ga-0dbnpb0CDBnt_oZnfh?X4wJuUKU%A>xeJ^ z_p8~?76q!;EC-k@huGjSq}wOTneCTVUi$yMi+OYFj~YQ-j^)4#=t<+o7kn-AJTNtv z?^iHq2}bqVBP7tGi-+$wLZ>aPKE1)+;Jc1w?}Ajc(C*$M9T+9lxNkUpFl^F~{SX#Y{=NZu$eH?EDm=8(p2@?oP9WgZxTD z5I`VlJFYlXsC2xi)R`~_atO5_=U;5QRG>-EDmm>Y;D8aKhR5f@><~;iKsTB$e%LZs zW=LWbh`gHp4R$1^C+af_Vz_ssNLLB8%Q@|M2W;CY2tFsLA?VSZx~UP*YsKY8j|}TG4a-ZHg+3E&LOfuWne{Swy$pI# zxksMu>N@omwN^u)^k|W^_V)EB^S$Zs#_>s(o)w2xmV9*$RlBtQWP=daE2A8$-PGu^ zv+%llctEJupxF7WxP3K$vG{$H^_={)I2LYR&~$5zz@19QH;UJC5>vg&uP{!x&QcO# z(X*_KIXfNj<-IRuC@9TaSm;0T*T<2j6FKz#Q06zRa}*iDE)zSX5#vGod0h4fRGYpxybtN1Tt7= z;pvy)*~ucZecKmtW(o?lbQAtqA+ud}Bg~>2$Gag`mdWGeT=-m=Ck#LAjTb-Axu24S zZW(q!urncrF(%fI_{yn=BLoJp@U$QT2YSCeb$?p?Im-Ki?Y24HDXaI!2HND|j-J1dwq!090F5SmVVbym=N%5Hh6`-{&q z!~A)YzFoTzJ(L)OJDoKtw@?vtSB98>st&wX8=TxTtILPC8L%qxa%0aQPdHidy!i8 zxM3fp(QI60FEo!<;y4WoApg%nI-e>s3gOJr-fGeH2Jg086+~2KCCnP;YSPHr_ORjK z8^0kqSTK@CE+iudOGTwt8?#@icpe5&3EI4OocmE=GxiA*1eE2vkXak)?%I@L@q<)W z{wbew?d+Eo7LkDT!!dsT0(k7I)az*}+z){LCMPEwR9eMfzI@rqo{*66zKl#zOl&M+ z!wggn^1D-otbeyf%Xwu${e+dCo_^*W7S}DA+GD$c44yCG*Crp0^INW)6u8+Is8E7V zoV)NE`E1ck^VGtk2tw{dyHbP7LaqDv3F~`Ho6ZU!wY*$xA(7ISm6eUou-IAX2Qaq1 z;agee6Njt9eH@J`aSW$>sj$cwysho2C!5VU=8wGSkYHIv+;D7cY-y^lTK)}Ht_oL8 zPTT%43Nxw2|G&(x8fU?kbD>Lca|2foF z7deBv%+cBL`ff802@9OwrAwE*ju!KG=OU#rhlxHKX}aK((blFYr=T!jN2fUH99iSH zF0pvv2ue+0oUM?$e0G|<-BWIG!9f)28UF)3e?_>4UR2aT3Vl)zwUSuxYrJN!mPSfT z1+0fzBqgUa|FqDGy2cdh#-mU<8d9Kd0rz&M+yQPgAKmX)0F_MB?N|3B#D)SByq1Xx zb`v-at-6wKi@GdC;SpV>YbTnqn`tz@pQoz%;6W0VkX_pPnNFd$rndGVYc;(r#AL)* z=^8eI`2Ff#j#sPLu}Q5mynBphJoB}yOQsA0_z04b(Q*ro)#(O9din5sR#t^6g0_X< z!zky~iNK6Q4x-D{4sD!Y%hM%iSKs8ZZ>H9Ojr9nk70oxftz?1m3zXK5Eo-T%w;#i-cN3 zZ196<;LIQ0YsV>l2@-j(>S;D`wE*^nQGk1pX7SjRV`Xk#k?_OB1z4dk5CZqt;c-X1 ze5;#t5Mk0C*iKpPQ6=UuXF_b{KtT}TKnsY=*I-nYEw~EONfXcSA2`SJ-ni@zVcri% z2=)t1O>6fI_{{h$V?YCOi;U~G_O8xPO5p+lX|lbJudQxfd29^xcG}sY1||%UBP{=l zkjlc~`EYRys17Rr?nu)u3 zBZ^8$&`=Hg4iK95x76{HrA)_b2Ed|%TE9F;CEdS8x-(tMt*>jNZh>Ddb1DDD`3n`3i>D!v%T8=8KL~}!4Isvi%&2^V4iu}sq8*&Id0!%3juH7>VaeV zYAS=oUXpfg#gtdTV>$$Tf6Ag{z9+uQrp(|%03zsH>lyjpNS}(o%}qeMj1&o!f0I*X z+bt4c-)ISk9mv9WpGu&kf{d(t0gLn-K|lb5#L35+4B>tbsEsqH@Z1@%*9XwkYvT_1 zj_>jaF2myA)Lg*kQ3JW{v6!t!-4JJyIIXaZym}K#7S%ME;2W!*?EYm1J%sr zPp1loXRF9a!S2gW(M3hFLb9)lL~8qXJGj`e%QRmUC0>0J^%DMG(;3PUHE_NF({Npb!46 zBv4=W5hbsm=h`vR;k~bz&!H-4HLfk{^zFmxxn|!CAJhjt2Y`|~6JX8CH((9Pt+6A7 z2Y*w|(Hh9w=DNQMB;w9zV7P7q0f1Bo2JI{{>J;3vonY!th9gAIehH&KKFWhMTkAQV z+cK697zH^4uNIZ)U`1;j@J8|huDi?nE-RbULU>3lLInSFvQJ?m0ZkK!PVjc05)t-{RFA-1mAgyzyI&Dam2 zCB?`h*(34AHKCpl!~}E|D-+CbsGg3~D=0th952S-z`>?-U^1>%th83~-d7&T!&twP z_Y&*&@zm!seuuX(A|U_R7ME%4%yE@bMWrdpL5NJt(omkeDKBA@g6t>^LWWIukNi1a za&qAEa2XFjDVvj14n7S}nAD78h|82Kc*GO1LvlIQaHweJovxJsh5XeamG}FHZT-gj zoep+5lao+*Vu(I6i6m6O8VEF?7jK34)`dD76j84(X}zHWk)MDcA=k-OMxE}b?(y~Z z5fb2*N7DmoqRXU-;~1L~BxXT>Ta)en4;$j@e-hoauuMN*GD(KT-I4BuNDRv{uv{}Y z)MdRPPmP<7)aOR4>PcIBbgf(bcvYqGrHAK0F$^Om%)N*j4>8dg*p2XYC}EoV+j?!8 z`VN6^2!PCOhTcb4KUgX6g<_O+|5G!?NJ%U3N327IPpfH~Z-Gp?GCaO=TuqHLONfTr ze1@m|c+LGnJ#KCSY%Vzjw?}SuT&u}1@3e~#9G-*u^03mAA4y;>N`Rs-V{ys@CXQSrQZr?_-H{6?z!DH{uselk$eZX%` z1Qbko2FoHoIx~(WFMK2A< z%SS>cN$NllAg%OlShe=hyu68pRxrRg)1T`Nt^If`lhU8giDG;LX_zKQd^fHpLtG!Q zK3J$-d~|fR&fMuf-wq)Ltd5H-DitK`GJl#ZmxXmOGZ#5eeA>7XgxLc~TN$Oooji3% z@=z8yS|G}`t;2Z~P-6t|HZ@ybLIPt3-)l~949uYI%<*tG8gTHh9-Zp};Dg07TVCl; zhZ4J~M;lX#4TJAq5Ot6SFJ7@;rvaFB-7azm7JVR7>` zMA&WR!BD|{1x-!O(8$OUU{(-H?8RzGY638-0}(g`XhlGTF!*Hlnzks|y8h(T@69B} zN4g(A!kSLx!+8jc6%u3+6^1|CiLH9idSA(~#n3IJCtRL<-1O6zmwfYoK+Z#VuD+oX(CW>Q5P1CU)h+7G=;(JzNmq08^Vvm3kJ79| z2yJG#jD5;5MWxMF)N1ua2?2p5AcbnF%HDg5kj2qOJg18xH} zihH@r@fRfiLOG$Mt*y-+EA%%M;l=}EcD|7Tx&oH=5WMs5n(bH9fjsb_<1m-if8gds z)6B)|ne+qwXb5;XGLhFLyeb|)e(c2U=;-*p@H*G_!5pSsoTuhlM}e@UXlCb>Hu9=8 zbfvM%$&LMc+%W6$3QH;ht2pzXv;sDrLUON3;I71HWMp`c)w&kZ7RmM%-Miy;WH~@1 zpnSBg0qXeW*LLsrFzSZXbecNW+Xw$4n)jZGlsr;=jz^5@O6J=pzDdcz%c}*+^C4HW z{x#uykPC8{{+!~zD*e-KEpDxTKsAF(e3S!?{dN=vO!yCIdBY6UNiblxTBf8iwV2zE zLV-iwt5@~ugutUp;I1~fNZR+QTE=T;GfE|2wP3biHtsx1|xo4MU&K zpjjm)O$rJMFmxDTrxYh5)t~=TElzu@0Z+55>micwwWq%r#@#hNUT&`Rv||)xdF>_{ z;tfn1JrhqkOxTX0(U^BrahlW%yFYI0Nf(QciHY$(IoOp`?!?h<`^TO}7Au!TA7v{lCBWR&2n^_Ex9Z zr<}sh`vX(&}V#^fb59I#<+{A(^E0 zXg4=ofw0M1JFdc!{wC>{`N2$k1u_9e908?kg>QA0hyT8pKk;c{J~%Naxcbq)kB*P; zVB?Wx8)^A;vTASJ;@3>GM!#Ab8M=GvC)=m)rd>~{9d-TPXrXw9JJ{PI-=6dQzV$!oZs^#nbBtrI?Jcev>XSVQ~fv)1nYEDy@g z>U!KK-3N)-;^91_5{Z2HK%&+Cg?A@DIE^hbeczj9Vm1mc(dSrXXu6GUjcft z!*AS90)g9Gy2!-%Xya}qbQAyC3wrEu4h%!Ej@>HFQLL{iRa067L z4ef4A6{U-%IwLJDwtDAgxj_}K%E~H2fELfLB6Zew7(ifSJ*+4+!$B=-#>Cs5bTj^O zA_-j-oZCaqZ29TKfVm&ZBsz3`HluV?F0mE$GOGhb6hN< zA;76-g_1S+Bpdp=%=k)x3})sz30fXO-J_rQU=6Q#Cm#)D$(dJy7uu0RKQTvt3b{_& z2U6szoyE3X(q`r8>MFS*f^w5{C|Vro=9r;Z$N`|M*Yta6RS~uW9?bjzmYz&>E7`Vk z+OyS8!XdMIItJYBhwYKN>f{a0st>D$2sr?71m@52=XcM5!4D+SDX-a!5l_w*KBN5x z8qt1>&q*FEog=t~*~pt4mQ|x2?a^MlF)@%8zPOhV`(*gtjoV*OyuT4Fz7gtLBtU}j zn#!avxte<4ji6-|#ac2|O|!2ERDw|eogNVCQ5DWVL( z1A)%!gbEpR%AOROb;>ys$e`6sxQ%h-<>?bpZ2Z)E-99NZ8zQ23ES5VM*F1ao^e8!3 zY^|o%=N0-ee@rHV#F_l}a%s0`cAa}_gO5db%y3j1)Fw zuY#RSG^9cuRdxX?fFo4b>}c_Qb8lxv0ojuDpoOHWD5%K?-Qe;k9%X2s>YH zeAslex;ll5KqfYJPaos^Y-!zuRQlM1=`@E|cZn4N)bT)sRy1J>u!dL9j>zRO!If zFfP_aqMUkEU__C-!E7%`Zez4cZE2{JW2xRQU4PHwD=ry<{sakzCWLGO(<%AVcv0e* z(7|iwuAZB9?h_?M7BR`J4F_3+Zv(IWA|~Vu6GLx>@jr_?6hcmd-EomaC~*(uDb2N} z{`@Hy)^+cfo+<1!Sig3VV?o>|>Drp@PzT38uN%kw5f%&79-9lqX)HG$mJ2(a9s!XL zwL7BHEB=Iuf{w>#+^p)*dZcG(N2i!gErY{%n|S#MOBkxz0bh;%Y8I8CjUkOt`0QM}EG?ypx2p;U0;IgV&$V)ZDvys?ltE34%5} z66(gaNxOe$rJ7D2W1Bq3%gg5$GsNV4v7KN%-@lEPkV;1Z(!i1;>8px({=UR|>njK4 zz{ht0n7C}bU0sry`0?g18jVuPC6VN5D5Dllx_J~#*nQ}aVDLvIwB)hez`)LchB@VH zfJuTWyBfdJ`n7T0Vzh0+TP&$l?Sq;P*@<0^oI#D2!AEJUF_c>{5(Fqs^BA-+l|OF- z-dixGvY2}cP2q5Hx5#s0cM%bQiM6Wjw6f(u< zU{Z2$I`f%{9r1j1Ar;SVx~@EvA(9A^tF}(mZ(Wa3QC(kZrA?h3sVP4{v^}rVojOr+ zS^Cw6@Wu7Mc{Y0bySH!O29~^@i%Z3-6H*7ojO9MH^c2byD#17!zfqzm37po|jncn` zQCojdpB>EwOWj|mFFYH__F8X9o54RoGG}m`AOFNBU}?@%LtVF5jAiG ze9EXh#ci1o7aWZK5U9y(0TWKcl)jPj{6)=kvt&MgWJp*rE!`|XYCw&^XO4Jrf2P~^ z=TEKuwLCo@$!Q|`Oa?}|@$&qQkT-AQt%kNVyBnduOT@xyT6}T2bbIdsm#GQ#s;#d0 zl)CBK8Z1mb-*d&mdev)>{jh8U2bcVgTUGLB9`o>>KZ%~zraooKqkT2y$>NtUy#qhF z+18&OEM=q}r+;CxhMKJT}*^XaSu)sZrM;G zbl-wQRH~<~xZtWUZD3N%v~w%>*wt|sSr;#OqVQB;?Sq!W#Nzu4{&yw426ssD%t*(Z zObyCpfUHe+Qf1qDY~bNF%l~tM9wz-2zd|n(=4ZOYADSHhO-AA2|4MyuM#3r`ry_ z0zk=OB&(jf)yy<*V+3_~O%y^3K@xIODs!&DTF<$?gSWG_E3-74Z=RLyk6z_~R%Ps% z{XvqlbupFYo0KI5f&NIVtc`G{n4WFZhHKYQ37qCZogMK^8@~JLE(?7HttKwf(59Bx zy8YvN_b4ZVdlh;s>wp<-T@nqBe0j}7-<^NreNRehguz?(N`rpUx9q#mO9n|i_iIeF zjM4^kUW+7doNN;|#j(bs4{x-tJak%laMlUjZoG~J&U}-zlk(Ydc5NO$cjyB&`Zd?> z*N%|R!WOVQxC=}5c9Kl`pKP%xxb7aCp89zv){ftTVgQ$gR$h3rrT61eG0OQ&`<{uq zU!nEQz~d3ACP9K!70)$8cgfP9L{nW%^@HCsOK95TE~iGh-d`XZ#RE)GX+6Tm#igzx zu^V@^zrNlSgq4<%&0{+(H~t}vU!i(_`mANkMS%FYU6pODR%g7fdRE1)3i9_Fa;94{ zq2c`V%TV>mHD*FikwrnDpivpFD=I2fqT07=J$k zeY!t0RQ5!H_!_V5{d{#DEv*Dm5+FJI*1+qN^HLwagsrTrSd8OAF?m#rWr(TO&@^{-sY17la!UwFZv!U> zhwAzB=atpfxi%!B_Xzrg+T0E6+*Wtm=pzp*bdlg9oz53&QIqF?^J9h>CA*B}0H{Yt zPmk_Uq0aV{Nu`yJEi^GTALr!dQF!>s%fF1cA(F+*+^&vlZAe47tmg_GJcWJuat!&Lu%X-Jd^yDyyh;x_cWKqyYU- zch>ix|8D{maBlBR3H86x27@?876d`CRTXiYQMoEbK!F32Tt@>Z;3i5+N>IVn`KRSc zr*V$a63Ax3G~57yMgz04#~r%S2?PNpVPw61Ta%1QweNR(Y^C3+7csS{cF09)2sv>W zlBeq+!T(Dm1@&0l>3aQ0sk6_5HluH=r@d78EP4iK&h~^1xnZPcZkflIUoq?g?XAvz zefZTS`nZ&oLJac_4x)xk!fkD`0H~XjbHLmyI!8#cVDEpNYT~^rF!VwHJ{hyR8L>1p zSZv9dQ%hfepg;iB`j&Zj0%9y21bO=mEaQJ}re`us2rT;|7MLOth=mZ)O!wLfalep3 zIDS~}UWYFg=)rt1z)Kwbi;9=Mv2hU67Z+~`zXg!b4|y*Wd|}H& zobhNB(_9l4HZWXlc-QI8EfdoX!rhU!WkSDqXH=fP^F59?sD&M*1*`_|JxCJkeVL8_ z@hZ?fwad)`SW$StF4Fa1kAC4EG^Hcfr2obanimz^VCUTDe3ReJSX+U%|)|=30tgT(pSx8L9e@|OGj;n5km(%ET zpTZDj_gMW84FHyHC#rOSRD6Y+x)?~dmhCZ&hY_COTr>_%VCci6r{qgECNtpb2z{iK z)YQ7&jp%$ydjIZT+41|e=>hn2_T&@n)#LpkJ<|g4Lxq~zFU4sh|MsMN|2W2-TYuE& z@bEimZ5oJmNJ&-I1u(%-mrP0%k1tTRc7*4aPk{cXKHrm`UhBFnst|FbUBbLA>ZZDp zQCe(V+;1OPOOKA8@gF1TpKJ8G7Hl=s*f5ZzL}u|l^h(c&ssHLAU{^JB5n}4znZN=4 zTc?`B-@fS~-UAJm3Eo|Q-eC^O`rrO@L@d@^y z<9T5VG}Zq4%sK-DgD|*W2e8Ij24-e4PQxk}_~L_vb>ZtXEr24nu^QJ_@B!js0q(eG z_RI5hP!v(1>7Ju$^4vDh2dafSxb1SgDbe{iLemakTUydF^>UhRpn-^j1L^_V?r-Qk z?D1SG$WPPCRTjYz05nWh9`)=ANCE`c6UN=R@WH4CbqJSE+Rv5{F>v@!&_0^(-38o> zd08MF9Gm3kAIp5@Iq7?4`eW#ivI1hKjmIi&1q24 zhB<`2)rqhZVdGjCN9fXf=~Hd&KIPg2Ob?^Sl^J2qzP`W}cCK|<)LIz9!5jkPzJyVY zQ!ah`A@D4m?8`SLK%TkIpab(~Vq#(w1)yHgOu2v^=@0!Y>A?B;Q&()!o1qRjixYY( zTY)4W2q}jVuOlPf*BbVxFMp_C0I^cQS0B9W0bO-M8xQU?`wLHu%o7KI?OMExyrwwQ z^2-nXw<~$hL!8EtEyMAFJ1;isspx4d{k7Fx1p+PI3W1( zX=xlvm=Zt)tyIQ(5?sFT!GLO75AHfSeZn-Q{_RekpH2X>`p1tvr}L01pq}JFTkMyh zbKnzZSxr+#T?-#Sc{23b@d+`lq)`Y7EmzriSe9_9{o&T1Lap3T3>W0x-4Ltw>4q9p z!npNJZQhx?N)T=*w$6$C;m7y)BjMxY17dig54!YUy|^qpXaOK}&uqpF0<+(z6^-6o zA)0QZ#(j0n*LBsAa$?QU z`?2`<H+7$pub#!;yx)?X ze2tBpJO9_O$Ne*#J1)dcey7Sng%T`H4-CXg-G^(f>>7Hml_Vu2gQ>>oY~-rONN8Kl zjYQP)F28_)TySvk*Cd1DyrUy8h=cN8zdjEi#ARft`}+E_pSky;f2;a|99*{ls*Tq; zGk5p&pjIcUS=rgcYU}D)IXU0FIy14*80i~2@;y6AhOPzqH*el#Ihqz^mY=MO?g%|Y zrru{|1r-(+CcIf(U(W|u?)gl9?Ww)r+C5WK4REDpM_-*REekfBwX`k+Ww38;s;=Ee z@V<{vy-Fnbmn&DUoR?x|WYlzW;)Ujtf$jNTRxq-~ZPBmy1x|LCDX&maT;{uPYg@cf z760L;aDG`?G#fj6`$L{rRQ|=fDh9>6!s;3t`LOEy#>Ujf4PHan?^mW86f;#CaNfwz z&yO9m$y-{os`W=VQd<1=BPJ#$zjW!2r>AF@qkjqQ2XX!YJD0N)Pf@p((NDLbP^6wN z#>N)<%Fmylikg};JTf9eOIbOTlut-VVQy|N%RO3NK_U3K?(6hPR#w(C8yi)ANhv9f znI`{gYZ`8o`nO-c-0^}&2>%~e{8YDY@g}Qs36W%UtoJ%YmrP8%S=P$R%9T>uXbYzb zeoQy?;PfEeE{syPCqG}|sUEbj)%WG{-&a@xVX6H#RbKC zI|`M7mA2cKuSedz(J`)G-{$M=7G}Qp_s(U@jy?M(+}PN-=KAZM`}fb+kKgy^Yi@$f$&bL`m64D*VQOl+D5|%= zpW6^P%Xa71ty5{6XU6ZZI}2=9$p{G@0u};x^W#fPO}B5^@&Q=*{{ZfLuX%PxGH!2` z>GrK#ISu!PIJ zF29`7vF2ZCNePFc%MsvK8__;-aq*wP&dZt*ElUfF8Jjn6t}G}BD7p9YWu{%}E0OiF zyQcxe&(hw0{{EVuj~1@^2W(!N>*?t|1J+l3{QS=!AMfv;6`l6_&y1O`U%iUZo8BGQ zqGMP0=LE3aTN9vR0W@=a&do!qsi|={*)LrDQ}^%B$?EU#o|eA8Ru~W5_FQ=_7v6?GT z{OwKT!y2!pllWyU6mF?sxGZyW{i;=6z#_v^MWscQi#0tpwXnK+_f=pRw)NS$A3uIP zQg8Za;3lM;dFQXcO0V4hDgXS=|EEh|=!Y2^um7<846t7W9P-}4uK=VQjx1mV9yi7? zLBgZ~NHGXCvjLAHV{k~i06LhA!GlN20Z1`)Ix+*h77Pl88K7gX7*yDJ6@b*>*Zh Date: Tue, 29 May 2018 00:00:31 +0200 Subject: [PATCH 2/3] Add files via upload --- grove/pyqcl/__init__.py | 15 ++ grove/pyqcl/optimizer.py | 262 ++++++++++++++++++++++++++++++++ grove/pyqcl/qcl.py | 318 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 595 insertions(+) create mode 100644 grove/pyqcl/__init__.py create mode 100644 grove/pyqcl/optimizer.py create mode 100644 grove/pyqcl/qcl.py diff --git a/grove/pyqcl/__init__.py b/grove/pyqcl/__init__.py new file mode 100644 index 0000000..30c0ff9 --- /dev/null +++ b/grove/pyqcl/__init__.py @@ -0,0 +1,15 @@ +############################################################################## +# Copyright 2016-2017 Rigetti Computing +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +############################################################################## \ No newline at end of file diff --git a/grove/pyqcl/optimizer.py b/grove/pyqcl/optimizer.py new file mode 100644 index 0000000..ff874f1 --- /dev/null +++ b/grove/pyqcl/optimizer.py @@ -0,0 +1,262 @@ +############################################################################## +# Copyright 2016-2017 Rigetti Computing +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +############################################################################## +import math +import numpy as np + +import pyquil.api as api + +class OptResults(dict): + """ + Object for holding theta optimization results from QCL. + """ + def __getattr__(self, name): + try: + return self[name] + except KeyError: + raise AttributeError(name) + + __setattr__ = dict.__setitem__ + __delattr__ = dict.__delitem__ + + +class GradientOptimizer(): + """ + Implementation of gradient descent method. + + :param initial_theta: (ndarray) initial parameters for optimization. + :param loss: (string) loss function definition. + :params learning_rate: (float) learning rate for gradient descend method. + Default=1.0 + :params epochs: (int) number of gradient descend epochs. Default=1 + :params batch_size: (int) batch size. Default=None + :params verbose: (bool) Print intermediate results. Default=False + """ + def __init__(self, initial_theta, loss, learning_rate=1.0, epochs=1, + batch_size=None, verbose=False): + + self.loss_mse = 'mean_squared_error' + self.loss_entropy = 'binary_crossentropy' + self.available_losses = [self.loss_mse, self.loss_entropy] + + self.initial_theta = initial_theta + + # Loss function + self.loss = loss + if self.loss not in self.available_losses: + raise ValueError("Available losses are " + self.available_losses) + + # Learning rate + self.learning_rate = learning_rate + if not isinstance(self.learning_rate, float) and not isinstance(self.learning_rate, int): + raise TypeError("learning_rate variable must be a number") + if isinstance(self.learning_rate, int): + self.learning_rate = float(self.learning_rate) + if self.learning_rate <= 0: + raise ValueError("learning_rate variable must be a postive number") + + # Epochs and batch size + self.epochs = epochs + if not isinstance(self.epochs, int): + raise TypeError("epochs variable must be an integer") + if self.epochs <= 0: + raise ValueError("epochs variable must be a postive integer") + self.batch_size = batch_size + if self.batch_size is not None: + if not isinstance(self.batch_size, int): + raise TypeError("batch size variable must be an integer") + if self.batch_size <= 0: + raise ValueError("batch size variable must be a postive integer") + + self.verbose = verbose + + def gradient_descend(self, X, y, state_generators, operator_programs=None, + qvm=None): + """ + Perform theta optimization using gradient descend method. + + :param X: (ndarray) Training data of shape (n_samples,n_features). + :param y: (ndarray) Training labels of shape (n_samples,n_classes) for + classification task and (n_samples,) for regression task. + :param state_generators: (dict) Dictionary with pyQuil programs generating + input state, output state and gradient states. + :param list operator_programs: A list of Programs, each specifying an + operator whose expectation to compute. Default is a list containing + only the empty Program. + :param qvm: (optional, QVM) forest connection object. + + :return (qcl.OptResult()) object :func:`OptResult `. + The following fields are initialized in OptResult: + -theta: set of optimized parameters + -coeff: scalar value of optimized mulitiplicative coefficient. + -history_theta: a list of all intermediate parameter vectors. + -history_loss: a list of all intermediate losses. + -history_grad: a list of all intermediate gradient arrays. + """ + history_theta, history_loss, history_grad = [], [], [] + coeff, theta = 1.0, self.initial_theta + + prog_input_gen = state_generators['input'] + prog_output_gen = state_generators['output'] + prog_output_grad = state_generators['grad'] + + n_samples = len(X) + n_theta = len(theta) + + if qvm is None: + self.qvm = api.QVMConnection() + else: + self.qvm = qvm + + # Check operators + if not isinstance(operator_programs, list): + operator_programs = [operator_programs] + n_operators = len(operator_programs) + + # Check batch size + if self.batch_size is None: + self.batch_size = n_samples + self.batch_size = min(self.batch_size, n_samples) + + # Loop over epochs + for e in range(self.epochs): + + # Loop over batches + batches = self.generate_batches(X, y, self.batch_size) + n_batches = len(batches) + for i, batch in enumerate(batches): + + batch_X, batch_y = batch + n_samples_in_batch = len(batch_X) + + # Predictions + batch_y_pred = np.zeros((n_samples_in_batch, n_operators)) + for k in range(n_samples_in_batch): + prog = prog_input_gen(batch_X[k,:]) + prog += prog_output_gen(theta) + batch_y_pred[k,:] = coeff * np.array(qvm.expectation(prog, operator_programs)) + if self.loss == self.loss_entropy: + batch_y_pred[k,:] = np.exp(batch_y_pred[k,:]) / np.sum(np.exp(batch_y_pred[k,:])) + + # Comput loss + loss_value = self._compute_loss(batch_y, batch_y_pred) + + # Display status + if self.verbose: + print('Epoch: {}/{} ::: Batch: {}/{} ::: Loss: {:.5f}'.format(e+1, self.epochs, i+1, n_batches, loss_value)) + + # Gradient + if not (e == self.epochs - 1 and i == n_batches - 1): + grad = np.zeros((n_samples_in_batch, n_operators, n_theta)) + for k in range(n_samples_in_batch): + + # Define input state + prog_input = prog_input_gen(batch_X[k,:]) + + # Caclulate gradient for each theta_j + for j in range(n_theta): + + # Gradient +/- + for sign in [1,-1]: + grad_sign = np.zeros(n_operators) + grad_progs = prog_output_grad(theta, j, sign) + # Generally, the gradient programs could return + # a program or list of programs (in case the + # gradient +/- is the sum of expectations) + if not isinstance(grad_progs, list): + grad_progs = [grad_progs] + for grad_prog in grad_progs: + prog = prog_input + prog += grad_prog + # B_j +/- expectation + grad_sign += np.array(qvm.expectation(prog, operator_programs)) + # Gradient = (B_j+ - B_j-) / 2 + grad[k, :, j] += sign / 2.0 * grad_sign + + # Gradient update + grad_full = self._compute_grad_full(batch_y, batch_y_pred, grad) + if self.loss == self.loss_mse: + grad_full_coeff = -2.0 * np.mean((batch_y - batch_y_pred) * batch_y_pred) + + # Update theta + theta -= self.learning_rate * grad_full + if self.loss == self.loss_mse: + coeff -= self.learning_rate * grad_full_coeff + + # Append to history + history_loss.append(loss_value) + history_theta.append(theta) + history_grad.append(grad) + + # Prepare results + results = OptResults() + results.theta, results.coeff = theta, coeff + results.loss = loss_value + results.history_loss = history_loss + results.history_theta = history_theta + results.history_grad = history_grad + + return results + + @staticmethod + def generate_batches(X, y, batch_size): + """ + Creates a list of shuffled batches from (X, y) + + :param X: (ndarray) Training data of shape (n_samples,n_features). + :param y: (ndarray) Training labels of shape (n_samples,n_classes) for + classification task and (n_samples,) for regression task. + :param batch_size: (int) batch size + + :return batches: returns a list of tuples (batch_X, batch_y) + """ + m = len(X) + batches = [] + + # Shuffle + permutation = list(np.random.permutation(m)) + shuff_X = X[permutation,:] + shuff_y = y[permutation] + + # Partition + num_complete_batches = math.floor(m/batch_size) + for k in range(0, num_complete_batches): + batch_X = shuff_X[k*batch_size:(k+1)*batch_size, :] + batch_y = shuff_y[k*batch_size:(k+1)*batch_size] + batches.append((batch_X, batch_y)) + + # End case (last mini-batch < mini_batch_size) + if m % batch_size != 0: + batch_X = shuff_X[num_complete_batches*batch_size:m] + batch_y = shuff_y[num_complete_batches*batch_size:m] + batches.append((batch_X, batch_y)) + + return batches + + def _compute_loss(self, y, y_pred): + if self.loss == self.loss_mse: + return np.mean((y - y_pred) ** 2) + elif self.loss == self.loss_entropy: + return -1.0 * np.mean(np.sum(y * np.log(y_pred), axis=1)) + else: + raise ValueError("Available losses are " + self.available_losses) + + def _compute_grad_full(self, y, y_pred, grad): + if self.loss == self.loss_mse: + return -2.0 * np.mean((y - y_pred) * grad[:,0,:], axis=0) + elif self.loss == self.loss_entropy: + return -1.0 * np.mean((grad[:,0,:] - grad[:,1,:]) * (y[:,0,np.newaxis] - y_pred[:,0,np.newaxis]), axis=0) + else: + raise ValueError("Available losses are " + self.available_losses) \ No newline at end of file diff --git a/grove/pyqcl/qcl.py b/grove/pyqcl/qcl.py new file mode 100644 index 0000000..e55b128 --- /dev/null +++ b/grove/pyqcl/qcl.py @@ -0,0 +1,318 @@ +############################################################################## +# Copyright 2016-2017 Rigetti Computing +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +############################################################################## + +import itertools +import numpy as np +from scipy import linalg + +import pyquil.api as api +import pyquil.quil as pq +from pyquil.gates import RX, RY, RZ + +from grove.pyqcl.optimizer import GradientOptimizer + +class QCL(object): + """ + The Quantum Circuit Learning algorithm + + QCL is an object that encapsulates the Quantum Circuit Learning algorithm - + a classical-quantum hybrid algorithm for machine learning on near-term + quantum processors. The main components of the QCL algorithm are options for + gradient descend options performing the loss function minimization, + a unitary encoding input data, a unitary generating output state and + operator programs on which expectation values are measured in the output state. + + Using this object: + + 1) initilze with `inst = QCL(state_generators, operator_programs, **params) + + 2) call `inst.fit(X, y)` to fit QCL to input data + + 3) call `inst.predict(X)` to get predictions of fitted QCL + + :param state_generators: (dict) Dictionary with pyQuil programs generating + input state, output state and gradient states. + :param operator_programs: (list) A list of Programs, each specifying an + operator whose expectation to compute. Default is a list containing + only the empty Program. + :param qvm: (optional, QVM) forest connection object. + :param initial_theta: (ndarray) initial parameters for optimization. + :param loss: (string) loss function definition. + :params learning_rate: (float) learning rate for gradient descend method. + Default=1.0 + :params epochs: (int) number of gradient descend epochs. Default=1 + :params batch_size: (int) batch size. Default=None + :params verbose: (bool) Print intermediate results. Default=False + + """ + + def __init__(self, state_generators, initial_theta, loss, operator_programs=None, + learning_rate=1.0, epochs=1, batch_size=None, qvm=None, verbose=False): + + self.loss_mse = 'mean_squared_error' + self.loss_entropy = 'binary_crossentropy' + + self.initial_theta = initial_theta + self.loss = loss + self.learning_rate = learning_rate + self.epochs = epochs + self.batch_size = batch_size + + self.state_generators = state_generators + if 'input' not in self.state_generators.keys(): + raise ValueError("Provide pyQuil program responsible for " + + "encoding X into quantum state!") + if 'output' not in self.state_generators.keys(): + raise ValueError("Provide pyQuil program responsible for " + + "genering X output state!") + if 'grad' not in self.state_generators.keys(): + raise ValueError("Provide pyQuil program responsible for " + + "calculating a gradient!") + + self.operator_programs = operator_programs + + if qvm is None: + self.qvm = api.QVMConnection() + else: + self.qvm = qvm + + self.verbose = verbose + + self.fit_OK = False + + def fit(self, X, y): + """ + Fit QCL. + + :param X: (ndarray) Training data of shape (n_samples,n_features). + :param y: (ndarray) Training labels of shape (n_samples,n_classes) for + classification task and (n_samples,) for regression task. + + :return self: returns an instance of self. + """ + + X, y = self._make_data(X, y) + + optimizer = GradientOptimizer(self.initial_theta, self.loss, + self.learning_rate, self.epochs, + self.batch_size, self.verbose) + self.results = optimizer.gradient_descend(X, y, self.state_generators, + self.operator_programs, self.qvm) + + self.fit_OK = True + + return self + + def predict(self, X): + """ + Predict QCL. + + :param X: (ndarray) Testing data of shape (n_samples,n_features). + + :return y_pred: (ndarray) Predictions of shape (n_samples,n_classes) for + classification task and (n_samples,) for regression task. + """ + if not self.fit_OK: + raise ValueError("The QCL instance must be fitted before.") + + X = self._make_data(X) + n_samples = len(X) + + # Check operators + if not isinstance(self.operator_programs, list): + operator_programs = [self.operator_programs] + else: + operator_programs = self.operator_programs + n_operators = len(self.operator_programs) + + prog_input_gen = self.state_generators['input'] + prog_output_gen = self.state_generators['output'] + + y_pred = np.zeros((n_samples, n_operators)) + for k in range(n_samples): + prog = prog_input_gen(X[k,:]) + prog += prog_output_gen(self.results.theta) + y_pred[k,:] = self.results.coeff * np.array(self.qvm.expectation(prog, operator_programs)) + if self.loss == self.loss_entropy: + y_pred[k,:] = np.exp(y_pred[k,:]) / np.sum(np.exp(y_pred[k,:])) + + return y_pred + + def get_results(self): + """ + Extract fitting results. + + :return (qcl.OptResult()) object :func:`OptResult `. + The following fields are initialized in OptResult: + -theta: set of optimized parameters + -coeff: scalar value of optimized mulitiplicative coefficient. + -history_theta: a list of all intermediate parameter vectors. + -history_loss: a list of all intermediate losses. + -history_grad: a list of all intermediate gradient arrays. + """ + if not self.fit_OK: + raise ValueError("The QCL instance must be fitted before.") + + return self.results + + def _make_data(self, X, y=None): + if np.ndim(X) == 1: + X = X.reshape(-1,1) + + if y is not None: + if np.ndim(y) == 1: + y = y.reshape(-1,1) + if self.loss == self.loss_entropy and y.shape[1] == 1: + y = np.c_[y, 1-y] + return X, y + else: + return X + +def default_input_state_gen(n_qubits): + """ + Generates Quil program encoding input data in a way described in + https://arxiv.org/abs/1803.00745 + + :param n_qubits: (int) number of qubits in a circuit. + + :return (Program) Quil program for input data encoding + """ + def prog(sample): + p = pq.Program() + n_features = len(sample) + for j in range(n_qubits): + p.inst(RY(np.arcsin(sample[j % n_features]), j)) + p.inst(RZ(np.arccos(sample[j % n_features]**2), j)) + return p + return prog + +def default_output_state_gen(ising_prog, n_qubits, depth): + """ + Generates Quil program preparing output state in a way described in + https://arxiv.org/abs/1803.00745 + + :param ising_prog: (Program) Program evolving system according to fully + connected transverse Ising hamiltionian. + :param n_qubits: (int) number of qubits in a circuit. + :param depth: (int) depth of a circuit. + + :return (Program) Quil program for preparing output state + """ + def prog(theta): + p = pq.Program() + theta = theta.reshape(3,n_qubits,depth) + for i in range(depth): + p += ising_prog + for j in range(n_qubits): + p.inst(RX(theta[0,j,i], j)) + p.inst(RZ(theta[1,j,i], j)) + p.inst(RX(theta[2,j,i], j)) + return p + return prog + +def default_grad_state_gen(ising_prog, n_qubits, depth): + """ + Generates Quil program preparing a state allowing for calculation of + gradients in a way described in https://arxiv.org/abs/1803.00745. Gradients + are created by inserting R(+/- pi/2) rotations in a chain of unitary + transformations. + + :param ising_prog: (Program) Program evolving system according to fully + connected transverse Ising hamiltionian. + :param n_qubits: (int) number of qubits in a circuit. + :param depth: (int) depth of a circuit. + + :return (Program) Quil program for preparing a state allowing for gradient + calculation. + """ + def prog(theta, idx, sign): + theta = theta.reshape(3,n_qubits,depth) + idx = np.unravel_index(idx, theta.shape) + p = pq.Program() + for i in range(depth): + p += ising_prog + for j in range(n_qubits): + p.inst(RX(theta[0,j,i], j)) + if idx == (0,j,i): + p.inst(RX(sign*np.pi/2.0, j)) + p.inst(RZ(theta[1,j,i], j)) + if idx == (1,j,i): + p.inst(RZ(sign*np.pi/2.0, j)) + p.inst(RX(theta[2,j,i], j)) + if idx == (2,j,i): + p.inst(RX(sign*np.pi/2.0, j)) + return p + return prog + +def ising_prog_gen(trotter_steps, T, n_qubits): + """ + Generates Quil program evolving system according to fully connected + transverse Ising hamiltionian. The exponential of a sum of non-commuting + Pauli operators is generated by Trotter Suzuki approximation of first order + (Lie product formula). + Important: The generation of matrix exponential could be done only on QVM + connection. + + :param trotter_steps: (int) trotter steps. + :param T: (int) evolution time. + :param n_qubits: (int) number of qubits in a circuit. + + :return (Program) Quil program evolving system according to fully connected + transverse Ising hamiltionian. + """ + gate_I = np.eye(2) + gate_X = np.array([[0.0,1.0], + [1.0,0.0]]) + gate_Z = np.array([[1.0,0.0], + [0.0,-1.0]]) + + def multi_kron(*args): + ret = np.array([[1.0]]) + for q in args: + ret = np.kron(ret, q) + return ret + + def multi_dot(*args): + for i, q in enumerate(args): + if i == 0: + ret = q + else: + ret = np.dot(ret, q) + return ret + + # Initilize coefficients + h_coeff = np.random.uniform(-1.0, 1.0, size=n_qubits) + J_coeff = dict() + for val in itertools.combinations(range(n_qubits),2): + J_coeff[val] = np.random.uniform(-1.0, 1.0) + + # Unitary + for steps in range(trotter_steps): + + non_inter = [linalg.expm(-(1j)*T/trotter_steps*multi_kron(*[h * gate_X if i == j else gate_I for i in range(n_qubits)])) for j, h in enumerate(h_coeff)] + inter = [linalg.expm(-(1j)*T/trotter_steps*multi_kron(*[J * gate_Z if i == k[0] else gate_Z if i == k[1] else gate_I for i in range(n_qubits)])) for k, J in J_coeff.items()] + ising_step = multi_dot(*non_inter+inter) + + if steps == 0: + ising_gate = ising_step + else: + ising_gate = multi_dot(ising_step, ising_gate) + + ising_name = 'ISING_GATE' + ising_prog = pq.Program().defgate(ising_name, ising_gate) + ising_prog.inst(tuple([ising_name] + list(reversed(range(n_qubits))))) + + return ising_prog \ No newline at end of file From ffb1078fe50d921b7eff6a290128bf2e00d0d727 Mon Sep 17 00:00:00 2001 From: dawidkopczyk <32499114+dawidkopczyk@users.noreply.github.com> Date: Tue, 29 May 2018 00:20:50 +0200 Subject: [PATCH 3/3] Update qcl.rst --- docs/qcl.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/qcl.rst b/docs/qcl.rst index 0fe4f75..5b04af5 100644 --- a/docs/qcl.rst +++ b/docs/qcl.rst @@ -133,6 +133,7 @@ We fit a QCL estimator based on our data and labels, get the results for inspect (Training can take a while as :math:`3*n_qubits*depth*m*3*epochs=12960` expectation values need to be simulated on QVM machine.) .. code:: python + est.fit(X,y) results = est.get_results() @@ -163,6 +164,7 @@ In this example, QCL will try to perform a simple nonlinear classification task. The algorithm structure is very similar to the QCL regression example. We generate a data with sklearn make_circles method: .. code:: python + from sklearn.datasets import make_circles np.random.seed(0) m = 10 @@ -171,6 +173,7 @@ The algorithm structure is very similar to the QCL regression example. We genera Next, we produce a function generating input, output and gradient states. The default methods of QCL can be used. .. code:: python + from grove.pyqcl.qcl import (ising_prog_gen, default_input_state_gen, default_output_state_gen, default_grad_state_gen) @@ -186,6 +189,7 @@ We increase the number of qubits and depth of quantum circuit in comparison to r (Training can take a while as many expectation values need to be simulated on QVM machine.) .. code:: python + initial_theta = np.random.uniform(0.0, 2*np.pi, size=3*n_qubits*depth) operator = [pq.Program(Z(n_qubits-1)), pq.Program(Z(n_qubits-2))] est = QCL(state_generators, initial_theta, loss="binary_crossentropy", @@ -198,6 +202,7 @@ We increase the number of qubits and depth of quantum circuit in comparison to r Now, we can plot the decision surface of a fitted QCL estimator: .. code:: python + import matplotlib.pyplot as plt from matplotlib.colors import ListedColormap cm = plt.cm.RdBu