u*e)dZddlZddlZddlZddlZddlZddlZddlmZddl m Z ej e Z dZejdejZhdZgdZd Zd Zd ejfd Zd ZddiZdZddZdZddZdS)a Given a conda /.cache directory or a .tar.bz2 of that directory, Create a sqlite database with the same data. Intended to be used as a transition between the older format and a newer version of `conda-build index`, or as the foundation of a package database. N)ichunked)commonz (?P[^/]*)/ (?P[^/]*)/ .cache/ (?P(stat.json| (?Precipe|index|icon|about|recipe_log|run_exports|post_install)/(?P\S*?)(?P.\w+$)) )> clones/conda-forgeclones/conda-forge/linux-64"clones/conda-forge/linux-64/.cache'clones/conda-forge/linux-64/.cache/icon(clones/conda-forge/linux-64/.cache/about(clones/conda-forge/linux-64/.cache/index)clones/conda-forge/linux-64/.cache/recipe-clones/conda-forge/linux-64/.cache/recipe_log.clones/conda-forge/linux-64/.cache/run_exports/clones/conda-forge/linux-64/.cache/post_installclones)abouticon index_json post_install recipe_logrecipe run_exportsc|d|d|d|d|d|d|d|d|d |d d S) z: Create schema. Safe to call on every connection. zDCREATE TABLE IF NOT EXISTS about (path TEXT PRIMARY KEY, about BLOB)zNCREATE TABLE IF NOT EXISTS index_json (path TEXT PRIMARY KEY, index_json BLOB)zFCREATE TABLE IF NOT EXISTS recipe (path TEXT PRIMARY KEY, recipe BLOB)zNCREATE TABLE IF NOT EXISTS recipe_log (path TEXT PRIMARY KEY, recipe_log BLOB)zPCREATE TABLE IF NOT EXISTS run_exports (path TEXT PRIMARY KEY, run_exports BLOB)zRCREATE TABLE IF NOT EXISTS post_install (path TEXT PRIMARY KEY, post_install BLOB)zFCREATE TABLE IF NOT EXISTS icon (path TEXT PRIMARY KEY, icon_png BLOB)a;CREATE TABLE IF NOT EXISTS stat ( stage TEXT NOT NULL DEFAULT 'indexed', path TEXT NOT NULL, mtime NUMBER, size INTEGER, sha256 TEXT, md5 TEXT, last_modified TEXT, etag TEXT )z@CREATE UNIQUE INDEX IF NOT EXISTS idx_stat ON stat (path, stage)z?CREATE INDEX IF NOT EXISTS idx_stat_stage ON stat (stage, path)N)execute)conns ?lib/python3.11/site-packages/conda_index/index/convert_cache.pycreater@s  LLWXXX  LLX LLP LLX LLZ LL\ LLP LL     LLSTTTLLRSSSSSc|dd}|tkrtd|dkrdSt ||ddS)zh Call on every connection to ensure we have the correct schema. Call inside a transaction. zPRAGMA user_versionrzEconda-index cache is too new: version {user_version} > {USER_VERSION}NzPRAGMA user_version=1)rfetchone USER_VERSION ValueError remove_prefix)r user_versions rmigrater%us << 566??AA!DLl"" S   a$ LL()))))rrctdd} |dd|dn(#t$r|dd|YnwxYwtdgzD]}|d |d d S) z Store bare filenames in database instead of {channel}/{subdir}/{fn} Could add a view or a virtual column to restore globally-unique paths. Call inside a transaction. zMigrate databasecft|ts|S|ddS)N/) isinstancestrrsplit)paths rbasenamezremove_prefix..basenames0$$$ K{{3##rmigrate_basenamerT)nargfunc deterministic)r0r1statzUPDATE OR IGNORE z7 SET path=migrate_basename(path) WHERE INSTR(path, '/')N)loginfocreate_function TypeError TABLE_NAMESr)rr.tables rr#r#sHH    $$$ H  QXT      HHH /ahGGGGGHx'   ^ ^ ^ ^      s9"AAc#Kt|ds J|dtj|D]\}}}tdtj||D]}tj||}d| tj }t |}|rc t|d5}||fVdddn #1swxYwY#t$r&} td|| Yd} ~ d} ~ wwxYwԐdS)z Yield interesting (match, ) members of filesystem at path. path should be an individual cache directory, e.g. /linux-64/.cache .cachez must end with .cachezCONVERT r(rbNzPermission error: %s %s)r+endswithoswalkr4r5r-r.joinsplitsep PATH_INFOsearchopenPermissionErrorwarn) r-root_filesfilefullpath posixpath path_infoentryes rextract_cache_filesystemrQs t99  h ' 'GGD)G)G)GGGG'$-- E Ea 4BG,,T2244555 E EDw||D$//H!7!788I!((33I EEh--/'....///////////////&EEEHH6!DDDDDDDDE  E E E Es<:D) D D)D! !D)$D! %D)) E3EEindexric|dS)zk Convert match to a database primary key. Retain the option to implement globally unique keys. r.)matchoverride_channels rdb_pathrWs J !!rc tt|tD]\}}td||5|D]\}}|ddkr]|dt j|D] \}}||d<|d|!o|ddkrt |d|d} |d |d |d t|| d #tj$r:} td |d| Yd} ~ 'd} ~ wwxYw|ddkr:|dt|| d utd| dddn #1swxYwYdS)z Convert old style `conda index` cache to sqlite cache. conn: sqlite3 connection cache_generator: extract_cache() or extract_cache_filesystem() override_channel: if channel_name is not in path Commits own transactions. z BEGIN BATCH r-z stat.jsonz&DELETE FROM stat WHERE stage='indexed'z_INSERT OR REPLACE INTO stat (path, mtime, size, stage) VALUES (:path, :mtime, :size, 'indexed')extz.jsonkindz/ INSERT OR IGNORE INTO  (path, zN) VALUES (:path, json(:data)) )r-datazSQL error. Not JSON? %s %srNrz INSERT OR IGNORE INTO icon (path, icon_png) VALUES (:path, :data) z Unhandled %r) enumerater CHUNK_SIZEr4r5rjsonloaditems TABLE_MAPgetrWreadsqlite3OperationalErrorrGgroups groupdict) rcache_generatorichunkrUmemberkeyvaluer9rPs r convert_cacherosh CCDD1@1@5 ###$$$ / @/ @!&. @. @ v=K//LL!IJJJ&*i&7&7&=&=&?&? U(+f  }!5\W,, &MM%-vGGE S /4>C )0(.     #3SSS!=u||APQRRRRRRRRS6]f,,LL %,ENN$*KKMM    HH^U__->->????]. @/ @/ @/ @/ @/ @/ @/ @/ @/ @/ @/ @/ @/ @/ @/ @1@1@s>B%G<.?D.-G<.E7 =/E2 ,G<2E7 7A8G<<H H merged.dbcddlm}tj|}t d|tj|}|5t|dddn #1swxYwY|D]y}tj ||dd}tj |sE|d|d}t dtj |tj |d ||d |ftD]}d d i||} |d kr!d |d| d| d|d| d| d| d| d} |5|| |fdddn #1swxYwYo#t"j$rt d| wxYw|d{dS)z Combine {channel_root}/*/.cache/cache.db databases into a single database. Useful for queries. Not used by any part of the indexing process. r)DEFAULT_SUBDIRSzMerge caches for channel Nr;zcache.dbr(zMerge z as zATTACH DATABASE ? AS subdirricon_pngzINSERT INTO main.r[z ) SELECT ? || path, z FROM subdir.zS WHERE true -- avoid syntax ambiguity ON CONFLICT (path) DO UPDATE SET z = excluded.z WHERE z != excluded.z% -- does this avoid work zOperationalError on %szDETACH DATABASE subdir)conda_index.utilsrrr>r-r.r4r5rconnectrr@existsrelpathdirnamerr8rcrerferror) channel_root output_dbrr channel_name combined_dbsubdircache_dbchannel_prefixr9columnquerys rmerge_index_cacher s  2111117##L11LHH 7 7 7888.++K {!667<< fh KKw~~h''  (446444  cRW__Xrw|/L/LMM c cSa c c    9H;GGG   Ej)--eU;;F%%49.4BH)/E   BB''/@AAABBBBBBBBBBBBBBB+    2E:::  45555966sBA55A9<A9F7F+ F7+F/ /F72F/ 3F77,G#)N)rp)__doc__r_loggingr>os.pathreremore_itertoolsrrr getLogger__name__r4r!compileVERBOSErCTYPICAL_DIRECTORIESr8rr% Connectionr#rQrbr^rWrorrTrrrsn   ######g!! BJ J       2T2T2Tj***, *    :EEE,l #  """"<@<@<@~(6(6(6(6(6(6r