G@d1|dZddlmZddlZddlZdZdZGddZGdd eZ Gd d Z dS) a Main module for computing DAFSA/DAWG graphs from list of strings. The library computes a Deterministic Acyclic Finite State Automata from a list of sequences in a non incremental way, with no plans to expand to incremental computation. The library was originally based on public domain code by `Steve Hanov (2011) `__. Adapted from dafsa/dafsa.py of `DAFSA `_. )CounterNcd}ttt|t|D]}||||krn|dz }|S)a Return the length of the common prefix between two sequences. Parameters ---------- seq_a : iter An iterable holding the first sequence. seq_b : iter An iterable holding the second sequence. Returns ------- length: int The length of the common prefix between `seq_a` and `seq_b`. Examples -------- >>> import dafsa >>> dafsa.utils.common_prefix_length("abcde", "abcDE") 3 >>> dafsa.utils.common_prefix_length("abcde", "ABCDE") 0 r)rangeminlen)seq_aseq_bcommon_prefix_lenis Alib/python3.11/site-packages/spyder/utils/external/dafsa/dafsa.pycommon_prefix_lengthrsc, 3s5zz3u::.. / / 8uQx   EQ cptj|\}}t|dt||S)af Iterate pairwise over an iterable. The function follows the recipe offered on Python's `itertools` documentation. Parameters ---------- iterable : iter The iterable to be iterate pairwise. Examples -------- >>> import dafsa >>> list(dafsa.utils.pairwise([1,2,3,4,5])) [(1, 2), (2, 3), (3, 4), (4, 5)] N) itertoolsteenextzip)iterableelem_aelem_bs r pairwiser<s7 ]8,,NFF vv  rc<eZdZdZdZdZdZdZdZdZ dZ d S) DAFSANodea Class representing node objects in a DAFSA. Each object carries an internal ``node_id`` integer identifier which must be locally unique within a DAFSA, but is meaningless. There is no implicit order nor a sequential progression must be observed. As in previous implementation by Hanov (2011), minimization is performed by comparing nodes, with equivalence determined by the standard Python ``.__eq__()`` method which overloads the equality operator. Nodes are considered identical if they have identical edges, which edges pointing from or to the same node. In particular, edge weight and node finalness, respectively expressed by the ``.weight`` and ``.final`` properties, are *not* considered. This allows to correctly count edges after minimization and to have final pass-through nodes. Parameters ---------- node_id : int The global unique ID for the current node. c>i|_d|_d|_||_dS)z* Initializes a DAFSANode. FrN)edgesfinalweightnode_id)selfrs r __init__zDAFSANode.__init__ks&     rcndfdtjD}|S)a Return a textual representation of the node. The representation lists any edge, with ``id`` and ``attr``ibute. The edge dictionary is sorted at every call, so that, even if more expansive computationally, the function is guaranteed to be idempotent in all implementations. Please note that, as counts and final state are not accounted for, the value returned by this method might be ambiguous, with different nodes returning the same value. For unambigous representation, the ``.__repr__()`` method must be used. .. code:: python >>> from dafsa import DAFSANode, DAFSAEdge >>> node = DAFSANode(0) >>> node.final = True >>> node.edges["x"] = DAFSAEdge(DAFSANode(1), 1) >>> str(node) 'x|1' Returns ------- string : str The (potentially ambiguous) textual representation of the current node. ;cHg|]}d|j|jjfzS)z%s|%irnoder.0labelr s r z%DAFSANode.__str__..s@   5$*U"3"8"@AA   r)joinsortedrr bufs` r __str__zDAFSANode.__str__xsO>hh    #DJ//      rcddfdtjDg}jdkrd|z}njrd|z}nd|z}|S)a Return an unambigous textual representation of the node. The representation lists any edge, with all properties. The edge dictionary is sorted at every call, so that, even if more expansive computationally, the function is guaranteed to be idempotent in all implementations. Please note that, as the return value includes information such as edge weight, it cannot be used for minimization. For such purposes, the potentially ambiguous ``.__str__()`` method must be used. .. code:: python >>> from dafsa import DAFSANode, DAFSAEdge >>> node = DAFSANode(0) >>> node.final = True >>> node.edges["x"] = DAFSAEdge(DAFSANode(1), 1) >>> repr(node) '0(#1/0:/1)' Returns ------- string : str The unambiguous textual representation of the current node. r#|cvg|]5}dj|jjj|j|jfz6S)z#%i/%i:<%s>/%i)rr&rrr's r r*z&DAFSANode.__repr__..sZ   ") Ju-2: K! Ju-4    rrz0(%s)zF(%s)zn(%s))r+r,rrrr-s` r __repr__zDAFSANode.__repr__s:hh    &,DJ%7%7        & <1  C-CC Z C-CCC-C rct|jt|jkrdS|j|jkrdS|jD]A}||jvrdS|j|jj|j|jjkrdSBdS)a Checks whether two nodes are equivalent. Please note that this method checks for *equivalence* (in particular, disregarding edge weight), and not for *equality*. Paremeters ---------- other : DAFSANode The DAFSANode to be compared with the current one. Returns ------- eq : bool A boolean indicating if the two nodes are equivalent. FT)rrrr&r)r otherr)s r __eq__zDAFSANode.__eq__s* tz??c%+.. . .5 : $ $5Z  EEK''uu 5!&.;u%*233uu3 trcV||kS)a@ Return a "greater than" comparison between two nodes. Internally, the method reuses the ``.__str__()`` method, so that the logic for comparison is implemented in a single place. As such, while it guarantees idempotency when sorting nodes, it does not check for properties suc like "node length", "entropy", or "information amount", only providing a convenient complementary method to ``.__eq__()``. Paremeters ---------- other : DAFSANode The DAFSANode to be compared with the current one. Returns ------- gt : bool A boolean indicating if the current node is greater than the one it is compared with (that is, if it should be placed after it in an ordered sequence). )r/)r r5s r __gt__zDAFSANode.__gt__s0||~~ //rcN|S)a@ Return a hash for the node. The returned has is based on the potentially ambigous string representation provided by the ``.__str__()`` method, allowing to use nodes as, among others, dictionary keys. The choice of the potentially ambiguous ``.__str__()`` over ``.__repr__()`` is intentional and by design and complemented by the ``.repr_hash()`` method. Returns ------- hash : number The hash from the (potentially ambigous) textual representation of the current node. r/__hash__r s r r;zDAFSANode.__hash__"||~~&&(((rcN|S)a Return a hash for the node. The returned has is based on the unambigous string representation provided by the ``.__repr__()`` method, allowing to use nodes as, among others, dictionary keys. The method is complemented by the ``.__hash__()`` one. Returns ------- hash : number The hash from the unambigous textual representation of the current node. r3r;r<s r repr_hashzDAFSANode.repr_hash1 }}'')))rN) __name__ __module__ __qualname____doc__r!r/r3r6r8r;r@rr rrRs0   &&&P777r)))V0004)))&*****rrc<eZdZdZdfd ZdZdZdZdZxZ S) DAFSAEdgeaS Class representing edge objects in a DAFSA. This class overloads a normal Python dictionary, and in simpler implementations could potentially be replaced with a pure dictionary. It was implemented as its own object for homogeneity and for planned future expansions, particularly in terms of fuzzy automata. Parameters ---------- node : DAFSANode Reference to the target node, mandatory. Please note that it must be a DAFSANode object and *not* a node id. weight : int Edge weight as collected from training data. Defaults to 0. rctt|tst d||_||_dS)z+ Initializes a DAFSA edge. z=`node` must be a DAFSANode (perhaps a `node_id` was passed?).N)superr! isinstancer TypeErrorr&r)r r&r __class__s r r!zDAFSAEdge.__init__VsX $ ** O   rc.d|jj|jfzS)aG Return a textual representation of the node. The representation only include the ``node_id``, without information on the node actual contents. Returns ------- string : str The (potentially ambiguous) textual representation of the current edge. z{node_id: %i, weight: %i})r&rrr<s r r/zDAFSAEdge.__str__fs+di.?-MMMrc>dt|j|jfzS)a Return a full textual representation of the node. The representation includes information on the entire contents of the node. Returns ------- string : str The unambiguous textual representation of the current edge. z{node: <%s>, weight: %i})reprr&rr<s r r3zDAFSAEdge.__repr__vs*T$)__dk,JJJrcN|S)a@ Return a hash for the edge. The returned has is based on the potentially ambigous string representation provided by the ``.__str__()`` method, allowing to use edges as, among others, dictionary keys. The choice of the potentially ambiguous ``.__str__()`` over ``.__repr__()`` is intentional and by design and complemented by the ``.repr_hash()`` method. Returns ------- hash : number The hash from the (potentially ambigous) textual representation of the current edge. r:r<s r r;zDAFSAEdge.__hash__r=rcN|S)a Return a hash for the edge. The returned has is based on the unambigous string representation provided by the ``.__repr__()`` method, allowing to use edges as, among others, dictionary keys. The method is complemented by the ``.__hash__()`` one. Returns ------- hash : number The hash from the unambigous textual representation of the current edge. r?r<s r r@zDAFSAEdge.repr_hashrAr)r) rBrCrDrEr!r/r3r;r@ __classcell__)rMs@r rHrHDs" NNN K K K)))&*******rrHcVeZdZdZdZdZdZdZdZdZ dd Z d Z d Z d Z d ZdS)DAFSAaL Class representing a DAFSA object. Parameters ---------- sequences : list List of sequences to be added to the DAFSA object. weight : bool Whether to collect edge weights after minimization. Defaults to ``True``. condense: bool Whether to join sequences of transitions into single compound transitions whenever possible. Defaults to ``False``. delimiter : str The delimiter to use in case of joining single path transitions. Defaults to a single white space (`" "`). minimize : bool Whether to minimize the trie into a DAFSA. Defaults to ``True``; this option is implemented for development and testing purposes and it is not intended for users (there are specific and better libraries and algorithms to build tries). c |dd|_|dd}tj|_dt t |ji|_d|_g|_ d|_ t|}t||_ tdg|zD]\}}|||||d||ddr||t#j|j|_|d d r|dSdS) z- Initializes a DAFSA object. delimiter minimizeTrNrcondenseF)get _delimiterrcount_iditerrrnodes lookup_nodes_unchecked_nodes_num_sequencesr,rr_insert_single_seq _minimize_collect_weightscopydeepcopyr[)r sequenceskwargsrY previous_seqseqs r r!zDAFSA.__init__sa!**[#66::j$//!(( 4 #5#5667  !##9%% !)nn"*2$*:!;!; A A L#  # #Cx @ @ @ @ q(### ::h % % -  ! !) , , ,!M$*55 ::j% ( (  MMOOOOO  rczt||}||||js|jd}n|jdd}||dD]Z}t t |j}t||j|<|j |||d|}[d|_ dS)a Internal method for single sequence insertion. Parameters ---------- seq: sequence The sequence being inserted. previous_seq : sequence The previous sequence from the sorted list of sequences, for common prefix length computation. minimize : bool Flag indicating whether to perform minimization or not. rchildN)parenttokenroT) rrerbr`rrr_rHrappendr)r rlrkrY prefix_lenr&rqros r rdzDAFSA._insert_single_seq s(*#|<<  z8,,,$ 6:a=DD(,W5D%  Ed4<0011E )% 0 0DJu   ! ( (%%@@   DD rc d}tt|j|z D]}|j}|d}|d}|d}|s||j|j<Ed} |jD]\} } | |kr| } n| rI|j|jj rd|j| _ |j| |j|_d}||j|j<|sdS)a Internal method for graph minimization. Minimize the graph from the last unchecked item until ``index``. Final minimization, with ``index`` equal to zero, will traverse the entire data structure. The method allows the minimization to be overridden by setting to ``False`` the ``minimize`` flag (returning a trie). Due to the logic in place for the DAFSA minimization, this ends up executed as a non-efficient code, where all comparisons fail, but it is necessary to do it this way to clean the list of unchecked nodes. This is not an implementation problem: this class is not supposed to be used for generating tries (there are more efficient ways of doing that), but it worth having the flag in place for experiments. Parameters ---------- index : int The index until the sequence minimization, right to left. minimize : bool Flag indicating whether to perform minimization or not. TFrprqroN) rrrbpopr`ritemsrr&r) r indexrY graph_changed_unchecked_noderprqro child_idxnode_idxr&s r rezDAFSA._minimize;s7>/ !M3t455=>>' :' :"&!6!:!:!DJy1737:i3H U+0)- 49 5=11! _/ rc: |dkrdS)a Condenses the automaton, merging single-child nodes with their parents. The function joins paths of unique edges into single edges with compound transitions, removing redundant nodes. A redundant node is defined as one that (a) is not final, (b) emits a single transition, (b) receives a single transition, and (d) its source emits a single transition. Internally, the function will call the ``._joining_round()`` method until no more candidates for joining are available. Performing everything in a single step would require a more complex logic. TrN)_joining_roundr<s r r[zDAFSA.condenses(" ""$$)) rc g}jD]\ | fd jDz }td|D}td|D}g}g jD]\ | dkr| dkr jr' fd|Dd fdj djDd}t jd}t fd  Dr z | ||d |D]}j |d |d g}tj|d dj|d j j|d dj|d j j|d dj|<j|d dj|d j|d dt|S)a Internal method for the unique-edge joining algorithm. This function will be called a successive number of times by ``._join_transitions()``, until no more candidates for unique-edge joining are available (as informed by its return value). Returns ------- num_operations: int The number of joining operations that was performed. When zero, it signals that no more joining is possible. cDg|]}j|jjdS)sourcetargetr%)r(r)r& source_ids r r*z(DAFSA._joining_round..s>% 50A0F0NOOrcg|] }|d S)rrFr(edges r r*z(DAFSA._joining_round..<<<<.rrrc,g|]}|dk|SrrF)r(rrs r r*z(DAFSA._joining_round..s'MMM$4>W3L3L3L3L3Lrrcxg|]6}jdj|jjdk4|7Sr)r`rr&r)r(r) edge_infor s r r*z(DAFSA._joining_round..sX:i128?DLX&'''''rrcg|]}|vSrFrF)r(rtransitions_nodess r r*z(DAFSA._joining_round..sNNNG#44NNNr)r label_fromlabel_torrrr)r`rvrrrlistkeysallrrr]r+rHr&rrur)r rsourcestargets transitionsrr transition new_labelrr&rrrs` @@@@@r r~zDAFSA._joining_rounds&#z//11  OIt !Z EE< ><(    JNN:f-h7 8 8 8 8;rc|D]a}|jd}|xjdz c_|D]?}|j|xjdz c_|j|j}|xjdz c_@bdS)a- Internal method for collecting node and edge weights from sequences. This method requires the minimized graph to be already in place. Parameters ---------- sequences : list List of sequences whose node and edge weights will be collected. rrN)r`rrr&)r rirlr&rqs r rfzDAFSA._collect_weightss ! !C:a=D KK1 KK ! ! 5!((A-((z%(- q  ! ! !rFc|jd}d}|D]@}||jvrdS||j|jz }|j|j}|r |jrnA|jsdS||fS)a Check if a sequence can be expressed by the DAFSA. The method does not return all possible potential paths, nor the cumulative weight: if this is needed, the DAFSA object should be converted to a Graph and other libraries, such as ``networkx``, should be used. Parameters ---------- sequence : sequence Sequence to be checked for presence/absence. Returns ------- node : tuple of DAFSANode and int, or None Either a tuple with a DAFSANode referring to the final state that can be reached by following the specified sequence, plus the cumulative weight for reaching it, or None if no path can be found. rN)rarrr&r)r sequencestop_on_prefixr& cum_weightrqs r lookupz DAFSA.lookups0 #   EDJ&&tt $*U+2 2J:e$)D $* z 4Zrc*t|jS)z Return the number of minimized nodes in the structure. Returns ------- node_count : int Number of minimized nodes in the structure. )rr`r<s r count_nodeszDAFSA.count_nodes;s4:rcbtd|jDS)z Return the number of minimized edges in the structure. Returns ------- edge_count : int Number of minimized edges in the structure. c6g|]}t|jSrF)rr)r(r&s r r*z%DAFSA.count_edges..Qs DDDC OODDDr)sumr`valuesr<s r count_edgeszDAFSA.count_edgesGs/DD 0A0A0C0CDDDEEErc|jS)a Return the number of sequences inserted in the structure. Please note that the return value mirrors the number of sequences provided during initialization, and *not* a set of it: repeated sequences are accounted, as each will be added a single time to the object. Returns ------- seq_count : int Number of sequences in the structure. )rcr<s r count_sequenceszDAFSA.count_sequencesSs ""rcfd|||fzg}t|jD]I}|j|}|d|t |d|jDfzgz }Jd|S)z Return a readable multiline textual representation of the object. Returns ------- string : str The textual representation of the object. z8DAFSA with %i nodes and %i edges (%i inserted sequences)z +-- #%i: %s %sc0g|]\}}||jjfSrF)r&r)r(attrns r r*z!DAFSA.__str__..~s%NNNadAFN+NNNr ) rrrr,r`rPrrvr+)r r.rr&s r r/z DAFSA.__str__ds G!!4#3#3#5#5t7K7K7M7MN O dj))  G:g&D "JJNN4:;K;K;M;MNNN CCyy~~rN)F)rBrCrDrEr!rdrer[r~rfrrrrr/rFrr rUrUs.EEEN///bNNN`*W W W r!!!.) ) ) ) V    F F F###"rrU) rE collectionsrrgrrrrdictrHrUrFrr rs    >,o*o*o*o*o*o*o*o*dd*d*d*d*d*d*d*d*NXXXXXXXXXXr