c d Z ddlZddlZddlZddlZddlZddlmZ ej e Z ddZ
edk r ej dej ej
ej e dd
Ze d d d
d e ddd
d e dddd
dd e Ze dd ej e
ej ej ej e dej ej d dS dS ) a) This script allows converting word-vectors from word2vec format into Tensorflow 2D tensor and metadata format.
This script used for word-vector visualization on `Embedding Visualization `_.
How to use
----------
#. Convert your word-vector with this script (for example, we'll use model from
`gensim-data `_) ::
python -m gensim.downloader -d glove-wiki-gigaword-50 # download model in word2vec format
python -m gensim.scripts.word2vec2tensor -i ~/gensim-data/glove-wiki-gigaword-50/glove-wiki-gigaword-50.gz -o /tmp/my_model_prefix
#. Open http://projector.tensorflow.org/
#. Click "Load Data" button from the left menu.
#. Select "Choose file" in "Load a TSV file of vectors." and choose "/tmp/my_model_prefix_tensor.tsv" file.
#. Select "Choose file" in "Load a TSV file of metadata." and choose "/tmp/my_model_prefix_metadata.tsv" file.
#. ???
#. PROFIT!
For more information about TensorBoard TSV format please visit:
https://www.tensorflow.org/versions/master/how_tos/embedding_viz/
Command line arguments
----------------------
.. program-output:: python -m gensim.scripts.word2vec2tensor --help
:ellipsis: 0, -7
N)utilsFc . t j j | | }|dz }|dz }t j |d 5 }t j |d 5 }|j D ]}| t j | t j d z d d || D } | t j | t j d z ddd n # 1 swxY w Y ddd n # 1 swxY w Y t d | t d
| dS ) a Convert file in Word2Vec format and writes two files 2D tensor TSV file.
File "tensor_filename"_tensor.tsv contains word-vectors, "tensor_filename"_metadata.tsv contains words.
Parameters
----------
word2vec_model_path : str
Path to file in Word2Vec format.
tensor_filename : str
Prefix for output files.
binary : bool, optional
True if input file in binary format.
)binaryz _tensor.tsvz
_metadata.tsvwb
c 3 4 K | ]}t | V d S )N)str).0xs >lib/python3.11/site-packages/gensim/scripts/word2vec2tensor.py z"word2vec2tensor..K s( "?"?a3q66"?"?"?"?"?"? Nz2D tensor file saved to %sz Tensor metadata file saved to %s) gensimmodels KeyedVectorsload_word2vec_formatr open index_to_keywriteto_utf8joinloggerinfo)
word2vec_model_pathtensor_filenamer model
outfiletsvoutfiletsvmeta file_vector
file_metadataword
vector_rows
r
word2vec2tensorr$ 5 s
M & ; ; > AUAUVZA[A[ [ \ \ \ \ ]] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] KK,j999
KK2NCCCCCs7 E CD=1 E=E EE EEE__main__z6%(asctime)s - %(module)s - %(levelname)s - %(message)s)formatleveliv)formatter_class descriptionz-iz--inputTz%Path to input file in word2vec format)requiredhelpz-oz--outputz Prefix path for output filesz-bz--binary store_constzGSet this flag if word2vec model in binary format (default: %(default)s))actionconstdefaultr+ z
running %s zfinished running %s)F)__doc__ossysloggingargparser r getLogger__name__r r$ basicConfigINFOArgumentParserRawDescriptionHelpFormatterparser add_argument
parse_argsargsr r argvinputoutputr pathbasename r r
rF s @
8 $ $D D D D: z
FG W_f_kllll
$X
$X5Ygnotptotgu
v
v
vF
i$=deee
j4>\]]]
j dE
V D
KK chhsx 0 0111ODJ T[999
KK%rw'7'7 'D'DEEEEE
F
Fr