HR-e\ d Z ddlmZ ddlZddlZddlmZ ddlm Z m
Z
ddl m Z ddl
mZmZ dd lmZmZ erdd
lmZmZ ej e d d
Zd Zej e d d Zej e d d Zej e d d Z ej e d d Z ej e d d Z ej e d d Z ej ed d Z ej e d d Z!ej e d d Z"ej e d d Z#ej e d d Z$d Z%ej e d d Z&d Z'ej e d d Z(ej e d d Z)ej e d d! Z*d" Z+d# Z,d$ Z-d% Z.d& Z/d' Z0ej e d d( Z1dS ))z
This module tests some of the methods related to the ``HTML``
reader/writer and aims to document its functionality.
Requires `BeautifulSoup `_
to be installed.
)StringION)ascii)corehtml)Table)
HAS_BLEACHHAS_BS4 )setup_functionteardown_function)
BeautifulSoupFeatureNotFoundzrequires BeautifulSoup4)reasonc t dd } t j | }t |t sJ t |t j sJ |dk sJ |j | u sJ dS )zB
Test to make sure the class SoupString behaves properly.
z1foo
html.parserN)r
r
SoupString
isinstancestrsoup)r soup_strs @lib/python3.11/site-packages/astropy/io/ascii/tests/test_html.pytest_soupstringr s ;] D t$$H h $ $$$$ h 0 0000 J J J J J =D c g } t j | }t d D ]}| | dD ]}| | | g dk sJ dS )zB
Test to make sure the class ListWriter behaves properly.
abcde)
r r
abcdeN)r
ListWriterrangewrite)lstwriterichs r test_listwriterr, + s
C
_S
!
!F
1XX Q R ::: : : : : : :r c t dd } t j | i d du sJ t j di d du sJ t dd j } t j | i d du sJ t j | i d d u sJ t j | d
did du sJ t j | d
did d u sJ t j | d
d id du sJ t j | d
d id d u sJ dS )
z
Test to make sure that identify_table() returns whether the
given BeautifulSoup tag is the correct table to process.
zr r FNz>r r
Ttable_idbarfoo)r
r identify_tabletable)r s r test_identify_tabler3 ; sO 5} E ED tR + +u 4 4 4 4 tR + +u 4 4 4 4 H tR + +u 4 4 4 4 tR + +t 3 3 3 3 tj!_a 8 8E A A A A tj!_a 8 8D @ @ @ @ tj%%8! < < E E E E tj%%8! < < D D D D D Dr c g d} t j | d }|j du sJ t j |d j ddgk sJ |d j j dk sJ g d} t j | dd g
}|j du sJ t j |d j ddgk sJ |d j j dk sJ d S ) z0
Test reading a table with missing data
)
ascii.htmlformatFATr* )r5 r6 z| ... |
r7 r8 )z...0)r; fill_valuesN)r readmaskednpallmaskdtypekindtable_indats r test_missing_datarI W s H *Xl
3
3
3C :
6#c(-D%=0 1 1111 s8> # % % % % H *Xl
O
O
OC :
6#c(-D%=0 1 1111 s8> # % % % % % %r c D g d} t j | dddg }|j ddgk sJ t | dk sJ t j | dddgdg }|j dgk sJ t | dk sJ t j |d dk sJ d S )
z0
Test reading a table and renaming cols
)r5 z | A | B |
|---|
z | 1 | 2 |
r8 r9 Br< r; namesr
)r; rM
include_namesr N)r r? colnameslenrA rB rF s r test_rename_colsrQ w s
H *Xl3*
E
E
EC 2 | r8 r9 r: col1r r rL N)r r? rO rP rF s r
test_no_namesrT s
UTTH
*Xl
3
3
3C | A |
|---|
z| B |
r9 r. bad_idFr; htmldictguessNz(ERROR: HTML table id 'bad_id' not found$r z%ERROR: HTML table number 3 not found$)pytestraisesr InconsistentTableErrorr r? match)rG errs r test_identify_table_failr_ si 67TUH t2 3 3
s
\Z4JRW
99? @ @@@@ t2 3 3 Ys
8LJ?RWXXXXY Y Y Y Y Y Y Y Y Y Y Y Y Y Y 99< = ======s# A
AA
B11B58B5c dD ]4} t j ddd| id # t $ r
| dk r Y 1w xY wt j t 5 t j ddddid d d d d S # 1 swxY w Y d S )
z
Make sure the user can specify which back-end parser to use
and that an error is raised if the parser is invalid.
)lxmlxmlr html5libdata/html2.htmlr9 parserFrW r r0 N)r r? r rZ r[ )re s r test_backend_parsersrf s =
J!#"F+
&& '&
' '
&
s !55A99A= A=zrequires no BeautifulSoup4c t j } t j t j 5 | g ddd dS # 1 swxY w Y dS )z_
This should return an OptionalTableImportError if BeautifulSoup
is not installed.
N)r HTMLInputterrZ r[ r OptionalTableImportError
process_lines)inputters r test_htmlinputter_no_bs4rl s ""H t4 5 5 # # r"""# # # # # # # # # # # # # # # # # #s AA Ac z d} t | 5 }| }ddd n # 1 swxY w Y t j }i |_ g d}d | | D |k sJ ddi|_ t j t j 5 | | ddd n # 1 swxY w Y d|j d< g d}d | | D |k sJ d
|j d< g d }d | | D |k sJ dS )
z
Test to ensure that HTMLInputter correctly converts input
into a list of SoupStrings representing table elements.
data/html.htmlN)<| Column 1 | Column 2 | Column 3 |
|---|
*| 1 | a | 1.05 |
z*| 2 | b | 2.75 |
+| 3 | c | -1.25 |
c , g | ]}t | S r .0xs r
z%test_htmlinputter.. 6 6 6qCFF 6 6 6r r. r second)<| Column A | Column B | Column C |
|---|
*| 4 | d | 10.5 |
z*| 5 | e | 27.5 |
+| 6 | f | -12.5 |
c , g | ]}t | S rs rt ru s r rx z%test_htmlinputter.. ry r r )*| C1 | C2 | C3 |
|---|
+| 7 | g | 105.0 |
z+| 8 | h | 275.0 |
,| 9 | i | -125.0 |
c , g | ]}t | S rs rt ru s r rx z%test_htmlinputter..
ry r ) openr? r rh get_linesrZ r[ r r\ )ffdr2 rk expecteds r test_htmlinputterr s A
a B ""HHM H 7 6H . .u 5 5 6 6 6( B B B B OHM t2 3 3 " "5!!!" " " " " " " " " " " " " " " !)HM* H 7 6H . .u 5 5 6 6 6( B B B B !"HM* H 7 6H . .u 5 5 6 6 6( B B B B B Bs 377(C
CCc n t j } t j t dd j t j t dd j g}ddgddgg}t | | |k sJ | d t j t 5 t | | d d d n # 1 swxY w Y t j t j 5 t | g d d d d S # 1 swxY w Y d S )
z
Test to make sure that HTMLSplitter correctly inputs lines
of type SoupString to return a generator that gives all
header and data elements.
z4r z6zCol 1zCol 2zData 1zData 2z'| Data 3 | Data 4 |
N) r HTMLSplitterr r
trlistappendrZ r[ TypeErrorr r\ )splitterlines
expected_datas r test_htmlsplitterr
s ""H
F
H-
E w'(H)=>M M 1 1 1 1
LL:;;; y ! ! XXe__
t2 3 3 XXb\\ s$ 6CC "C D**D.1D.c d} t | 5 }| }ddd n # 1 swxY w Y t j }i |_ t j }| | }t
|| | dk sJ d|j d< | | }t
|| | dk sJ d|j d< | | }t
|| | dk sJ t j t d d
j
t j t d d
j g}| | J | d t j t 5 | | ddd dS # 1 swxY w Y dS )
z
Test to ensure that the start_line method of HTMLHeader
returns the first line of header data. Uses t/html.html
for sample input.
rn Nro rz r. r{ r r z%r Text
z| Header |
|---|
)r r? r rh
HTMLHeaderr r
start_liner r
r pr rZ r[ r )r r r2 rk headerr s r test_htmlheader_startr 0 s A
a B ""HHM
_
F u % %E E&##E** +,, I J J J J !)HM* u % %E E&##E** +,, I J J J J !"HM* u % %E E&##E** +,, 7 8 8 8 8
A= Q Q T
m]CCEFF
E U # # + + +
LL+,,, y ! ! ! !% ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !s 377G>>HHc ^ d} t | 5 }| }ddd n # 1 swxY w Y t j }i |_ t j }| | }t
|| | dk sJ t
|| | dz
dk sJ d|j d< | | }t
|| | dk sJ t
|| | dz
d k sJ d
|j d< | | }t
|| | d k sJ t
|| | dz
d k sJ t j t d
d j t j t dd j g}t j t j 5 | | ddd n # 1 swxY w Y | | J | d t j t$ 5 | | ddd n # 1 swxY w Y t j t$ 5 | | ddd dS # 1 swxY w Y dS )z
Test to ensure that the start_line and end_lines methods
of HTMLData returns the first line of table data. Uses
t/html.html for sample input.
rn Nrp r
rq rz r. r| r} r r r z r r z| Data |
)r r? r rh HTMLDatar r r end_liner r
divr rZ r[ r r\ r r )r r r2 rk datar s r
test_htmldatar a s A
a B ""HHM =??D u % %E E$//%(( )** 7 8 8 8 8
E$--&&* +,, 8 9 9 9 9 !)HM* u % %E E$//%(( )** 7 8 8 8 8 E$--&&* +,, 8 9 9 9 9 !"HM* u % %E E$//%(( )** 8 9 9 9 9 E$--&&* +,, 9 : : : :
m]CCGHH
m]CCEFF
E
t2 3 3 == ' ' '
LL)*** y ! ! y ! !
e sB 377I))I-0I-9KK "K ?L""L&)L&c g d} g d}g d}t | ||gd }d}t j | d }|| k sJ dS )
Test to make sure that the HTML writer writes multidimensional
columns (those with iterable elements) using the colspan
attribute of .
r
r r ) ?r ) @r ) @r )r r r r! r! r! r" r" r" C1C2C3rM al
| C1 |
C2 |
C3 |
| 1 |
1.0 |
1.0 |
a |
a |
a |
| 2 |
2.0 |
2.0 |
b |
b |
b |
| 3 |
3.0 |
3.0 |
c |
c |
c |
r Nr r HTMLr' striprS col2col3r2 r outs r test_multicolumn_writer s 99D / / /D > > >D 4t$,> ? ? ?E*HV )++
E
"
"1
%
+
+
-
-C (.."" " " " " " "r zrequires bleachc g d} g d}g d}t | ||gd }d}t j ddi | d
}|| k sJ d S ) r r r ))r r )zr! r! r r r a~
| C1 |
C2 |
C3 |
| 1 |
1.0 |
1.0 |
|
|
a |
| 2 |
2.0 |
2.0 |
|
b |
b |
| 3 |
3.0 |
3.0 |
c |
c |
c |
raw_html_colsr )rX r Nr r s r test_multicolumn_write_escaper s 99D / / /D P P PD 4t$,> ? ? ?E*HV )ot 4
5
5
5
;
;E
B
B1
E
K
K
M
MC (.."" " " " " " "r c g d} g d}g d}t | ||gd }d}t j ddi | d | k sJ d
S ) z
Test to make sure that the HTML writer will not use
multi-dimensional columns if the multicol parameter
is False.
r r r r r a
| C1 |
C2 |
C3 |
| 1 |
1.0 .. 1.0 |
a .. a |
| 2 |
2.0 .. 2.0 |
b .. b |
| 3 |
3.0 .. 3.0 |
c .. c |
multicolFr Nr )rS r r r2 r s r test_write_no_multicolsr s 99D / / /D > > >D 4t$,> ? ? ?E!HD 9j%( ) ) / / 6 6q 9 ? ? A AX^^EUEU U U U U U Ur c t j dd } t j t df }t t j ddgdfddgd fgd
|d fd g
}t j | |k sJ dS )a'
Test to make sure that the HTML reader inputs multidimensional
columns (those with iterable elements) using the colspan
attribute of | .
Ensure that any string element within a multidimensional column
casts all elements to string prior to type conversion operations.
rd r9 r: 1z2.5000000000000000001r 1ag @r< )r )rK z > >Exb ""H
+,a
0D#;2D E4(,7
H 6%8# $ $$$$$$r c t dgdggddg } t }| |dddi d }|| v sJ t }| |dddgi || v sJ t }| |ddddgi d
}|| v sJ d S ) F
Test that columns can contain raw HTML which is not escaped.
z
x
yr r! r r9 r r; rX zL
| x |
<em>y</em> |
z@
| x |
y |
N)r r r' getvalue)tr r s r test_raw_html_writer ` s
~ ~.sCjAAAA **CGGC /EGFFF H
s||~~ % % % % **CGGC #/GGHHH s||~~ % % % % **CGGC #s/LGMMM H
s||~~ % % % % % %r c ddl } t dgdgdggg d }t }| |dd |j i
d }|| v sJ t }| |d|j d t
| j d
gz id
d}|| v sJ dS )r r Nzzy r )r r! r" r r9 r r zv
| <script>x</script> |
<p>y</p> |
y |
tagsr )r raw_html_clean_kwargszj
| <script>x</script> |
y |
y |
)bleachr r r' rO r r ALLOWED_TAGS)r r r r s r test_raw_html_write_cleanr s
MMM
* ~>ooo A
**CGGC /LGMMM H s||~~ % % % % **CGG Z&,d63F.G.G3%.O%P
H s||~~ % % % % % %r c H t } t dgdggd }t j || dd t dgdggd }t }t j ||d | | k sJ d
S ) zG
Test that passing fill_values should replace any matching row
r
r r r! r )r Hello worldr r> r; r r: Nr r r r' r
buffer_outputr
t_expectedbuffer_expecteds r !test_write_table_html_fill_valuesr s JJM
sQCj
+++A K=.B6RRRR !-Z@@@J jjO K
OF;;;; ! ! # #'?'?'A'A A A A A A Ar c H t } t dgdggd }t j || dd t dgdggd }t }t j ||d | | k sJ d S )
zc
Test that passing optional column in fill_values should only replace
matching columns
r
r r )r r r! r r r r: Nr r s r 2test_write_table_html_fill_values_optional_columnsr s
JJM
sQCj
+++A K=.GPVWWWWm_-Z@@@J jjO K
OF;;;; ! ! # #'?'?'A'A A A A A A Ar c t } t dgdggddd }t j j |d< t j || t
j dfd t dgdggd
}t }t j ||d | | k sJ d S )
zi
Test that passing masked values in fill_values should only replace
masked columns or values
r
r T)i4i8)rM r@ rD r TESTr r r r: N)r r rA mar@ r r' r r s r (test_write_table_html_fill_values_maskedr s
JJM
sQCj
4|LLLA
U\AcF K=u|V.DVTTTT1#j999J jjO K
OF;;;; ! ! # #'?'?'A'A A A A A A Ar c t g d} g d}g d}t }t | ||gd }t j ||dd g d} g d}g d }t }t | ||gd }t j ||d
| | k sJ d S ) zy
Test to make sure that the HTML writer writes multidimensional
columns with correctly replaced fill_values.
r r r r r )r zr r ))r r r r r r: Nr )rS r r r r r r s r 'test_multicolumn_table_html_fill_valuesr s
99D / / /D > > >D JJM
tT4 (:;;;A K=jHHHH 99D / / /D > > >D jjOdD)1CDDDJ K
OF;;;; ! ! # #'?'?'A'A A A A A A Ar c t } t g dg dgdd }t j j |d dd< t j j |d dd< t j || t
j d
fgd t g d
g dgd }t }t j ||d t | | | k sJ dS )z
Test that passing masked values in fill_values should only replace
masked columns or values for multidimensional tables
)r
r r r )--r r r! r T)rM r@ r r r r! MASKEDr r )r r r r )r r r r! r r: N) r r rA r r@ r r' printr r s r 5test_multi_column_write_table_html_fill_values_maskedr s
JJM
||| 3 3 34JtTTTA%,AcF1Q3K%,AcF1Q3K K= h/G.HQWXXXX # # #%D%D%DEZ J jjO K
OF;;;; /
"
"
$
$%%% ! ! # #'?'?'A'A A A A A A Ar c j ddg} ddg}ddd}t | |g| }d }t 5 }| |d
| | }d d d n # 1 swxY w Y || k sJ d S )
z^
Test to make sure that the HTML writer writes out using the
supplied formatting.
r
r gz#э%=g ,l04dz.2e)r r r ai
| C1 |
C2 |
| 0001 |
1.23e-11 |
| 0002 |
-9.88e+11 |
r )r; formatsN)r keysr r' r r )rS r r r2 r spr s r "test_write_table_formatted_columnsr s
q6D , 'D%((G 4,gllnn 5 5 5EH4
$r
Bvw 777kkmm!!##$ $ $ $ $ $ $ $ $ $ $ $ $ $ $ (.."" " " " " " "s ?BBBc ~ g d} t j | d }t j |d ddgk sJ dS )z8
Test reading an HTML table with unicode values
)r5 z| Δ | u | Δ | r8 r9 r: rS u ΔN)r r? rA rB rF s r test_read_html_unicoder 2 sR
H *Xl
3
3
3C
6#f+$ - . ......r )2__doc__ior numpyrA rZ
astropy.ior astropy.io.asciir r
astropy.tabler "astropy.utils.compat.optional_depsr r commonr r bs4r
r markskipifr r, r3 rI rQ rT r_ rf rl r r r r r r r r r r r r r r r r r rs r r r s
' ' ' ' ' ' ' ' B B B B B B B B 5 5 5 5 5 5 5 5
322222222 K(ABB ! ! CB !
;
;
; K(ABBE E CBE6 K(ABB & & CB &> K(ABB! ! CB!2 K(ABB CB K(ABB> > CB>$ K(ABB
CB
8 G$@AA# # BA# K(ABB-C -C CB-C` K(ABB CB D K(ABB-! -! CB-!` K(ABB@ @ CB@ F7# 7# 7#t
N+<==7# 7# >=7#t-V -V -V` K(ABB% % CB%*
N+<== & & >= &@
N+<==%& %& >=%&P B B B
B
B
B B B B"B B B0B B B*(# (# (#V K(ABB / / CB / / /r |