CODEWITHSUNDEEP
pythonweb-scrapingscrapy
Christian Read
Christian Read

Reputation: 143

How to get HTML code inside Class using Scrapy

Is it possible to get the whole HTML code inside div class messageContent Including HTML code itself?

This is the URL.

But I can't get the whole message and its format is that possible? What I have tried is:

item.css('div.messageContent blockquote::text').extract()

<div class="messageContent">
<article>
<blockquote class="messageText SelectQuoteContainer ugc baseHtml">
Since I accidentally killed the 2018 thread, here&#039;s a cross-post of             
that fateful missive.<br />
<br />
Happy New Year, all! Here&#039;s to 2019, the year we see (my 
predictions...):<br />
<br />
<ul>
<li>The fabled $35k Tesla become reality</li>
<li>The Model Y (give it to me now!)</li>
<li>HW 3 and some minor FSD umbrella features (but definitely not FSD)        
</li>
<li>Tesla getting customer communications under control (where 
&#039;control&#039; indicates at least third-grader aptitude)</li>
<li>Elon doing something incredibly stupid</li>
</ul>What are your predictions?<br />
<br />
Enjoy!<br />
<br />
 <img 
  src="https://teslamotorsclub.com/tmc/attachments/fb_img_1546317769765- 
  jpg.365117/" class="bbCodeImage LbImage" alt="[&#x200B;IMG]" data- 
  url="https://teslamotorsclub.com/tmc/attachments/fb_img_1546317769765- 
  jpg.365117/" />
<div class="messageTextEndMarker">&nbsp;</div>
</blockquote>
</article>
</div>

Upvotes: 0

Views: 37

Answers (1)

eLRuLL
eLRuLL

Reputation: 18799

Yes, you can totally do that. The problem is that you are using ::text, which indicates the selector to extract only the inner text inside the tags.

Use something like this:

item.css('div.messageContent blockquote').extract()

Which will return all the html inside the blockquote tag.

Upvotes: 1

Related Questions