CODEWITHSUNDEEP
pythonstring
Stan Shunpike
Stan Shunpike

Reputation: 2234

How do I get rid of the b-prefix in a string in python?

I have a string with a b-prefix:

b'I posted a new photo to Facebook'

I gather the b indicates it is a byte string.

How do I remove this b prefix? I tried:

b'I posted a new photo to Facebook'.encode("utf-8").decode("utf-8")

But this gives an error:

UnicodeEncodeError: 'charmap' codec can't encode characters in position 64-65: character maps to <undefined>

Upvotes: 143

Views: 238755

Answers (9)

Fernando D Jaime
Fernando D Jaime

Reputation: 302

On python 3.6 with django 2.0, decode on a byte literal does not work as expected. Yes I get the right result when I print it, but the b'value' is still there even if you print it right.

This is what I'm encoding

uid': urlsafe_base64_encode(force_bytes(user.pk)),

This is what I'm decoding:

uid = force_text(urlsafe_base64_decode(uidb64))

This is what django 2.0 says :

urlsafe_base64_encode(s)[source]

Encodes a bytestring in base64 for use in URLs, stripping any trailing equal signs.

urlsafe_base64_decode(s)[source]

Decodes a base64 encoded string, adding back any trailing equal signs that might have been stripped.

This is my account_activation_email_test.html file

{% autoescape off %}
Hi {{ user.username }},

Please click on the link below to confirm your registration:

http://{{ domain }}{% url 'accounts:activate' uidb64=uid token=token %}
{% endautoescape %}

This is my console response:

Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: Activate Your MySite Account From: webmaster@localhost To: [email protected] Date: Fri, 20 Apr 2018 06:26:46 -0000 Message-ID: <152420560682.16725.4597194169307598579@Dash-U>

Hi testuser,

Please click on the link below to confirm your registration:

http://127.0.0.1:8000/activate/b'MjU'/4vi-fasdtRf2db2989413ba/

as you can see uid = b'MjU'

expected uid = MjU

test in console:

$ python
Python 3.6.4 (default, Apr  7 2018, 00:45:33) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from django.utils.http import urlsafe_base64_encode, urlsafe_base64_decode
>>> from django.utils.encoding import force_bytes, force_text
>>> var1=urlsafe_base64_encode(force_bytes(3))
>>> print(var1)
b'Mw'
>>> print(var1.decode())
Mw
>>>

After investigating it seems like its related to python 3. My workaround was quite simple:

'uid': user.pk,

I receive it as uidb64 on my activate function:

user = User.objects.get(pk=uidb64)

and voila:

Content-Transfer-Encoding: 7bit
Subject: Activate Your MySite Account
From: webmaster@localhost
To: [email protected]
Date: Fri, 20 Apr 2018 20:44:46 -0000
Message-ID: <152425708646.11228.13738465662759110946@Dash-U>


Hi testuser,

Please click on the link below to confirm your registration:

http://127.0.0.1:8000/activate/45/4vi-3895fbb6b74016ad1882/

now it works fine.

Upvotes: 3

Joseph Boyd
Joseph Boyd

Reputation: 588

Assuming you don't want to immediately decode it again like others are suggesting here, you can parse it to a string and then just strip the leading 'b and trailing '.

x = "Hi there 😄" 
x = "Hi there 😄".encode("utf-8") 
x # b"Hi there \xef\xbf\xbd"
str(x)[2:-1]
# "Hi there \\xef\\xbf\\xbd"

Upvotes: 2

salmanwahed
salmanwahed

Reputation: 9647

You need to decode it to convert it to a string. Check the answer here about bytes literal in python3.

b'I posted a new photo to Facebook'.decode('utf-8')
# 'I posted a new photo to Facebook'

Upvotes: 8

Avinash Chougule
Avinash Chougule

Reputation: 71

How to remove b' ' chars which is a decoded string in python:

import base64
a='cm9vdA=='
b=base64.b64decode(a).decode('utf-8')
print(b)

Upvotes: 7

shengli
shengli

Reputation: 61

Alongside with @hiro protagonist answer, you can convert bytes to string by providing characters set into str:

b = b'1234'
str(b,'utf-8') # '1234'

Upvotes: 1

Jonathan Komar
Jonathan Komar

Reputation: 3096

The object you are printing is not a string, but rather a bytes object as a byte literal.

Consider creating a byte object by typing a byte literal (literally defining a byte object without actually using a byte object e.g. by typing b'') and converting it into a string object encoded in utf-8. (Note that converting here means decoding)

byte_object= b"test" # byte object by literally typing characters
print(byte_object) # Prints b'test'
print(byte_object.decode('utf8')) # Prints "test" without quotations

We simply applied the .decode(utf8) function.

String literals are described by the following lexical definitions:

https://docs.python.org/3.3/reference/lexical_analysis.html#string-and-bytes-literals

stringliteral   ::=  [stringprefix](shortstring | longstring)
stringprefix    ::=  "r" | "u" | "R" | "U"
shortstring     ::=  "'" shortstringitem* "'" | '"' shortstringitem* '"'
longstring      ::=  "'''" longstringitem* "'''" | '"""' longstringitem* '"""'
shortstringitem ::=  shortstringchar | stringescapeseq
longstringitem  ::=  longstringchar | stringescapeseq
shortstringchar ::=  <any source character except "\" or newline or the quote>
longstringchar  ::=  <any source character except "\">
stringescapeseq ::=  "\" <any source character>

bytesliteral   ::=  bytesprefix(shortbytes | longbytes)
bytesprefix    ::=  "b" | "B" | "br" | "Br" | "bR" | "BR" | "rb" | "rB" | "Rb" | "RB"
shortbytes     ::=  "'" shortbytesitem* "'" | '"' shortbytesitem* '"'
longbytes      ::=  "'''" longbytesitem* "'''" | '"""' longbytesitem* '"""'
shortbytesitem ::=  shortbyteschar | bytesescapeseq
longbytesitem  ::=  longbyteschar | bytesescapeseq
shortbyteschar ::=  <any ASCII character except "\" or newline or the quote>
longbyteschar  ::=  <any ASCII character except "\">
bytesescapeseq ::=  "\" <any ASCII character>

Upvotes: 27

hiro protagonist
hiro protagonist

Reputation: 46839

decode the bytes to produce a str:

b = b'1234'
print(b.decode('utf-8'))  # '1234'

Upvotes: 232

Kamol Roy
Kamol Roy

Reputation: 27

Although the question is very old, I think it may be helpful to who is facing the same problem. Here the texts is a string like below:

text= "b'I posted a new photo to Facebook'"

Thus you can not remove b by encoding it because it's not a byte. I did the following to remove it.

cleaned_text = text.split("b'")[1]

which will give "I posted a new photo to Facebook"

Upvotes: -2

DevJoe
DevJoe

Reputation: 343

I got it done by only encoding the output using utf-8. Here is the code example

new_tweets = api.GetUserTimeline(screen_name = user,count=200)
result = new_tweets[0]
try: text = result.text
except: text = ''

with open(file_name, 'a', encoding='utf-8') as f:
    writer = csv.writer(f)
    writer.writerows(text)

i.e: do not encode when collecting data from api, encode the output (print or write) only.

Upvotes: 1

Related Questions