CODEWITHSUNDEEP
dartflutter
sgon00
sgon00

Reputation: 5817

How to delete/detect any emoji as a whole from a string in Flutter?

I would like to simulate a keyboard backspace delete event from a string in Flutter (or Dart). Something like:

String str = "hello🇵🇬你们😀😀👨‍👩‍👦"
myBackspace(str) // will return "hello🇵🇬你们😀😀"
myBackspace(str) // will return "hello🇵🇬你们😀"
myBackspace(str) // will return "hello🇵🇬你们"
myBackspace(str) // will return "hello🇵🇬你"
myBackspace(str) // will return "hello🇵🇬"
myBackspace(str) // will return "hello"
myBackspace(str) // will return "hell"

Upvotes: 14

Views: 9408

Answers (9)

mezoni
mezoni

Reputation: 11220

If try to use information about Unicode mapping sequences, you can get a more or less normal result.

import 'package:sequence_processor/sequence_processor.dart';
import 'package:unicode/decomposers/canonical.dart';
import 'package:unicode/emoji/emoji.dart';

void main(List<String> args) {
  var str = 'hello🇵🇬你们😀😀👨‍👩‍👦';
  print('"$str"');
  while (str.isNotEmpty) {
    str = myBackspace(str);
    print('"$str"');
  }

  str = 'I 💗 you! ❤️‍🔥 ';
  print('"$str"');
  while (str.isNotEmpty) {
    str = myBackspace(str);
    print('"$str"');
  }

  str = 'Amélie';
  print('"$str"');
  while (str.isNotEmpty) {
    str = myBackspace(str);
    print('"$str"');
  }
}

// Should be stored in a static member for performance reasons.
final _processor = () {
  final emojis = getUnicodeEmojiList();
  final processor = SequenceProcessor<int, Object>();
  for (final emoji in emojis) {
    processor.addSequence(emoji.codePoints, emoji);
  }

  const decomposer = CanonicalDecomposer();
  final mappingList = decomposer.getMappingList();
  for (var i = 0; i < mappingList.length; i++) {
    final mapping = mappingList[i];
    final sequence = mapping.$2;
    if (sequence.length > 1) {
      if (!processor.hasSequence(sequence)) {
        processor.addSequence(mapping.$2, mapping.$1);
      }
    }
  }

  return processor;
}();

String myBackspace(String text) {
  if (text.isEmpty) {
    return '';
  }

  final result = _processor.process(text.runes.toList());
  if (result.isEmpty) {
    return '';
  }

  return result
      .take(result.length - 1)
      .map((e) => e.data == null
          ? String.fromCharCode(e.element!)
          : String.fromCharCodes(e.sequence!))
      .join();
}

Output:

"hello🇵🇬你们😀😀👨‍👩‍👦"
"hello🇵🇬你们😀😀"
"hello🇵🇬你们😀"
"hello🇵🇬你们"
"hello🇵🇬你"
"hello🇵🇬"
"hello"
"hell"
"hel"
"he"
"h"
""
"I 💗 you! ❤️‍🔥 "
"I 💗 you! ❤️‍🔥"
"I 💗 you! "
"I 💗 you!"
"I 💗 you"
"I 💗 yo"
"I 💗 y"
"I 💗 "
"I 💗"
"I "
"I"
""
"Amélie"
"Améli"
"Amél"
"Amé"
"Am"
"A"
""

Upvotes: 0

Demaro Create
Demaro Create

Reputation: 11

With RegExp and replaceAll:

 final regex = RegExp(
 "(\u00a9|\u00ae|[\u2000-\u3300]|\ud83c[\ud000-\udfff]|\ud83d[\ud000- 
 \udfff]|\ud83e[\ud000-\udfff])");

 final textReplace = String.replaceAll(regex, '');

Upvotes: 1

ThaiKV
ThaiKV

Reputation: 71

String formatText(String str) {

final RegExp regExp = RegExp(r'(?:[\u2700-\u27bf]|(?:\ud83c[\udde6-\uddff]){2}|[\ud800-\udbff][\udc00-\udfff]|[\u0023-\u0039]\ufe0f?\u20e3|\u3299|\u3297|\u303d|\u3030|\u24c2|\ud83c[\udd70-\udd71]|\ud83c[\udd7e-\udd7f]|\ud83c\udd8e|\ud83c[\udd91-\udd9a]|\ud83c[\udde6-\uddff]|\ud83c[\ude01-\ude02]|\ud83c\ude1a|\ud83c\ude2f|\ud83c[\ude32-\ude3a]|\ud83c[\ude50-\ude51]|\u203c|\u2049|[\u25aa-\u25ab]|\u25b6|\u25c0|[\u25fb-\u25fe]|\u00a9|\u00ae|\u2122|\u2139|\ud83c\udc04|[\u2600-\u26FF]|\u2b05|\u2b06|\u2b07|\u2b1b|\u2b1c|\u2b50|\u2b55|\u231a|\u231b|\u2328|\u23cf|[\u23e9-\u23f3]|[\u23f8-\u23fa]|\ud83c\udccf|\u2934|\u2935|[\u2190-\u21ff])');

if(str.contains(regExp)){
  str = str.replaceAll(regExp,'');
}

return str; }

Ex: Go to https://dartpad.dev/ to test:

String str = "ThaiKV受け行くけどよね😞😞😍😰😒😜" => ThaiKV受け行くけどよね

Upvotes: 7

Bilal Şimşek
Bilal Şimşek

Reputation: 5943

if someone need simple solution to remove emojies from string try this. 

String str = "hello🇵🇬你们😀😀👨‍👩‍👦"İ 
    final RegExp REGEX_EMOJI = RegExp(r'(\u00a9|\u00ae|[\u2000-\u3300]|\ud83c[\ud000-\udfff]|\ud83d[\ud000-\udfff]|\ud83e[\ud000-\udfff])');
    if(str.contains(REGEX_EMOJI)){
       str = str.replaceAll(REGEX_EMOJI,'');
     }

Upvotes: 2

Hugo Passos
Hugo Passos

Reputation: 8447

Update

Dart team released a helper package that helps achieving this. String.characters.skipLast(1) should be able to do what you expect.

Old answer

First, let's get to some definitions. According to this page:

  1. Bytes: 8-bit. The number of bytes that a Unicode string will take up in memory or storage depends on the encoding.
  2. Code Units: The smallest bit combination that can be used to express a single unit in text encoding. For example 1 code unit in UTF-8 would be 1 byte, 2 bytes in UTF-16, 4 bytes in UTF-32.
  3. Code Points [or rune]: Unicode character. A single integer value (from U+0000-U+10FFFF) on a Unicode space.
  4. Grapheme clusters: A single character perceived by the user. 1 grapheme cluster consists of several code points.

When you remove the last char using substring, you're actually removing the last code unit. If you run print(newStr.codeUnits) and print(str.codeUnits), you'll see that the rune 128512 is equivalent to the joint of the code units 55357 and 56832, so 55357 is actually valid, but doesn't represent anything without the "help" of another code unit.

In fact, you don't want to use substring() when there's non-ASCII chars in your String (like emojis or arabic letters). It'll never work. What you have to do is remove the last grapheme cluster. Something as simple as that:

str.graphemeClusters.removeLast()

However, Dart doesn't support this yet. There are several issues around this point. Some of those: https://github.com/dart-lang/language/issues/34
https://github.com/dart-lang/language/issues/49

This lack of support seams to result in some other of issues, like the one you mentioned and this one: https://github.com/flutter/flutter/issues/31818

Upvotes: 13

Kenneth Li
Kenneth Li

Reputation: 1742

This answer still has problem

Since dart does not provide the data type 'Grapheme Cluster', I try to use method channel to do this using kotlin:

Step 1: Create a new 'Flutter Plugin' project, name the project 'gmc01', 2 files will be created automatically, namely 'gmc01.dart' and 'main.dart'.

Step 2: replace the codes in gmc01.dart with the following:

import 'dart:async';

import 'package:flutter/services.dart';

class Gmc01 {
  static const MethodChannel _channel =
      const MethodChannel('gmc01');

  static Future<String> removeLastGMC(String strOriginal) async {
    final String version = await _channel.invokeMethod('removeLastGMC', strOriginal);
    return version;
  }
}

Step 3: Replace the codes in main.dart with the following:

import 'package:gmc01/gmc01.dart';

void main() async {
  String strTemp = '12345678我们5🇵🇬你😀👨‍👩‍👦';
  strTemp = await Gmc01.removeLastGMC(strTemp);
  print(strTemp);
  strTemp = await Gmc01.removeLastGMC(strTemp);
  print(strTemp);
  strTemp = await Gmc01.removeLastGMC(strTemp);
  print(strTemp);
  strTemp = await Gmc01.removeLastGMC(strTemp);
  print(strTemp);
  strTemp = await Gmc01.removeLastGMC(strTemp);
  print(strTemp);
  strTemp = await Gmc01.removeLastGMC(strTemp);
  print(strTemp);
  strTemp = await Gmc01.removeLastGMC(strTemp);
  print(strTemp);
}

Step 4: Inside android/build.gradle, change the minSdkVersion from 16 to 24.

Step 5: Inside example/android/app/build.gradle, change the minSdkVersion from 16 to 24.

Step 6: Click File->Open, select gmc01->android, then click 'OK', the kotlin part of the plugin will be opened (In another Window).

Step 7: Replace the codes in Gmc01Plugin.kt with the following: (Replace the first line with your own package name)

package com.example.gmc01   // replace the left with your own package name

import io.flutter.plugin.common.MethodCall
import io.flutter.plugin.common.MethodChannel
import io.flutter.plugin.common.MethodChannel.MethodCallHandler
import io.flutter.plugin.common.MethodChannel.Result
import io.flutter.plugin.common.PluginRegistry.Registrar
import android.icu.text.BreakIterator



class Gmc01Plugin: MethodCallHandler {
  companion object {
    @JvmStatic
    fun registerWith(registrar: Registrar) {
      val channel = MethodChannel(registrar.messenger(), gmc01)
      channel.setMethodCallHandler(Gmc01Plugin())
    }
  }

  override fun onMethodCall(call: MethodCall, result: Result) {
    var strArg: String
    strArg = call.arguments.toString()
    var boundary = BreakIterator.getWordInstance()
    boundary.setText(strArg);
    when (call.method) {
      removeLastGMC -> {
        result.success(removeLastGMC(boundary, strArg))
      }
      else -> {
        result.notImplemented()
      }
    }
  }

  fun removeLastGMC(boundary: BreakIterator, source: String):String {
    val end = boundary.last()
    val start = boundary.previous()
    return source.substring(0, start)
  }
}

Step 8: Go back to the window of the plugin, and click 'Run'.

Here are the output in the console:

I/flutter (22855): 12345678我们5🇵🇬你😀
I/flutter (22855): 12345678我们5🇵🇬你
I/flutter (22855): 12345678我们5🇵🇬
I/flutter (22855): 12345678我们5
I/flutter (22855): 12345678我们
I/flutter (22855): 12345678
I/flutter (22855):

As you can see, the 'Family Emoji', 'Face Emoji' and 'Country Flag' emoji are removed correctly, but the Chinese 2 chars '我们' and the digits '12345678' are removed by using a single removeLastGMC, so still need to figure out how to distinguish Chinese Double Bytes characters / English Chars / Emojis.

BTW, I don't know how to do the Swift part, can someone help?

Upvotes: 3

sgon00
sgon00

Reputation: 5817

The code is not working

The code is not working properly. I just put here for reference.

Trial 1

  • Problem: can not handle 🇵🇬 and 👨‍👩‍👦 properly.
String myBackspace(String str) {
  Runes strRunes = str.runes;
  str = String.fromCharCodes(strRunes, 0, strRunes.length - 1);
  print(str);
  return str;
}

Trial 2

  • Problem: can not handle connected emoji sequence 😀😀 and 👨‍👩‍👦 properly.

Based on the link

String myBackspace(String str) {
  int i = 0;
  while (str.length > 0) {
    i++;
    int removedCharCode = str.codeUnitAt(str.length - 1);
    if (isWellFormattedUTF16(removedCharCode)) break;
    str = str.substring(0, str.length - 1);
  }
  if (i == 1) str = str.substring(0, str.length - 1);
  print(str);
  return str;
}

bool isWellFormattedUTF16(int charCode) {
  int surrogateLeadingStart = 0xD800;
  int surrogateLeadingEnd = 0xDBFF;
  int surrogateTrailingStart = 0xDC00;
  int surrogateTrailingEnd = 0xDFFF;
  if (!(charCode >= surrogateLeadingStart && charCode <= surrogateLeadingEnd) && 
      !(charCode >= surrogateTrailingStart && charCode <=  surrogateTrailingEnd)) return true;
  return false;
}

Upvotes: 1

Jalil
Jalil

Reputation: 1246

You can do a method like this one

bool isValid(String prevString, String newString){
  if (prevString == newString)
    return true;

  else return false; 
}

then in your keyboard you validate with an onChange property 

TextField(
  onChanged: (text) {
    isValid(varYouHad ,text); //validate
  },
);

Upvotes: 0

Tinus Jackson
Tinus Jackson

Reputation: 3663

Its a bit unclear to what you want to check. I suggest you remove the -1 from the substring because it will break the emoji's code snip

 void main() { 
   var str = "abc😀";
   var newStr = str.substring(0, str.length); // i removed it here
   print(newStr);
   print(newStr.runes);
   print(str.runes);
 }

This will give the output of 

abc😀
(97, 98, 99, 128512)
(97, 98, 99, 128512)

Tested in https://dartpad.dartlang.org/

Upvotes: 1

Related Questions