Well the clue is a piece of fabric that isn't to expensive but not too cheap. I think this means it was either a high ranking servant, or concubine.
Amber is suspicious, but I doubt she is a murderer. More likely she told Big Madam that they had plans to flee. Because she was place with the FL as a spy.
I sometimes wonder if it was Nanny, but she would have only done it under orders. She doesn't have a person reason to commit the crime.
It's so early in the show, we don't know a lot about the Xu women. I feel like there are so many more secrets yet to be told.